2024-06-06T06:37:57.617 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-06T06:37:57.617 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165 --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} --name teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/orig.config.yaml 2024-06-06T06:37:57.672 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T06:37:57.763 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7743165' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5050 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 416/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 3d746ca5346f34eef7e826a1eb71e59e02b682ae syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHhgeuRTc478Db8G9kBoAVHn8tpV9f9oW9MPpv4eqDM/L9VptoCvCj4e2W6oqLoLyACW7miqYvBRHqnpTcjCIdU= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHK82Z/7DYEU15YNScjLH51nUhF98wEjTZpIY+wZ+3pF8FR1N245b2P4DB1gHDuektBLHKsOf5kUZD5T2tqZr4E= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCu7LzIrIraGhaCpFQE7m2egdksCPiPMD4Q8qY9ux++xSb+J1/9mayATu5giAz9pCRQnih4r4t0KGRpB/KXeOGw= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-05_21:24:16 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-06T06:37:57.763 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa; will attempt to use it 2024-06-06T06:37:57.764 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks 2024-06-06T06:37:57.764 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-06T06:37:57.766 INFO:teuthology.task.internal:Checking packages... 2024-06-06T06:37:57.786 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-06T06:37:57.786 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-06T06:37:57.786 INFO:teuthology.packaging:ref: None 2024-06-06T06:37:57.786 INFO:teuthology.packaging:tag: None 2024-06-06T06:37:57.786 INFO:teuthology.packaging:branch: squid 2024-06-06T06:37:57.786 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:37:57.787 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-06T06:37:58.011 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2960.g3d746ca5 2024-06-06T06:37:58.013 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-06T06:37:58.018 INFO:teuthology.task.internal:no buildpackages task found 2024-06-06T06:37:58.018 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-06T06:37:58.024 INFO:teuthology.task.internal:Saving configuration 2024-06-06T06:37:58.043 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-06T06:37:58.049 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-06T06:37:58.074 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 06:33:04.065803', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHhgeuRTc478Db8G9kBoAVHn8tpV9f9oW9MPpv4eqDM/L9VptoCvCj4e2W6oqLoLyACW7miqYvBRHqnpTcjCIdU='} 2024-06-06T06:37:58.095 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 06:33:04.064669', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHK82Z/7DYEU15YNScjLH51nUhF98wEjTZpIY+wZ+3pF8FR1N245b2P4DB1gHDuektBLHKsOf5kUZD5T2tqZr4E='} 2024-06-06T06:37:58.118 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-06 06:33:04.063446', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCu7LzIrIraGhaCpFQE7m2egdksCPiPMD4Q8qY9ux++xSb+J1/9mayATu5giAz9pCRQnih4r4t0KGRpB/KXeOGw='} 2024-06-06T06:37:58.119 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-06T06:37:58.125 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-06T06:37:58.125 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-06T06:37:58.125 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-06T06:37:58.125 INFO:teuthology.run_tasks:Running task console_log... 2024-06-06T06:37:58.213 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb49154fbe0>, signals=[15]) 2024-06-06T06:37:58.213 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-06T06:37:58.218 INFO:teuthology.task.internal:Opening connections... 2024-06-06T06:37:58.219 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T06:37:58.220 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:37:58.294 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-06-06T06:37:58.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:37:58.370 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-06-06T06:37:58.370 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:37:58.446 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-06T06:37:58.452 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-06-06T06:37:58.469 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-06-06T06:37:58.470 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-06-06T06:37:58.526 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T06:37:58.527 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T06:37:58.528 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-06-06T06:37:58.549 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-06-06T06:37:58.566 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-06-06T06:37:58.567 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-06-06T06:37:58.624 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="CentOS Stream" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="9" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:ID="centos" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE="rhel fedora" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="9" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:PLATFORM_ID="platform:el9" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:ANSI_COLOR="0;31" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:LOGO="fedora-logo-icon" 2024-06-06T06:37:58.625 INFO:teuthology.orchestra.run.smithi136.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T06:37:58.626 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://centos.org/" 2024-06-06T06:37:58.626 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T06:37:58.626 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T06:37:58.626 INFO:teuthology.orchestra.run.smithi136.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T06:37:58.626 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-06-06T06:37:58.649 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-06-06T06:37:58.667 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-06-06T06:37:58.667 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-06T06:37:58.724 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-06-06T06:37:58.725 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-06T06:37:58.725 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-06T06:37:58.725 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-06T06:37:58.725 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-06-06T06:37:58.749 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-06T06:37:58.757 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-06T06:37:58.762 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-06T06:37:58.763 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-06-06T06:37:58.765 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-06-06T06:37:58.768 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-06-06T06:37:58.784 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-06T06:37:58.789 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-06T06:37:58.790 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-06-06T06:37:58.822 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-06-06T06:37:58.824 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-06-06T06:37:58.852 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-06T06:37:58.932 INFO:teuthology.run_tasks:Running task kernel... 2024-06-06T06:37:58.946 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-06T06:37:58.947 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-06T06:37:58.947 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T06:37:58.948 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T06:37:58.949 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-06T06:37:58.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:37:58.965 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-06-06T06:37:58.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:37:58.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:37:58.967 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-06-06T06:37:58.967 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-06-06T06:37:59.022 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-437.el9.x86_64 2024-06-06T06:37:59.022 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.14.0-437.el9.x86_64 2024-06-06T06:37:59.022 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2024-06-06T06:37:59.024 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-437.el9.x86_64 2024-06-06T06:37:59.024 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.14.0-437.el9.x86_64 2024-06-06T06:37:59.024 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum install -y kernel 2024-06-06T06:37:59.025 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-437.el9.x86_64 2024-06-06T06:37:59.026 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.14.0-437.el9.x86_64 2024-06-06T06:37:59.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum install -y kernel 2024-06-06T06:38:04.216 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 8.1 MB 00:03 2024-06-06T06:38:10.692 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 836 kB/s | 8.1 MB 00:09 2024-06-06T06:38:13.484 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 11 MB/s | 19 MB 00:01 2024-06-06T06:38:19.095 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 449 kB/s | 8.1 MB 00:18 2024-06-06T06:38:21.062 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-06-06T06:38:29.698 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 803 kB/s | 19 MB 00:24 2024-06-06T06:38:32.241 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 419 kB/s | 6.2 MB 00:15 2024-06-06T06:38:37.381 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 4.2 kB/s | 17 kB 00:03 2024-06-06T06:38:38.634 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 19 MB/s | 21 MB 00:01 2024-06-06T06:38:39.940 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 953 kB/s | 6.2 MB 00:06 2024-06-06T06:38:42.514 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 17 kB 00:01 2024-06-06T06:38:43.733 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-06T06:38:43.756 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-06-06T06:38:45.787 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T06:38:45.787 INFO:teuthology.orchestra.run.smithi136.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T06:38:45.885 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-06-06T06:38:45.894 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: Package Architecture Version Repository Size 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:================================================================================ 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout:Install 4 Packages 2024-06-06T06:38:45.895 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:38:45.896 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 80 M 2024-06-06T06:38:45.896 INFO:teuthology.orchestra.run.smithi136.stdout:Installed size: 124 M 2024-06-06T06:38:45.896 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-06-06T06:38:46.250 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:46.251 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:46.254 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:46.391 INFO:teuthology.orchestra.run.smithi136.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.9 MB/s | 893 kB 00:00 2024-06-06T06:38:46.658 INFO:teuthology.orchestra.run.smithi136.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-06-06T06:38:47.075 INFO:teuthology.orchestra.run.smithi136.stdout:(3/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 38 MB/s | 35 MB 00:00 2024-06-06T06:38:47.375 INFO:teuthology.orchestra.run.smithi136.stdout:(4/4): kernel-modules-core-5.14.0-452.el9.x86_6 29 MB/s | 29 MB 00:00 2024-06-06T06:38:47.377 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:38:47.378 INFO:teuthology.orchestra.run.smithi136.stdout:Total 54 MB/s | 80 MB 00:01 2024-06-06T06:38:48.077 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-06-06T06:38:48.433 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-06-06T06:38:48.433 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-06-06T06:38:48.889 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-06-06T06:38:49.397 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-06-06T06:38:49.397 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-06-06T06:38:50.920 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T06:38:50.921 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T06:38:51.019 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout: Package Architecture Version Repository Size 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T06:38:51.027 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout:================================================================================ 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout:Install 4 Packages 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 80 M 2024-06-06T06:38:51.028 INFO:teuthology.orchestra.run.smithi073.stdout:Installed size: 124 M 2024-06-06T06:38:51.029 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-06-06T06:38:51.254 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:51.255 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:51.261 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:38:51.399 INFO:teuthology.orchestra.run.smithi073.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.9 MB/s | 893 kB 00:00 2024-06-06T06:38:51.658 INFO:teuthology.orchestra.run.smithi073.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 33 MB/s | 16 MB 00:00 2024-06-06T06:38:51.900 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-06-06T06:38:52.033 INFO:teuthology.orchestra.run.smithi073.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 46 MB/s | 29 MB 00:00 2024-06-06T06:38:52.358 INFO:teuthology.orchestra.run.smithi073.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 29 MB/s | 35 MB 00:01 2024-06-06T06:38:52.361 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:38:52.361 INFO:teuthology.orchestra.run.smithi073.stdout:Total 60 MB/s | 80 MB 00:01 2024-06-06T06:38:52.548 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:38:52.643 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:38:53.058 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-06-06T06:38:53.381 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-06-06T06:38:53.382 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-06-06T06:38:54.327 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-06-06T06:38:54.327 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-06-06T06:38:55.308 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:38:55.476 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:38:56.731 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-06-06T06:38:57.378 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:38:57.475 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:38:57.700 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 192 kB/s | 6.2 MB 00:33 2024-06-06T06:38:59.492 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 17 kB 00:00 2024-06-06T06:39:00.121 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:00.220 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:00.269 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:00.331 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:00.670 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-06-06T06:39:05.032 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:05.080 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:05.784 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-06-06T06:39:07.824 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-06T06:39:07.824 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-06T06:39:07.922 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-06-06T06:39:07.930 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-06-06T06:39:07.931 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:39:07.933 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 80 M 2024-06-06T06:39:07.933 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 124 M 2024-06-06T06:39:07.934 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-06-06T06:39:08.049 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] kernel-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:39:08.051 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] kernel-core-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-core-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:39:08.061 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] kernel-modules-5.14.0-452.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/kernel-modules-5.14.0-452.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:39:08.318 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.5 MB/s | 893 kB 00:00 2024-06-06T06:39:09.027 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-06-06T06:39:09.911 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:11.460 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 9.2 MB/s | 29 MB 00:03 2024-06-06T06:39:11.769 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 9.1 MB/s | 35 MB 00:03 2024-06-06T06:39:11.771 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:39:11.772 INFO:teuthology.orchestra.run.smithi170.stdout:Total 21 MB/s | 80 MB 00:03 2024-06-06T06:39:12.469 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-06-06T06:39:12.823 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-06-06T06:39:12.824 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-06-06T06:39:13.799 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-06-06T06:39:13.799 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-06-06T06:39:16.199 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-06-06T06:39:16.851 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:39:16.945 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:19.571 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:19.717 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:24.456 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:24.505 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:29.259 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:34.633 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:34.673 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:35.051 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:35.051 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:39:35.052 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:35.052 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:39:35.876 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-06-06T06:39:36.259 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:39:36.346 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:36.347 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:39:36.347 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T06:39:36.347 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T06:39:36.347 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T06:39:36.347 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-06T06:39:36.347 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T06:39:36.347 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:39:36.399 INFO:teuthology.orchestra.run.smithi136.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:36.399 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:39:36.400 DEBUG:teuthology.orchestra.run.smithi136:> sudo rpm -qi grub2-tools 2024-06-06T06:39:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:Name : grub2-tools 2024-06-06T06:39:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:Epoch : 1 2024-06-06T06:39:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:Version : 2.06 2024-06-06T06:39:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:Release : 68.el9 2024-06-06T06:39:36.430 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: x86_64 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Group : Unspecified 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Size : 8264254 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:License : GPLv3+ 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Packager : builder@centos.org 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Vendor : CentOS 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Summary : Support tools for GRUB. 2024-06-06T06:39:36.431 INFO:teuthology.orchestra.run.smithi136.stdout:Description : 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout:hardware devices. 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:39:36.432 INFO:teuthology.orchestra.run.smithi136.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T06:39:36.433 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T06:39:36.433 INFO:teuthology.task.kernel:Updating grub on smithi136 to boot 5.14.0-452.el9.x86_64 2024-06-06T06:39:36.433 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T06:39:37.052 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-06-06T06:39:38.459 INFO:teuthology.orchestra.run.smithi136.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T06:39:38.490 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-06-06T06:39:38.496 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-06-06T06:39:38.513 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.XgiljHD3Bj 2024-06-06T06:39:38.513 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XgiljHD3Bj 2024-06-06T06:39:38.582 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.XgiljHD3Bj 2024-06-06T06:39:38.716 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.XgiljHD3Bj is 6KB 2024-06-06T06:39:38.764 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.XgiljHD3Bj 2024-06-06T06:39:38.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T06:39:38.840 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T06:39:38.840 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T06:39:38.840 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T06:39:38.840 INFO:teuthology.orchestra.run.smithi136.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T06:39:38.841 DEBUG:teuthology.orchestra.run.smithi136:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T06:39:38.982 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-06-06T06:39:39.534 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:39.572 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:39.958 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:39.958 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:39:39.958 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:39.959 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:40.831 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:39:40.832 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-06-06T06:39:41.219 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:39:41.297 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:41.297 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:39:41.297 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T06:39:41.298 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T06:39:41.298 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T06:39:41.298 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-06T06:39:41.298 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T06:39:41.298 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:39:41.350 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:41.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:39:41.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -qi grub2-tools 2024-06-06T06:39:41.392 INFO:teuthology.orchestra.run.smithi073.stdout:Name : grub2-tools 2024-06-06T06:39:41.392 INFO:teuthology.orchestra.run.smithi073.stdout:Epoch : 1 2024-06-06T06:39:41.392 INFO:teuthology.orchestra.run.smithi073.stdout:Version : 2.06 2024-06-06T06:39:41.392 INFO:teuthology.orchestra.run.smithi073.stdout:Release : 68.el9 2024-06-06T06:39:41.393 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: x86_64 2024-06-06T06:39:41.393 INFO:teuthology.orchestra.run.smithi073.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T06:39:41.393 INFO:teuthology.orchestra.run.smithi073.stdout:Group : Unspecified 2024-06-06T06:39:41.393 INFO:teuthology.orchestra.run.smithi073.stdout:Size : 8264254 2024-06-06T06:39:41.393 INFO:teuthology.orchestra.run.smithi073.stdout:License : GPLv3+ 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Packager : builder@centos.org 2024-06-06T06:39:41.394 INFO:teuthology.orchestra.run.smithi073.stdout:Vendor : CentOS 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout:Summary : Support tools for GRUB. 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout:Description : 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T06:39:41.395 INFO:teuthology.orchestra.run.smithi073.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T06:39:41.396 INFO:teuthology.orchestra.run.smithi073.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T06:39:41.396 INFO:teuthology.orchestra.run.smithi073.stdout:hardware devices. 2024-06-06T06:39:41.396 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:39:41.396 INFO:teuthology.orchestra.run.smithi073.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T06:39:41.398 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T06:39:41.398 INFO:teuthology.task.kernel:Updating grub on smithi073 to boot 5.14.0-452.el9.x86_64 2024-06-06T06:39:41.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T06:39:42.017 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-06-06T06:39:43.397 INFO:teuthology.orchestra.run.smithi073.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T06:39:43.432 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-06-06T06:39:43.435 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-06-06T06:39:43.455 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.ArCUH1HEIq 2024-06-06T06:39:43.455 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ArCUH1HEIq 2024-06-06T06:39:43.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.ArCUH1HEIq 2024-06-06T06:39:43.660 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.ArCUH1HEIq is 6KB 2024-06-06T06:39:43.709 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.ArCUH1HEIq 2024-06-06T06:39:43.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T06:39:43.785 INFO:teuthology.orchestra.run.smithi073.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T06:39:43.786 INFO:teuthology.orchestra.run.smithi073.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T06:39:43.786 INFO:teuthology.orchestra.run.smithi073.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T06:39:43.786 INFO:teuthology.orchestra.run.smithi073.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T06:39:43.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T06:39:43.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-06-06T06:39:58.682 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:58.722 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:59.080 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:59.080 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-06T06:39:59.080 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-06T06:39:59.080 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-06T06:39:59.898 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:39:59.899 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-06-06T06:40:00.259 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:40:00.351 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:40:00.351 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:40:00.351 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-06T06:40:00.351 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-06T06:40:00.351 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-06T06:40:00.352 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-06-06T06:40:00.352 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-06-06T06:40:00.352 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-06-06T06:40:00.405 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-06T06:40:00.405 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-06T06:40:00.405 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 68.el9 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8264254 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-06T06:40:00.436 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:40:00.437 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2024-06-06T06:40:00.438 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-06T06:40:00.438 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 5.14.0-452.el9.x86_64 2024-06-06T06:40:00.438 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-06T06:40:01.036 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-06-06T06:40:02.448 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-06T06:40:02.476 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-06-06T06:40:02.479 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-06-06T06:40:02.496 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.i8tOIPkw90 2024-06-06T06:40:02.496 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.i8tOIPkw90 2024-06-06T06:40:02.561 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.i8tOIPkw90 2024-06-06T06:40:02.698 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.i8tOIPkw90 is 6KB 2024-06-06T06:40:02.746 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.i8tOIPkw90 2024-06-06T06:40:02.761 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2024-06-06T06:40:02.822 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-06T06:40:02.822 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-06T06:40:02.822 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-06T06:40:02.822 INFO:teuthology.orchestra.run.smithi170.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-06T06:40:02.823 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-06T06:40:02.973 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-06-06T06:40:08.990 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-06T06:40:08.990 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:40:08.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:13.940 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-06T06:40:13.940 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:40:13.941 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:27.422 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-06-06T06:40:32.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-06-06T06:40:32.976 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-06-06T06:40:32.977 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:40:32.977 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:36.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:40:36.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:39.486 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-06-06T06:40:41.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:40:41.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:44.354 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-06-06T06:40:51.488 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:40:51.489 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:40:51.490 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-06-06T06:40:56.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:40:56.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:41:00.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:41:00.496 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:41:03.554 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-06-06T06:41:15.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:41:15.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:41:23.272 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-06-06T06:41:23.717 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:41:23.717 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-452.el9.x86_64"... 2024-06-06T06:41:23.718 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-06-06T06:41:23.776 INFO:teuthology.orchestra.run.smithi136.stdout:5.14.0-452.el9.x86_64 2024-06-06T06:41:23.776 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T06:41:23.776 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T06:41:23.776 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T06:41:24.778 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-06T06:41:24.778 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T06:41:24.938 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-06-06T06:41:24.978 DEBUG:teuthology.parallel:result is None 2024-06-06T06:41:28.232 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-06-06T06:41:28.595 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:41:28.595 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-452.el9.x86_64"... 2024-06-06T06:41:28.595 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-06-06T06:41:28.653 INFO:teuthology.orchestra.run.smithi073.stdout:5.14.0-452.el9.x86_64 2024-06-06T06:41:28.654 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T06:41:28.654 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T06:41:28.654 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T06:41:29.654 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-06T06:41:29.655 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T06:41:29.779 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-06-06T06:41:29.826 DEBUG:teuthology.parallel:result is None 2024-06-06T06:41:47.458 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-06-06T06:41:48.199 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:41:48.200 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-452.el9.x86_64"... 2024-06-06T06:41:48.200 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-06-06T06:41:48.214 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-452.el9.x86_64 2024-06-06T06:41:48.214 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-06T06:41:48.214 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-06T06:41:48.215 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-06T06:41:49.215 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-06-06T06:41:49.216 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-06T06:41:49.280 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-06-06T06:41:49.315 DEBUG:teuthology.parallel:result is None 2024-06-06T06:41:49.315 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-06T06:41:49.323 INFO:teuthology.task.internal:Creating test directory... 2024-06-06T06:41:49.323 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T06:41:49.326 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T06:41:49.329 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-06T06:41:49.346 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-06T06:41:49.352 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-06T06:41:49.362 INFO:teuthology.task.internal:Creating archive directory... 2024-06-06T06:41:49.362 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T06:41:49.385 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T06:41:49.389 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-06T06:41:49.502 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-06T06:41:49.508 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-06T06:41:49.508 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T06:41:49.511 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T06:41:49.513 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-06T06:41:49.544 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.548 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.556 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.558 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.558 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.571 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-06T06:41:49.573 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-06T06:41:49.580 INFO:teuthology.task.internal:Configuring sudo... 2024-06-06T06:41:49.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T06:41:49.583 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T06:41:49.600 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-06T06:41:49.625 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-06T06:41:49.634 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-06T06:41:49.635 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T06:41:49.653 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T06:41:49.668 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-06T06:41:49.685 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:49.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:49.809 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:49.896 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:49.940 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:41:49.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T06:41:50.010 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:50.056 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:50.108 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:50.194 DEBUG:teuthology.orchestra.run.smithi136:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:50.235 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:41:50.235 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T06:41:50.302 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:50.346 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-06T06:41:50.400 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:50.483 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-06T06:41:50.524 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:41:50.525 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-06T06:41:50.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-06-06T06:41:50.592 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-06-06T06:41:50.595 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-06-06T06:41:50.640 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T06:41:50.651 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T06:41:50.672 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T06:41:51.188 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-06T06:41:51.197 INFO:teuthology.task.internal:Starting timer... 2024-06-06T06:41:51.197 INFO:teuthology.run_tasks:Running task pcp... 2024-06-06T06:41:51.210 INFO:teuthology.run_tasks:Running task selinux... 2024-06-06T06:41:51.219 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-06T06:41:51.220 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-06-06T06:41:51.295 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-06-06T06:41:51.298 DEBUG:teuthology.orchestra.run.smithi136:> sudo service auditd rotate 2024-06-06T06:41:51.373 INFO:teuthology.orchestra.run.smithi136.stdout:Rotating logs: 2024-06-06T06:41:51.375 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-06-06T06:41:51.436 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-06-06T06:41:51.438 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-06T06:41:51.438 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-06-06T06:41:51.483 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-06-06T06:41:51.483 DEBUG:teuthology.orchestra.run.smithi136:> /usr/sbin/getenforce 2024-06-06T06:41:51.514 INFO:teuthology.orchestra.run.smithi136.stdout:Permissive 2024-06-06T06:41:51.515 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-06-06T06:41:51.541 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-06-06T06:41:51.542 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi136.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-06-06T06:41:51.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T06:41:51.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:41:51.574 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T06:41:51.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:41:51.603 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T06:41:51.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T06:41:51.632 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-06T06:41:51.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-06-06T06:41:51.663 DEBUG:teuthology.orchestra.run.smithi136:> sudo /usr/sbin/setenforce permissive 2024-06-06T06:41:51.691 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-06-06T06:41:51.719 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-06T06:41:51.728 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-06T06:41:51.741 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-06T06:41:51.742 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-06T06:46:12.678 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-06-06T06:46:12.680 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:46:12.681 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:46:12.764 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-06-06T06:46:12.859 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-06-06T06:46:12.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:46:12.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:46:12.949 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-06-06T06:46:13.042 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-06-06T06:46:13.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:46:13.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-06T06:46:13.120 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-06-06T06:46:13.203 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-06-06T06:46:13.204 INFO:teuthology.run_tasks:Running task clock... 2024-06-06T06:46:13.216 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-06T06:46:13.216 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T06:46:13.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T06:46:13.219 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T06:46:13.219 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T06:46:13.221 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-06T06:46:13.221 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T06:46:13.247 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T06:46:13.248 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T06:46:13.262 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T06:46:13.265 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T06:46:13.275 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-06T06:46:13.286 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-06-06T06:46:13.287 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: ntpd: command not found 2024-06-06T06:46:13.290 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-06T06:46:13.297 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-06-06T06:46:13.298 INFO:teuthology.orchestra.run.smithi136.stdout:506 Cannot talk to daemon 2024-06-06T06:46:13.311 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T06:46:13.312 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T06:46:13.314 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-06-06T06:46:13.325 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-06-06T06:46:13.325 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T06:46:13.326 INFO:teuthology.orchestra.run.smithi136.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T06:46:13.338 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-06T06:46:13.352 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-06T06:46:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-06-06T06:46:13.385 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T06:46:13.385 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-06-06T06:46:13.385 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.385 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.386 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.386 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.392 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-06-06T06:46:13.395 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T06:46:13.395 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-06-06T06:46:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.396 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.414 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-06-06T06:46:13.417 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T06:46:13.417 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-06-06T06:46:13.417 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.417 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.417 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.418 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T06:46:13.418 INFO:teuthology.run_tasks:Running task install... 2024-06-06T06:46:13.426 DEBUG:teuthology.task.install:project ceph 2024-06-06T06:46:13.427 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-06T06:46:13.427 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-06T06:46:13.427 INFO:teuthology.task.install:Using flavor: default 2024-06-06T06:46:13.432 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-06T06:46:13.432 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-06T06:46:13.433 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T06:46:13.433 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:46:13.435 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T06:46:13.435 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:46:13.436 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-06T06:46:13.436 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:46:13.587 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T06:46:13.588 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T06:46:13.649 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T06:46:13.649 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T06:46:13.664 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-06T06:46:13.664 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-06T06:46:13.962 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T06:46:13.962 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:46:13.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T06:46:13.981 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T06:46:13.981 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:46:13.981 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T06:46:13.988 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T06:46:13.988 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T06:46:13.996 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-06T06:46:13.996 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:46:13.996 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-06T06:46:14.007 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T06:46:14.007 DEBUG:teuthology.orchestra.run.smithi170:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T06:46:14.023 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-06T06:46:14.023 DEBUG:teuthology.orchestra.run.smithi136:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-06T06:46:14.055 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T06:46:14.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T06:46:14.089 DEBUG:teuthology.orchestra.run.smithi136:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-06T06:46:14.130 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T06:46:14.147 DEBUG:teuthology.orchestra.run.smithi170:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T06:46:14.164 DEBUG:teuthology.orchestra.run.smithi136:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-06T06:46:14.195 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-06-06T06:46:14.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-06-06T06:46:14.211 INFO:teuthology.orchestra.run.smithi170.stdout:check_obsoletes = 1 2024-06-06T06:46:14.213 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum clean all 2024-06-06T06:46:14.227 INFO:teuthology.orchestra.run.smithi136.stdout:check_obsoletes = 1 2024-06-06T06:46:14.229 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum clean all 2024-06-06T06:46:14.430 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-06-06T06:46:14.446 INFO:teuthology.orchestra.run.smithi170.stdout:45 files removed 2024-06-06T06:46:14.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T06:46:14.469 INFO:teuthology.orchestra.run.smithi136.stdout:45 files removed 2024-06-06T06:46:14.473 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T06:46:14.497 DEBUG:teuthology.orchestra.run.smithi136:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-06T06:46:15.662 INFO:teuthology.orchestra.run.smithi136.stdout:ceph packages for x86_64 93 kB/s | 83 kB 00:00 2024-06-06T06:46:15.886 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 72 kB/s | 83 kB 00:01 2024-06-06T06:46:15.953 INFO:teuthology.orchestra.run.smithi170.stdout:ceph packages for x86_64 69 kB/s | 83 kB 00:01 2024-06-06T06:46:16.454 INFO:teuthology.orchestra.run.smithi136.stdout:ceph noarch packages 16 kB/s | 11 kB 00:00 2024-06-06T06:46:16.626 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-06-06T06:46:16.670 INFO:teuthology.orchestra.run.smithi170.stdout:ceph noarch packages 17 kB/s | 11 kB 00:00 2024-06-06T06:46:17.191 INFO:teuthology.orchestra.run.smithi136.stdout:ceph source packages 2.7 kB/s | 1.9 kB 00:00 2024-06-06T06:46:17.395 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-06-06T06:46:17.437 INFO:teuthology.orchestra.run.smithi170.stdout:ceph source packages 2.6 kB/s | 1.9 kB 00:00 2024-06-06T06:46:17.740 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-06-06T06:46:18.030 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-06T06:46:18.072 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.1 MB 00:00 2024-06-06T06:46:19.716 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-06-06T06:46:19.840 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-06-06T06:46:20.040 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-06-06T06:46:23.858 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-06-06T06:46:24.132 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 8.5 MB/s | 6.2 MB 00:00 2024-06-06T06:46:24.133 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-06-06T06:46:25.583 INFO:teuthology.orchestra.run.smithi136.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 17 kB 00:00 2024-06-06T06:46:25.765 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 17 kB 00:00 2024-06-06T06:46:25.783 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 17 kB 00:00 2024-06-06T06:46:27.236 INFO:teuthology.orchestra.run.smithi136.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-06-06T06:46:27.478 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-06-06T06:46:27.710 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-06-06T06:46:32.358 INFO:teuthology.orchestra.run.smithi136.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-06T06:46:32.700 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2024-06-06T06:46:32.858 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 17 kB/s | 1.7 kB 00:00 2024-06-06T06:46:34.734 INFO:teuthology.orchestra.run.smithi136.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:34.735 INFO:teuthology.orchestra.run.smithi136.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:34.746 INFO:teuthology.orchestra.run.smithi136.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T06:46:34.747 INFO:teuthology.orchestra.run.smithi136.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T06:46:34.747 INFO:teuthology.orchestra.run.smithi136.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T06:46:34.748 INFO:teuthology.orchestra.run.smithi136.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T06:46:34.750 INFO:teuthology.orchestra.run.smithi136.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T06:46:34.752 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T06:46:34.753 INFO:teuthology.orchestra.run.smithi136.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T06:46:34.815 INFO:teuthology.orchestra.run.smithi136.stdout:Dependencies resolved. 2024-06-06T06:46:34.836 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: Package Arch Version Repository Size 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout:Installing: 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T06:46:34.837 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T06:46:34.838 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T06:46:34.839 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrading: 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T06:46:34.840 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout:Installing dependencies: 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T06:46:34.841 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T06:46:34.842 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T06:46:34.843 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T06:46:34.844 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T06:46:34.845 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T06:46:34.846 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T06:46:34.847 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T06:46:34.848 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T06:46:34.849 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:Installing weak dependencies: 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction Summary 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:======================================================================================= 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:Install 148 Packages 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout:Upgrade 11 Packages 2024-06-06T06:46:34.850 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:46:34.851 INFO:teuthology.orchestra.run.smithi136.stdout:Total download size: 225 M 2024-06-06T06:46:34.851 INFO:teuthology.orchestra.run.smithi136.stdout:Downloading Packages: 2024-06-06T06:46:35.044 INFO:teuthology.orchestra.run.smithi170.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.044 INFO:teuthology.orchestra.run.smithi170.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.056 INFO:teuthology.orchestra.run.smithi170.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T06:46:35.057 INFO:teuthology.orchestra.run.smithi170.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.057 INFO:teuthology.orchestra.run.smithi170.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T06:46:35.058 INFO:teuthology.orchestra.run.smithi170.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T06:46:35.060 INFO:teuthology.orchestra.run.smithi170.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T06:46:35.062 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T06:46:35.063 INFO:teuthology.orchestra.run.smithi170.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T06:46:35.126 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T06:46:35.144 INFO:teuthology.orchestra.run.smithi170.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T06:46:35.145 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T06:46:35.146 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrading: 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T06:46:35.147 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T06:46:35.148 INFO:teuthology.orchestra.run.smithi170.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T06:46:35.149 INFO:teuthology.orchestra.run.smithi170.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T06:46:35.150 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T06:46:35.151 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T06:46:35.152 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T06:46:35.153 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T06:46:35.154 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T06:46:35.155 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T06:46:35.156 INFO:teuthology.orchestra.run.smithi170.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout:Installing weak dependencies: 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout:======================================================================================= 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout:Install 148 Packages 2024-06-06T06:46:35.157 INFO:teuthology.orchestra.run.smithi170.stdout:Upgrade 11 Packages 2024-06-06T06:46:35.158 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:46:35.158 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 225 M 2024-06-06T06:46:35.158 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-06-06T06:46:35.234 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.235 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.246 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-06T06:46:35.247 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-06T06:46:35.248 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-06T06:46:35.248 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-06T06:46:35.250 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-06T06:46:35.252 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-06T06:46:35.253 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-06T06:46:35.315 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-06T06:46:35.336 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-06T06:46:35.337 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-06T06:46:35.338 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-06T06:46:35.339 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-06T06:46:35.340 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-06T06:46:35.341 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-06T06:46:35.342 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-06T06:46:35.343 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-06T06:46:35.344 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-06T06:46:35.345 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-06T06:46:35.346 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-06T06:46:35.347 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-06T06:46:35.348 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout:======================================================================================= 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout:Install 148 Packages 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 11 Packages 2024-06-06T06:46:35.349 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:46:35.350 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 225 M 2024-06-06T06:46:35.350 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-06-06T06:46:35.627 INFO:teuthology.orchestra.run.smithi136.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-06-06T06:46:35.751 INFO:teuthology.orchestra.run.smithi170.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-06-06T06:46:36.135 INFO:teuthology.orchestra.run.smithi136.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 890 kB 00:00 2024-06-06T06:46:36.159 INFO:teuthology.orchestra.run.smithi073.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 24 kB/s | 6.6 kB 00:00 2024-06-06T06:46:36.235 INFO:teuthology.orchestra.run.smithi136.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.4 MB/s | 145 kB 00:00 2024-06-06T06:46:36.401 INFO:teuthology.orchestra.run.smithi170.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 890 kB 00:00 2024-06-06T06:46:36.527 INFO:teuthology.orchestra.run.smithi170.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.1 MB/s | 145 kB 00:00 2024-06-06T06:46:37.127 INFO:teuthology.orchestra.run.smithi136.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.6 MB/s | 2.4 MB 00:00 2024-06-06T06:46:37.193 INFO:teuthology.orchestra.run.smithi073.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 861 kB/s | 890 kB 00:01 2024-06-06T06:46:37.401 INFO:teuthology.orchestra.run.smithi073.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 695 kB/s | 145 kB 00:00 2024-06-06T06:46:37.594 INFO:teuthology.orchestra.run.smithi136.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-06-06T06:46:37.619 INFO:teuthology.orchestra.run.smithi170.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 2.4 MB 00:01 2024-06-06T06:46:38.211 INFO:teuthology.orchestra.run.smithi170.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-06-06T06:46:38.977 INFO:teuthology.orchestra.run.smithi136.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.4 MB/s | 4.7 MB 00:01 2024-06-06T06:46:39.194 INFO:teuthology.orchestra.run.smithi136.stdout:(7/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-06-06T06:46:39.586 INFO:teuthology.orchestra.run.smithi170.stdout:(6/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 5.4 MB 00:04 2024-06-06T06:46:39.669 INFO:teuthology.orchestra.run.smithi170.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 3.2 MB/s | 4.7 MB 00:01 2024-06-06T06:46:39.718 INFO:teuthology.orchestra.run.smithi073.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.4 MB/s | 5.4 MB 00:03 2024-06-06T06:46:39.760 INFO:teuthology.orchestra.run.smithi073.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-06-06T06:46:40.543 INFO:teuthology.orchestra.run.smithi073.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 1.5 MB 00:00 2024-06-06T06:46:42.419 INFO:teuthology.orchestra.run.smithi136.stdout:(8/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x86 4.9 MB/s | 17 MB 00:03 2024-06-06T06:46:42.494 INFO:teuthology.orchestra.run.smithi136.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 336 kB/s | 25 kB 00:00 2024-06-06T06:46:42.810 INFO:teuthology.orchestra.run.smithi073.stdout:(7/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 21 MB 00:06 2024-06-06T06:46:43.036 INFO:teuthology.orchestra.run.smithi136.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 2.8 MB/s | 21 MB 00:07 2024-06-06T06:46:43.119 INFO:teuthology.orchestra.run.smithi170.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 3.0 MB/s | 10 MB 00:03 2024-06-06T06:46:43.145 INFO:teuthology.orchestra.run.smithi136.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.0 MB/s | 221 kB 00:00 2024-06-06T06:46:43.211 INFO:teuthology.orchestra.run.smithi136.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 407 kB/s | 27 kB 00:00 2024-06-06T06:46:43.270 INFO:teuthology.orchestra.run.smithi136.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 544 kB/s | 32 kB 00:00 2024-06-06T06:46:43.369 INFO:teuthology.orchestra.run.smithi170.stdout:(9/159): ceph-common-19.0.0-2946.g3d89accf.el9. 2.7 MB/s | 21 MB 00:07 2024-06-06T06:46:43.386 INFO:teuthology.orchestra.run.smithi170.stdout:(10/159): ceph-selinux-19.0.0-2946.g3d89accf.el 94 kB/s | 25 kB 00:00 2024-06-06T06:46:43.479 INFO:teuthology.orchestra.run.smithi136.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.5 MB/s | 747 kB 00:00 2024-06-06T06:46:43.537 INFO:teuthology.orchestra.run.smithi170.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-06T06:46:43.546 INFO:teuthology.orchestra.run.smithi136.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.4 MB/s | 163 kB 00:00 2024-06-06T06:46:43.603 INFO:teuthology.orchestra.run.smithi170.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 406 kB/s | 27 kB 00:00 2024-06-06T06:46:43.613 INFO:teuthology.orchestra.run.smithi136.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-06T06:46:43.662 INFO:teuthology.orchestra.run.smithi170.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 545 kB/s | 32 kB 00:00 2024-06-06T06:46:43.771 INFO:teuthology.orchestra.run.smithi136.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 3.1 MB/s | 501 kB 00:00 2024-06-06T06:46:43.879 INFO:teuthology.orchestra.run.smithi170.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.4 MB/s | 747 kB 00:00 2024-06-06T06:46:43.893 INFO:teuthology.orchestra.run.smithi073.stdout:(8/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 1.1 MB/s | 4.7 MB 00:04 2024-06-06T06:46:43.955 INFO:teuthology.orchestra.run.smithi170.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-06T06:46:44.010 INFO:teuthology.orchestra.run.smithi073.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 216 kB/s | 25 kB 00:00 2024-06-06T06:46:44.021 INFO:teuthology.orchestra.run.smithi170.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.9 MB/s | 127 kB 00:00 2024-06-06T06:46:44.197 INFO:teuthology.orchestra.run.smithi170.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.8 MB/s | 501 kB 00:00 2024-06-06T06:46:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 5.2 MB/s | 10 MB 00:01 2024-06-06T06:46:44.846 INFO:teuthology.orchestra.run.smithi136.stdout:(18/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 1.8 MB/s | 10 MB 00:05 2024-06-06T06:46:44.860 INFO:teuthology.orchestra.run.smithi073.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-06T06:46:44.928 INFO:teuthology.orchestra.run.smithi073.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 406 kB/s | 27 kB 00:00 2024-06-06T06:46:44.947 INFO:teuthology.orchestra.run.smithi136.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 452 kB/s | 45 kB 00:00 2024-06-06T06:46:44.986 INFO:teuthology.orchestra.run.smithi073.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 546 kB/s | 32 kB 00:00 2024-06-06T06:46:45.064 INFO:teuthology.orchestra.run.smithi136.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.1 MB/s | 135 kB 00:00 2024-06-06T06:46:45.178 INFO:teuthology.orchestra.run.smithi073.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 3.8 MB/s | 747 kB 00:00 2024-06-06T06:46:45.253 INFO:teuthology.orchestra.run.smithi073.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-06T06:46:45.305 INFO:teuthology.orchestra.run.smithi136.stdout:(21/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 3.5 MB/s | 5.3 MB 00:01 2024-06-06T06:46:45.312 INFO:teuthology.orchestra.run.smithi073.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-06T06:46:45.331 INFO:teuthology.orchestra.run.smithi136.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 606 kB/s | 162 kB 00:00 2024-06-06T06:46:45.414 INFO:teuthology.orchestra.run.smithi170.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 4.4 MB/s | 5.3 MB 00:01 2024-06-06T06:46:45.421 INFO:teuthology.orchestra.run.smithi073.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.5 MB/s | 501 kB 00:00 2024-06-06T06:46:45.423 INFO:teuthology.orchestra.run.smithi136.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 2.7 MB/s | 321 kB 00:00 2024-06-06T06:46:45.464 INFO:teuthology.orchestra.run.smithi170.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 909 kB/s | 45 kB 00:00 2024-06-06T06:46:45.490 INFO:teuthology.orchestra.run.smithi136.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 99 kB 00:00 2024-06-06T06:46:45.539 INFO:teuthology.orchestra.run.smithi170.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T06:46:45.549 INFO:teuthology.orchestra.run.smithi136.stdout:(25/159): python3-rbd-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 300 kB 00:00 2024-06-06T06:46:45.574 INFO:teuthology.orchestra.run.smithi136.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-06T06:46:45.598 INFO:teuthology.orchestra.run.smithi170.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.7 MB/s | 162 kB 00:00 2024-06-06T06:46:45.658 INFO:teuthology.orchestra.run.smithi136.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-06T06:46:45.706 INFO:teuthology.orchestra.run.smithi170.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.9 MB/s | 321 kB 00:00 2024-06-06T06:46:45.733 INFO:teuthology.orchestra.run.smithi136.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-06T06:46:45.808 INFO:teuthology.orchestra.run.smithi136.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.9 MB/s | 142 kB 00:00 2024-06-06T06:46:45.832 INFO:teuthology.orchestra.run.smithi170.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.3 MB/s | 300 kB 00:00 2024-06-06T06:46:45.907 INFO:teuthology.orchestra.run.smithi170.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-06T06:46:45.982 INFO:teuthology.orchestra.run.smithi170.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.1 MB/s | 85 kB 00:00 2024-06-06T06:46:46.463 INFO:teuthology.orchestra.run.smithi073.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 5.1 MB/s | 5.3 MB 00:01 2024-06-06T06:46:46.538 INFO:teuthology.orchestra.run.smithi073.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 603 kB/s | 45 kB 00:00 2024-06-06T06:46:46.613 INFO:teuthology.orchestra.run.smithi073.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.8 MB/s | 135 kB 00:00 2024-06-06T06:46:46.688 INFO:teuthology.orchestra.run.smithi073.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-06T06:46:46.774 INFO:teuthology.orchestra.run.smithi170.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.9 MB/s | 3.1 MB 00:00 2024-06-06T06:46:46.775 INFO:teuthology.orchestra.run.smithi136.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.6 MB/s | 3.5 MB 00:00 2024-06-06T06:46:46.780 INFO:teuthology.orchestra.run.smithi073.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-06T06:46:46.833 INFO:teuthology.orchestra.run.smithi170.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-06-06T06:46:46.906 INFO:teuthology.orchestra.run.smithi073.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.3 MB/s | 300 kB 00:00 2024-06-06T06:46:46.908 INFO:teuthology.orchestra.run.smithi170.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-06T06:46:46.964 INFO:teuthology.orchestra.run.smithi073.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 99 kB 00:00 2024-06-06T06:46:46.983 INFO:teuthology.orchestra.run.smithi170.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.9 MB/s | 142 kB 00:00 2024-06-06T06:46:47.048 INFO:teuthology.orchestra.run.smithi073.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-06T06:46:47.292 INFO:teuthology.orchestra.run.smithi136.stdout:(31/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 1.8 MB/s | 3.1 MB 00:01 2024-06-06T06:46:47.476 INFO:teuthology.orchestra.run.smithi136.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.3 MB/s | 249 kB 00:00 2024-06-06T06:46:47.584 INFO:teuthology.orchestra.run.smithi136.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 456 kB/s | 49 kB 00:00 2024-06-06T06:46:47.684 INFO:teuthology.orchestra.run.smithi136.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 165 kB/s | 16 kB 00:00 2024-06-06T06:46:47.823 INFO:teuthology.orchestra.run.smithi073.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 4.0 MB/s | 3.1 MB 00:00 2024-06-06T06:46:47.851 INFO:teuthology.orchestra.run.smithi136.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.5 MB/s | 264 kB 00:00 2024-06-06T06:46:47.907 INFO:teuthology.orchestra.run.smithi073.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.0 MB/s | 171 kB 00:00 2024-06-06T06:46:47.959 INFO:teuthology.orchestra.run.smithi170.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 3.6 MB/s | 3.5 MB 00:00 2024-06-06T06:46:47.973 INFO:teuthology.orchestra.run.smithi073.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 476 kB/s | 32 kB 00:00 2024-06-06T06:46:48.057 INFO:teuthology.orchestra.run.smithi073.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-06T06:46:48.218 INFO:teuthology.orchestra.run.smithi136.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 2.0 MB/s | 763 kB 00:00 2024-06-06T06:46:48.319 INFO:teuthology.orchestra.run.smithi136.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 182 kB/s | 18 kB 00:00 2024-06-06T06:46:48.365 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:48.436 INFO:teuthology.orchestra.run.smithi136.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 689 kB/s | 80 kB 00:00 2024-06-06T06:46:48.436 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:48.519 INFO:teuthology.orchestra.run.smithi136.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-06-06T06:46:48.520 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:48.558 INFO:teuthology.orchestra.run.smithi073.stdout:(30/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 17 MB 00:08 2024-06-06T06:46:48.561 INFO:teuthology.orchestra.run.smithi136.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-06T06:46:48.603 INFO:teuthology.orchestra.run.smithi136.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-06-06T06:46:48.662 INFO:teuthology.orchestra.run.smithi136.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 21 MB/s | 1.2 MB 00:00 2024-06-06T06:46:48.704 INFO:teuthology.orchestra.run.smithi136.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T06:46:48.746 INFO:teuthology.orchestra.run.smithi136.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-06-06T06:46:48.908 INFO:teuthology.orchestra.run.smithi073.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 4.1 MB/s | 3.5 MB 00:00 2024-06-06T06:46:48.966 INFO:teuthology.orchestra.run.smithi073.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 4.2 MB/s | 249 kB 00:00 2024-06-06T06:46:49.025 INFO:teuthology.orchestra.run.smithi073.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 849 kB/s | 49 kB 00:00 2024-06-06T06:46:49.075 INFO:teuthology.orchestra.run.smithi073.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 329 kB/s | 16 kB 00:00 2024-06-06T06:46:49.142 INFO:teuthology.orchestra.run.smithi073.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 264 kB 00:00 2024-06-06T06:46:49.180 INFO:teuthology.orchestra.run.smithi136.stdout:(45/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 7.2 MB/s | 48 MB 00:06 2024-06-06T06:46:49.205 INFO:teuthology.orchestra.run.smithi136.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 272 kB/s | 125 kB 00:00 2024-06-06T06:46:49.289 INFO:teuthology.orchestra.run.smithi136.stdout:(47/159): ceph-mgr-diskprediction-local-19.0.0- 2.9 MB/s | 7.4 MB 00:02 2024-06-06T06:46:49.289 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.290 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.309 INFO:teuthology.orchestra.run.smithi073.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.5 MB/s | 763 kB 00:00 2024-06-06T06:46:49.351 INFO:teuthology.orchestra.run.smithi073.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 437 kB/s | 18 kB 00:00 2024-06-06T06:46:49.356 INFO:teuthology.orchestra.run.smithi136.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-06-06T06:46:49.381 INFO:teuthology.orchestra.run.smithi136.stdout:(49/159): python3-urllib3-1.26.5-5.el9.noarch.r 1.0 MB/s | 215 kB 00:00 2024-06-06T06:46:49.382 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.384 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.397 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.415 INFO:teuthology.orchestra.run.smithi136.stdout:(50/159): elfutils-libelf-devel-0.191-4.el9.x86 654 kB/s | 22 kB 00:00 2024-06-06T06:46:49.440 INFO:teuthology.orchestra.run.smithi136.stdout:(51/159): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-06-06T06:46:49.466 INFO:teuthology.orchestra.run.smithi136.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T06:46:49.491 INFO:teuthology.orchestra.run.smithi136.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 650 kB/s | 33 kB 00:00 2024-06-06T06:46:49.518 INFO:teuthology.orchestra.run.smithi073.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-06-06T06:46:49.519 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.533 INFO:teuthology.orchestra.run.smithi136.stdout:(54/159): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 943 kB 00:00 2024-06-06T06:46:49.558 INFO:teuthology.orchestra.run.smithi136.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-06-06T06:46:49.575 INFO:teuthology.orchestra.run.smithi170.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 4.6 MB/s | 7.4 MB 00:01 2024-06-06T06:46:49.583 INFO:teuthology.orchestra.run.smithi136.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-06-06T06:46:49.609 INFO:teuthology.orchestra.run.smithi136.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-06T06:46:49.634 INFO:teuthology.orchestra.run.smithi136.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 898 kB/s | 45 kB 00:00 2024-06-06T06:46:49.643 INFO:teuthology.orchestra.run.smithi073.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 6.3 MB/s | 803 kB 00:00 2024-06-06T06:46:49.647 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:49.659 INFO:teuthology.orchestra.run.smithi170.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-06T06:46:49.694 INFO:teuthology.orchestra.run.smithi073.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 195 kB 00:00 2024-06-06T06:46:49.701 INFO:teuthology.orchestra.run.smithi136.stdout:(59/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-06-06T06:46:49.709 INFO:teuthology.orchestra.run.smithi170.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 991 kB/s | 49 kB 00:00 2024-06-06T06:46:49.726 INFO:teuthology.orchestra.run.smithi136.stdout:(60/159): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-06-06T06:46:49.736 INFO:teuthology.orchestra.run.smithi073.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 6.0 MB/s | 253 kB 00:00 2024-06-06T06:46:49.752 INFO:teuthology.orchestra.run.smithi136.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 508 kB/s | 25 kB 00:00 2024-06-06T06:46:49.777 INFO:teuthology.orchestra.run.smithi136.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-06-06T06:46:49.802 INFO:teuthology.orchestra.run.smithi136.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T06:46:49.811 INFO:teuthology.orchestra.run.smithi073.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-06-06T06:46:49.827 INFO:teuthology.orchestra.run.smithi136.stdout:(64/159): libtool-2.4.6-45.el9.x86_64.rpm 2.9 MB/s | 582 kB 00:00 2024-06-06T06:46:49.844 INFO:teuthology.orchestra.run.smithi136.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 841 kB/s | 35 kB 00:00 2024-06-06T06:46:49.853 INFO:teuthology.orchestra.run.smithi073.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-06-06T06:46:49.870 INFO:teuthology.orchestra.run.smithi136.stdout:(66/159): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-06-06T06:46:49.884 INFO:teuthology.orchestra.run.smithi170.stdout:(34/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 1.6 MB/s | 17 MB 00:10 2024-06-06T06:46:49.910 INFO:teuthology.orchestra.run.smithi170.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 82 kB/s | 16 kB 00:00 2024-06-06T06:46:49.953 INFO:teuthology.orchestra.run.smithi136.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 41 MB/s | 4.4 MB 00:00 2024-06-06T06:46:49.960 INFO:teuthology.orchestra.run.smithi073.stdout:(44/159): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-06-06T06:46:50.004 INFO:teuthology.orchestra.run.smithi073.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 2.8 MB/s | 125 kB 00:00 2024-06-06T06:46:50.035 INFO:teuthology.orchestra.run.smithi170.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 1.7 MB/s | 264 kB 00:00 2024-06-06T06:46:50.054 INFO:teuthology.orchestra.run.smithi073.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T06:46:50.058 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.062 INFO:teuthology.orchestra.run.smithi136.stdout:(68/159): openblas-openmp-0.3.21-2.el9.x86_64.r 20 MB/s | 4.7 MB 00:00 2024-06-06T06:46:50.069 INFO:teuthology.orchestra.run.smithi170.stdout:(37/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.7 MB/s | 763 kB 00:00 2024-06-06T06:46:50.086 INFO:teuthology.orchestra.run.smithi170.stdout:(38/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 363 kB/s | 18 kB 00:00 2024-06-06T06:46:50.104 INFO:teuthology.orchestra.run.smithi136.stdout:(69/159): postgresql-13.14-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-06-06T06:46:50.129 INFO:teuthology.orchestra.run.smithi136.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-06-06T06:46:50.149 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] fuse-2.9.9-15.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/fuse-2.9.9-15.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.157 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] libgfortran-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libgfortran-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.171 INFO:teuthology.orchestra.run.smithi136.stdout:(71/159): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-06-06T06:46:50.213 INFO:teuthology.orchestra.run.smithi136.stdout:(72/159): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-06-06T06:46:50.239 INFO:teuthology.orchestra.run.smithi136.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-06-06T06:46:50.261 INFO:teuthology.orchestra.run.smithi170.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 418 kB/s | 80 kB 00:00 2024-06-06T06:46:50.265 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] libquadmath-11.4.1-3.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libquadmath-11.4.1-3.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.281 INFO:teuthology.orchestra.run.smithi136.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-06T06:46:50.328 INFO:teuthology.orchestra.run.smithi170.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-06-06T06:46:50.368 INFO:teuthology.orchestra.run.smithi073.stdout:(47/159): automake-1.16.2-8.el9.noarch.rpm 2.1 MB/s | 689 kB 00:00 2024-06-06T06:46:50.370 INFO:teuthology.orchestra.run.smithi170.stdout:(41/159): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-06-06T06:46:50.371 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.381 INFO:teuthology.orchestra.run.smithi136.stdout:(75/159): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-06-06T06:46:50.395 INFO:teuthology.orchestra.run.smithi170.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-06-06T06:46:50.415 INFO:teuthology.orchestra.run.smithi136.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-06-06T06:46:50.429 INFO:teuthology.orchestra.run.smithi073.stdout:(48/159): bison-3.7.4-5.el9.x86_64.rpm 15 MB/s | 943 kB 00:00 2024-06-06T06:46:50.433 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.440 INFO:teuthology.orchestra.run.smithi136.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-06-06T06:46:50.446 INFO:teuthology.orchestra.run.smithi170.stdout:(43/159): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-06-06T06:46:50.465 INFO:teuthology.orchestra.run.smithi136.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T06:46:50.471 INFO:teuthology.orchestra.run.smithi170.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-06-06T06:46:50.480 INFO:teuthology.orchestra.run.smithi073.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-06-06T06:46:50.491 INFO:teuthology.orchestra.run.smithi136.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-06-06T06:46:50.496 INFO:teuthology.orchestra.run.smithi170.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-06-06T06:46:50.516 INFO:teuthology.orchestra.run.smithi136.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-06T06:46:50.522 INFO:teuthology.orchestra.run.smithi170.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-06T06:46:50.541 INFO:teuthology.orchestra.run.smithi136.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 691 kB/s | 35 kB 00:00 2024-06-06T06:46:50.547 INFO:teuthology.orchestra.run.smithi170.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-06T06:46:50.547 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] automake-1.16.2-8.el9.noarch.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/automake-1.16.2-8.el9.noarch.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.549 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] bison-3.7.4-5.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/bison-3.7.4-5.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.572 INFO:teuthology.orchestra.run.smithi073.stdout:(50/159): ceph-mgr-diskprediction-local-19.0.0- 3.7 MB/s | 7.4 MB 00:02 2024-06-06T06:46:50.597 INFO:teuthology.orchestra.run.smithi073.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 186 kB/s | 22 kB 00:00 2024-06-06T06:46:50.617 INFO:teuthology.orchestra.run.smithi136.stdout:(82/159): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-06-06T06:46:50.639 INFO:teuthology.orchestra.run.smithi073.stdout:(52/159): flexiblas-3.0.4-8.el9.x86_64.rpm 785 kB/s | 33 kB 00:00 2024-06-06T06:46:50.642 INFO:teuthology.orchestra.run.smithi136.stdout:(83/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-06-06T06:46:50.667 INFO:teuthology.orchestra.run.smithi136.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 4.3 MB/s | 222 kB 00:00 2024-06-06T06:46:50.681 INFO:teuthology.orchestra.run.smithi170.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 4.2 MB/s | 689 kB 00:00 2024-06-06T06:46:50.693 INFO:teuthology.orchestra.run.smithi136.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T06:46:50.718 INFO:teuthology.orchestra.run.smithi136.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-06-06T06:46:50.723 INFO:teuthology.orchestra.run.smithi170.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 5.3 MB/s | 943 kB 00:00 2024-06-06T06:46:50.723 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] boost-program-options-1.75.0-8.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/boost-program-options-1.75.0-8.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.726 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] elfutils-libelf-devel-0.191-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/AppStream/x86_64/os/Packages/elfutils-libelf-devel-0.191-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:50.731 INFO:teuthology.orchestra.run.smithi073.stdout:(53/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-06-06T06:46:50.743 INFO:teuthology.orchestra.run.smithi136.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-06-06T06:46:50.758 INFO:teuthology.orchestra.run.smithi073.stdout:(54/159): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-06-06T06:46:50.764 INFO:teuthology.orchestra.run.smithi170.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 107 kB 00:00 2024-06-06T06:46:50.769 INFO:teuthology.orchestra.run.smithi136.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T06:46:50.790 INFO:teuthology.orchestra.run.smithi170.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-06-06T06:46:50.794 INFO:teuthology.orchestra.run.smithi136.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T06:46:50.815 INFO:teuthology.orchestra.run.smithi170.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-06-06T06:46:50.819 INFO:teuthology.orchestra.run.smithi136.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T06:46:50.823 INFO:teuthology.orchestra.run.smithi073.stdout:(55/159): flexiblas-openblas-openmp-3.0.4-8.el9 198 kB/s | 18 kB 00:00 2024-06-06T06:46:50.840 INFO:teuthology.orchestra.run.smithi170.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-06-06T06:46:50.841 INFO:teuthology.orchestra.run.smithi073.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-06T06:46:50.844 INFO:teuthology.orchestra.run.smithi136.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T06:46:50.866 INFO:teuthology.orchestra.run.smithi073.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-06T06:46:50.882 INFO:teuthology.orchestra.run.smithi170.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-06-06T06:46:50.891 INFO:teuthology.orchestra.run.smithi073.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 899 kB/s | 45 kB 00:00 2024-06-06T06:46:50.928 INFO:teuthology.orchestra.run.smithi136.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-06-06T06:46:50.933 INFO:teuthology.orchestra.run.smithi073.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-06-06T06:46:50.949 INFO:teuthology.orchestra.run.smithi170.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-06-06T06:46:50.953 INFO:teuthology.orchestra.run.smithi136.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 383 kB/s | 42 kB 00:00 2024-06-06T06:46:50.975 INFO:teuthology.orchestra.run.smithi170.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-06-06T06:46:50.975 INFO:teuthology.orchestra.run.smithi073.stdout:(60/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 612 kB/s | 25 kB 00:00 2024-06-06T06:46:50.979 INFO:teuthology.orchestra.run.smithi136.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-06-06T06:46:51.000 INFO:teuthology.orchestra.run.smithi073.stdout:(61/159): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-06-06T06:46:51.012 INFO:teuthology.orchestra.run.smithi136.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-06-06T06:46:51.016 INFO:teuthology.orchestra.run.smithi170.stdout:(57/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-06T06:46:51.025 INFO:teuthology.orchestra.run.smithi073.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-06-06T06:46:51.037 INFO:teuthology.orchestra.run.smithi136.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-06-06T06:46:51.042 INFO:teuthology.orchestra.run.smithi170.stdout:(58/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-06-06T06:46:51.051 INFO:teuthology.orchestra.run.smithi073.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T06:46:51.063 INFO:teuthology.orchestra.run.smithi136.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-06-06T06:46:51.075 INFO:teuthology.orchestra.run.smithi170.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-06T06:46:51.076 INFO:teuthology.orchestra.run.smithi073.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T06:46:51.088 INFO:teuthology.orchestra.run.smithi136.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T06:46:51.101 INFO:teuthology.orchestra.run.smithi073.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-06-06T06:46:51.109 INFO:teuthology.orchestra.run.smithi170.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-06T06:46:51.114 INFO:teuthology.orchestra.run.smithi136.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T06:46:51.134 INFO:teuthology.orchestra.run.smithi170.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-06-06T06:46:51.139 INFO:teuthology.orchestra.run.smithi136.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T06:46:51.160 INFO:teuthology.orchestra.run.smithi170.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-06T06:46:51.165 INFO:teuthology.orchestra.run.smithi136.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T06:46:51.181 INFO:teuthology.orchestra.run.smithi136.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-06-06T06:46:51.182 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:51.183 INFO:teuthology.orchestra.run.smithi136.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:51.185 INFO:teuthology.orchestra.run.smithi170.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-06T06:46:51.210 INFO:teuthology.orchestra.run.smithi073.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-06-06T06:46:51.211 INFO:teuthology.orchestra.run.smithi170.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-06T06:46:51.236 INFO:teuthology.orchestra.run.smithi170.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-06-06T06:46:51.282 INFO:teuthology.orchestra.run.smithi136.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-06-06T06:46:51.285 INFO:teuthology.orchestra.run.smithi073.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-06-06T06:46:51.336 INFO:teuthology.orchestra.run.smithi073.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-06T06:46:51.336 INFO:teuthology.orchestra.run.smithi170.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 38 MB/s | 4.7 MB 00:00 2024-06-06T06:46:51.378 INFO:teuthology.orchestra.run.smithi073.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 9.7 MB/s | 911 kB 00:00 2024-06-06T06:46:51.395 INFO:teuthology.orchestra.run.smithi073.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T06:46:51.395 INFO:teuthology.orchestra.run.smithi170.stdout:(67/159): postgresql-13.14-1.el9.x86_64.rpm 27 MB/s | 1.6 MB 00:00 2024-06-06T06:46:51.432 INFO:teuthology.orchestra.run.smithi136.stdout:(104/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-06-06T06:46:51.462 INFO:teuthology.orchestra.run.smithi073.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-06-06T06:46:51.463 INFO:teuthology.orchestra.run.smithi170.stdout:(68/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-06-06T06:46:51.491 INFO:teuthology.orchestra.run.smithi136.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 2.6 MB/s | 548 kB 00:00 2024-06-06T06:46:51.496 INFO:teuthology.orchestra.run.smithi170.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-06-06T06:46:51.507 INFO:teuthology.orchestra.run.smithi136.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 1.4 MB/s | 19 kB 00:00 2024-06-06T06:46:51.521 INFO:teuthology.orchestra.run.smithi170.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-06T06:46:51.533 INFO:teuthology.orchestra.run.smithi136.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.4 MB/s | 863 kB 00:00 2024-06-06T06:46:51.546 INFO:teuthology.orchestra.run.smithi073.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-06-06T06:46:51.574 INFO:teuthology.orchestra.run.smithi073.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 806 kB/s | 87 kB 00:00 2024-06-06T06:46:51.574 INFO:teuthology.orchestra.run.smithi170.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-06T06:46:51.591 INFO:teuthology.orchestra.run.smithi136.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.9 MB/s | 309 kB 00:00 2024-06-06T06:46:51.613 INFO:teuthology.orchestra.run.smithi073.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-06T06:46:51.617 INFO:teuthology.orchestra.run.smithi136.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-06-06T06:46:51.642 INFO:teuthology.orchestra.run.smithi136.stdout:(110/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 239 kB/s | 26 kB 00:00 2024-06-06T06:46:51.667 INFO:teuthology.orchestra.run.smithi136.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-06T06:46:51.701 INFO:teuthology.orchestra.run.smithi136.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-06T06:46:51.705 INFO:teuthology.orchestra.run.smithi073.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-06-06T06:46:51.730 INFO:teuthology.orchestra.run.smithi073.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.0 MB/s | 244 kB 00:00 2024-06-06T06:46:51.743 INFO:teuthology.orchestra.run.smithi136.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-06-06T06:46:51.755 INFO:teuthology.orchestra.run.smithi073.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-06-06T06:46:51.768 INFO:teuthology.orchestra.run.smithi136.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-06T06:46:51.780 INFO:teuthology.orchestra.run.smithi073.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-06-06T06:46:51.793 INFO:teuthology.orchestra.run.smithi136.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-06T06:46:51.806 INFO:teuthology.orchestra.run.smithi073.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-06-06T06:46:51.819 INFO:teuthology.orchestra.run.smithi136.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 638 kB/s | 32 kB 00:00 2024-06-06T06:46:51.831 INFO:teuthology.orchestra.run.smithi073.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-06T06:46:51.844 INFO:teuthology.orchestra.run.smithi136.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 298 kB/s | 15 kB 00:00 2024-06-06T06:46:51.856 INFO:teuthology.orchestra.run.smithi073.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-06-06T06:46:51.869 INFO:teuthology.orchestra.run.smithi136.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.4 MB/s | 172 kB 00:00 2024-06-06T06:46:51.903 INFO:teuthology.orchestra.run.smithi136.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-06-06T06:46:51.906 INFO:teuthology.orchestra.run.smithi073.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-06-06T06:46:51.928 INFO:teuthology.orchestra.run.smithi136.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.0 MB/s | 303 kB 00:00 2024-06-06T06:46:51.954 INFO:teuthology.orchestra.run.smithi136.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T06:46:51.973 INFO:teuthology.orchestra.run.smithi073.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-06-06T06:46:51.987 INFO:teuthology.orchestra.run.smithi136.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-06-06T06:46:51.999 INFO:teuthology.orchestra.run.smithi073.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-06-06T06:46:52.013 INFO:teuthology.orchestra.run.smithi136.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-06-06T06:46:52.013 INFO:teuthology.orchestra.run.smithi170.stdout:(72/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 5.6 MB/s | 48 MB 00:08 2024-06-06T06:46:52.024 INFO:teuthology.orchestra.run.smithi073.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-06T06:46:52.038 INFO:teuthology.orchestra.run.smithi170.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 187 kB/s | 87 kB 00:00 2024-06-06T06:46:52.049 INFO:teuthology.orchestra.run.smithi073.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-06-06T06:46:52.075 INFO:teuthology.orchestra.run.smithi073.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-06-06T06:46:52.080 INFO:teuthology.orchestra.run.smithi136.stdout:(124/159): libarrow-9.0.0-11.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-06-06T06:46:52.100 INFO:teuthology.orchestra.run.smithi073.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-06-06T06:46:52.105 INFO:teuthology.orchestra.run.smithi136.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-06-06T06:46:52.122 INFO:teuthology.orchestra.run.smithi170.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 9.2 MB/s | 5.8 MB 00:00 2024-06-06T06:46:52.125 INFO:teuthology.orchestra.run.smithi073.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-06T06:46:52.130 INFO:teuthology.orchestra.run.smithi136.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-06T06:46:52.147 INFO:teuthology.orchestra.run.smithi170.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-06-06T06:46:52.150 INFO:teuthology.orchestra.run.smithi073.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T06:46:52.156 INFO:teuthology.orchestra.run.smithi136.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-06-06T06:46:52.173 INFO:teuthology.orchestra.run.smithi170.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-06-06T06:46:52.176 INFO:teuthology.orchestra.run.smithi073.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-06T06:46:52.181 INFO:teuthology.orchestra.run.smithi136.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 389 kB/s | 20 kB 00:00 2024-06-06T06:46:52.198 INFO:teuthology.orchestra.run.smithi170.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-06T06:46:52.207 INFO:teuthology.orchestra.run.smithi136.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T06:46:52.223 INFO:teuthology.orchestra.run.smithi170.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-06T06:46:52.226 INFO:teuthology.orchestra.run.smithi073.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 832 kB/s | 42 kB 00:00 2024-06-06T06:46:52.232 INFO:teuthology.orchestra.run.smithi136.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-06T06:46:52.249 INFO:teuthology.orchestra.run.smithi170.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-06-06T06:46:52.257 INFO:teuthology.orchestra.run.smithi136.stdout:(131/159): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-06-06T06:46:52.268 INFO:teuthology.orchestra.run.smithi073.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-06T06:46:52.274 INFO:teuthology.orchestra.run.smithi170.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-06-06T06:46:52.299 INFO:teuthology.orchestra.run.smithi136.stdout:(132/159): python3-grpcio-1.46.7-10.el9.x86_64. 5.5 MB/s | 2.0 MB 00:00 2024-06-06T06:46:52.299 INFO:teuthology.orchestra.run.smithi170.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-06-06T06:46:52.318 INFO:teuthology.orchestra.run.smithi073.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T06:46:52.333 INFO:teuthology.orchestra.run.smithi136.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-06-06T06:46:52.341 INFO:teuthology.orchestra.run.smithi170.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-06T06:46:52.358 INFO:teuthology.orchestra.run.smithi136.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-06-06T06:46:52.360 INFO:teuthology.orchestra.run.smithi073.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-06-06T06:46:52.383 INFO:teuthology.orchestra.run.smithi170.stdout:(83/159): python3-oauthlib-3.1.1-5.el9.noarch.r 5.2 MB/s | 222 kB 00:00 2024-06-06T06:46:52.383 INFO:teuthology.orchestra.run.smithi136.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 195 kB/s | 16 kB 00:00 2024-06-06T06:46:52.409 INFO:teuthology.orchestra.run.smithi136.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-06-06T06:46:52.426 INFO:teuthology.orchestra.run.smithi136.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 734 kB/s | 31 kB 00:00 2024-06-06T06:46:52.451 INFO:teuthology.orchestra.run.smithi136.stdout:(138/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 763 kB/s | 90 kB 00:00 2024-06-06T06:46:52.467 INFO:teuthology.orchestra.run.smithi170.stdout:(84/159): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-06-06T06:46:52.468 INFO:teuthology.orchestra.run.smithi136.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-06-06T06:46:52.485 INFO:teuthology.orchestra.run.smithi136.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-06-06T06:46:52.502 INFO:teuthology.orchestra.run.smithi136.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-06T06:46:52.527 INFO:teuthology.orchestra.run.smithi073.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-06-06T06:46:52.528 INFO:teuthology.orchestra.run.smithi136.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-06-06T06:46:52.534 INFO:teuthology.orchestra.run.smithi170.stdout:(85/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-06-06T06:46:52.550 INFO:teuthology.orchestra.run.smithi170.stdout:(86/159): python3-packaging-20.9-5.el9.noarch.r 462 kB/s | 77 kB 00:00 2024-06-06T06:46:52.553 INFO:teuthology.orchestra.run.smithi136.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-06T06:46:52.553 INFO:teuthology.orchestra.run.smithi073.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-06-06T06:46:52.576 INFO:teuthology.orchestra.run.smithi170.stdout:(87/159): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-06-06T06:46:52.578 INFO:teuthology.orchestra.run.smithi136.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-06-06T06:46:52.578 INFO:teuthology.orchestra.run.smithi073.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-06T06:46:52.595 INFO:teuthology.orchestra.run.smithi073.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-06-06T06:46:52.603 INFO:teuthology.orchestra.run.smithi136.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-06-06T06:46:52.610 INFO:teuthology.orchestra.run.smithi170.stdout:(88/159): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-06-06T06:46:52.620 INFO:teuthology.orchestra.run.smithi073.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 2.2 MB/s | 94 kB 00:00 2024-06-06T06:46:52.629 INFO:teuthology.orchestra.run.smithi136.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T06:46:52.634 INFO:teuthology.orchestra.run.smithi170.stdout:(89/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-06-06T06:46:52.637 INFO:teuthology.orchestra.run.smithi073.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2024-06-06T06:46:52.660 INFO:teuthology.orchestra.run.smithi170.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-06T06:46:52.662 INFO:teuthology.orchestra.run.smithi136.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-06-06T06:46:52.663 INFO:teuthology.orchestra.run.smithi073.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 1.0 MB/s | 45 kB 00:00 2024-06-06T06:46:52.663 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:52.666 INFO:teuthology.orchestra.run.smithi073.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:52.685 INFO:teuthology.orchestra.run.smithi170.stdout:(91/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 279 kB 00:00 2024-06-06T06:46:52.710 INFO:teuthology.orchestra.run.smithi170.stdout:(92/159): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-06-06T06:46:52.721 INFO:teuthology.orchestra.run.smithi073.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-06-06T06:46:52.752 INFO:teuthology.orchestra.run.smithi170.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-06T06:46:52.754 INFO:teuthology.orchestra.run.smithi136.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-06-06T06:46:52.778 INFO:teuthology.orchestra.run.smithi170.stdout:(94/159): python3-toml-0.10.2-6.el9.noarch.rpm 623 kB/s | 42 kB 00:00 2024-06-06T06:46:52.788 INFO:teuthology.orchestra.run.smithi073.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-06-06T06:46:52.806 INFO:teuthology.orchestra.run.smithi170.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-06-06T06:46:52.830 INFO:teuthology.orchestra.run.smithi073.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 7.3 MB/s | 309 kB 00:00 2024-06-06T06:46:52.837 INFO:teuthology.orchestra.run.smithi170.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-06-06T06:46:52.862 INFO:teuthology.orchestra.run.smithi170.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-06-06T06:46:52.864 INFO:teuthology.orchestra.run.smithi073.stdout:(106/159): protobuf-compiler-3.14.0-13.el9.x86_ 4.2 MB/s | 863 kB 00:00 2024-06-06T06:46:52.887 INFO:teuthology.orchestra.run.smithi170.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-06-06T06:46:52.889 INFO:teuthology.orchestra.run.smithi073.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 332 kB/s | 19 kB 00:00 2024-06-06T06:46:52.904 INFO:teuthology.orchestra.run.smithi136.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-06T06:46:52.913 INFO:teuthology.orchestra.run.smithi170.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-06T06:46:52.938 INFO:teuthology.orchestra.run.smithi170.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-06T06:46:52.939 INFO:teuthology.orchestra.run.smithi073.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.1 MB/s | 26 kB 00:00 2024-06-06T06:46:52.963 INFO:teuthology.orchestra.run.smithi136.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 31 MB/s | 1.8 MB 00:00 2024-06-06T06:46:52.963 INFO:teuthology.orchestra.run.smithi170.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-06T06:46:52.964 INFO:teuthology.orchestra.run.smithi073.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-06-06T06:46:52.989 INFO:teuthology.orchestra.run.smithi170.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 891 kB/s | 45 kB 00:00 2024-06-06T06:46:52.989 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] lua-devel-5.4.4-4.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/lua-devel-5.4.4-4.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:52.990 INFO:teuthology.orchestra.run.smithi170.stdout:[MIRROR] protobuf-compiler-3.14.0-13.el9.x86_64.rpm: Status code: 404 for https://download-cc-rdu01.fedoraproject.org/pub/centos-stream/9-stream/CRB/x86_64/os/Packages/protobuf-compiler-3.14.0-13.el9.x86_64.rpm (IP: 8.43.85.72) 2024-06-06T06:46:52.990 INFO:teuthology.orchestra.run.smithi073.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-06-06T06:46:53.005 INFO:teuthology.orchestra.run.smithi136.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 998 kB/s | 41 kB 00:00 2024-06-06T06:46:53.023 INFO:teuthology.orchestra.run.smithi073.stdout:(111/159): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-06-06T06:46:53.047 INFO:teuthology.orchestra.run.smithi170.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-06-06T06:46:53.063 INFO:teuthology.orchestra.run.smithi136.stdout:(152/159): librados2-19.0.0-2946.g3d89accf.el9. 7.8 MB/s | 3.4 MB 00:00 2024-06-06T06:46:53.082 INFO:teuthology.orchestra.run.smithi073.stdout:(112/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-06-06T06:46:53.089 INFO:teuthology.orchestra.run.smithi136.stdout:(153/159): NetworkManager-tui-1.47.91-1.el9.x86 2.9 MB/s | 248 kB 00:00 2024-06-06T06:46:53.090 INFO:teuthology.orchestra.run.smithi170.stdout:(104/159): abseil-cpp-20211102.0-3.el9.x86_64.r 13 MB/s | 548 kB 00:00 2024-06-06T06:46:53.122 INFO:teuthology.orchestra.run.smithi136.stdout:(154/159): elfutils-debuginfod-client-0.191-4.e 627 kB/s | 37 kB 00:00 2024-06-06T06:46:53.132 INFO:teuthology.orchestra.run.smithi073.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-06-06T06:46:53.148 INFO:teuthology.orchestra.run.smithi136.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 3.5 MB/s | 207 kB 00:00 2024-06-06T06:46:53.158 INFO:teuthology.orchestra.run.smithi073.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-06-06T06:46:53.183 INFO:teuthology.orchestra.run.smithi073.stdout:(115/159): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-06-06T06:46:53.207 INFO:teuthology.orchestra.run.smithi136.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-06T06:46:53.208 INFO:teuthology.orchestra.run.smithi073.stdout:(116/159): python3-certifi-2021.10.8-2.el9.noar 602 kB/s | 15 kB 00:00 2024-06-06T06:46:53.232 INFO:teuthology.orchestra.run.smithi136.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.3 MB/s | 260 kB 00:00 2024-06-06T06:46:53.233 INFO:teuthology.orchestra.run.smithi073.stdout:(117/159): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-06-06T06:46:53.248 INFO:teuthology.orchestra.run.smithi170.stdout:(105/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-06-06T06:46:53.273 INFO:teuthology.orchestra.run.smithi170.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 793 kB/s | 19 kB 00:00 2024-06-06T06:46:53.282 INFO:teuthology.orchestra.run.smithi136.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-06-06T06:46:53.292 INFO:teuthology.orchestra.run.smithi073.stdout:(118/159): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-06-06T06:46:53.307 INFO:teuthology.orchestra.run.smithi170.stdout:(107/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-06-06T06:46:53.332 INFO:teuthology.orchestra.run.smithi170.stdout:(108/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-06-06T06:46:53.334 INFO:teuthology.orchestra.run.smithi073.stdout:(119/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 7.1 MB/s | 303 kB 00:00 2024-06-06T06:46:53.360 INFO:teuthology.orchestra.run.smithi073.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-06-06T06:46:53.382 INFO:teuthology.orchestra.run.smithi170.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 2.0 MB/s | 49 kB 00:00 2024-06-06T06:46:53.385 INFO:teuthology.orchestra.run.smithi073.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T06:46:53.416 INFO:teuthology.orchestra.run.smithi170.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 2.1 MB/s | 67 kB 00:00 2024-06-06T06:46:53.418 INFO:teuthology.orchestra.run.smithi073.stdout:(122/159): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-06-06T06:46:53.441 INFO:teuthology.orchestra.run.smithi170.stdout:(111/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 194 kB/s | 26 kB 00:00 2024-06-06T06:46:53.444 INFO:teuthology.orchestra.run.smithi073.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 427 kB/s | 11 kB 00:00 2024-06-06T06:46:53.467 INFO:teuthology.orchestra.run.smithi170.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-06-06T06:46:53.469 INFO:teuthology.orchestra.run.smithi073.stdout:(124/159): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-06-06T06:46:53.511 INFO:teuthology.orchestra.run.smithi073.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-06-06T06:46:53.533 INFO:teuthology.orchestra.run.smithi170.stdout:(113/159): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-06-06T06:46:53.536 INFO:teuthology.orchestra.run.smithi073.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-06-06T06:46:53.562 INFO:teuthology.orchestra.run.smithi073.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 384 kB/s | 19 kB 00:00 2024-06-06T06:46:53.567 INFO:teuthology.orchestra.run.smithi170.stdout:(114/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 839 kB 00:00 2024-06-06T06:46:53.587 INFO:teuthology.orchestra.run.smithi073.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-06-06T06:46:53.592 INFO:teuthology.orchestra.run.smithi170.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 740 kB/s | 43 kB 00:00 2024-06-06T06:46:53.618 INFO:teuthology.orchestra.run.smithi170.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 639 kB/s | 32 kB 00:00 2024-06-06T06:46:53.622 INFO:teuthology.orchestra.run.smithi073.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-06-06T06:46:53.624 INFO:teuthology.orchestra.run.smithi136.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.3 MB/s | 3.1 MB 00:00 2024-06-06T06:46:53.630 INFO:teuthology.orchestra.run.smithi136.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:46:53.630 INFO:teuthology.orchestra.run.smithi136.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-06T06:46:53.643 INFO:teuthology.orchestra.run.smithi170.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-06-06T06:46:53.646 INFO:teuthology.orchestra.run.smithi073.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-06-06T06:46:53.668 INFO:teuthology.orchestra.run.smithi170.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 3.3 MB/s | 172 kB 00:00 2024-06-06T06:46:53.695 INFO:teuthology.orchestra.run.smithi073.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 7.7 MB/s | 1.0 MB 00:00 2024-06-06T06:46:53.722 INFO:teuthology.orchestra.run.smithi073.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-06-06T06:46:53.735 INFO:teuthology.orchestra.run.smithi170.stdout:(119/159): libarrow-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 4.4 MB 00:00 2024-06-06T06:46:53.747 INFO:teuthology.orchestra.run.smithi073.stdout:(133/159): python3-portend-3.1.0-2.el9.noarch.r 661 kB/s | 16 kB 00:00 2024-06-06T06:46:53.760 INFO:teuthology.orchestra.run.smithi170.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-06-06T06:46:53.772 INFO:teuthology.orchestra.run.smithi073.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-06-06T06:46:53.785 INFO:teuthology.orchestra.run.smithi170.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 4.0 MB/s | 206 kB 00:00 2024-06-06T06:46:53.797 INFO:teuthology.orchestra.run.smithi073.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-06-06T06:46:53.811 INFO:teuthology.orchestra.run.smithi170.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.1 MB/s | 303 kB 00:00 2024-06-06T06:46:53.823 INFO:teuthology.orchestra.run.smithi073.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-06-06T06:46:53.836 INFO:teuthology.orchestra.run.smithi170.stdout:(123/159): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-06-06T06:46:53.848 INFO:teuthology.orchestra.run.smithi073.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 612 kB/s | 31 kB 00:00 2024-06-06T06:46:53.861 INFO:teuthology.orchestra.run.smithi170.stdout:(124/159): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-06-06T06:46:53.865 INFO:teuthology.orchestra.run.smithi073.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-06-06T06:46:53.879 INFO:teuthology.orchestra.run.smithi170.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 418 kB/s | 18 kB 00:00 2024-06-06T06:46:53.882 INFO:teuthology.orchestra.run.smithi073.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2024-06-06T06:46:53.899 INFO:teuthology.orchestra.run.smithi073.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-06-06T06:46:53.904 INFO:teuthology.orchestra.run.smithi170.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 547 kB/s | 23 kB 00:00 2024-06-06T06:46:53.916 INFO:teuthology.orchestra.run.smithi073.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-06-06T06:46:53.921 INFO:teuthology.orchestra.run.smithi170.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 462 kB/s | 19 kB 00:00 2024-06-06T06:46:53.938 INFO:teuthology.orchestra.run.smithi170.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 587 kB/s | 20 kB 00:00 2024-06-06T06:46:53.941 INFO:teuthology.orchestra.run.smithi073.stdout:(142/159): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-06-06T06:46:53.964 INFO:teuthology.orchestra.run.smithi170.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-06-06T06:46:53.966 INFO:teuthology.orchestra.run.smithi073.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.3 MB/s | 230 kB 00:00 2024-06-06T06:46:53.983 INFO:teuthology.orchestra.run.smithi073.stdout:(144/159): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-06-06T06:46:53.997 INFO:teuthology.orchestra.run.smithi170.stdout:(130/159): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-06-06T06:46:54.017 INFO:teuthology.orchestra.run.smithi073.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-06-06T06:46:54.042 INFO:teuthology.orchestra.run.smithi073.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.2 MB/s | 192 kB 00:00 2024-06-06T06:46:54.056 INFO:teuthology.orchestra.run.smithi170.stdout:(131/159): python3-grpcio-1.46.7-10.el9.x86_64. 6.9 MB/s | 2.0 MB 00:00 2024-06-06T06:46:54.081 INFO:teuthology.orchestra.run.smithi170.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-06-06T06:46:54.114 INFO:teuthology.orchestra.run.smithi170.stdout:(133/159): python3-kubernetes-26.1.0-2.el9.noar 5.3 MB/s | 1.0 MB 00:00 2024-06-06T06:46:54.140 INFO:teuthology.orchestra.run.smithi170.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 280 kB/s | 16 kB 00:00 2024-06-06T06:46:54.142 INFO:teuthology.orchestra.run.smithi073.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-06-06T06:46:54.165 INFO:teuthology.orchestra.run.smithi170.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-06-06T06:46:54.190 INFO:teuthology.orchestra.run.smithi170.stdout:(136/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-06-06T06:46:54.216 INFO:teuthology.orchestra.run.smithi170.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-06-06T06:46:54.241 INFO:teuthology.orchestra.run.smithi170.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-06-06T06:46:54.266 INFO:teuthology.orchestra.run.smithi170.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-06-06T06:46:54.292 INFO:teuthology.orchestra.run.smithi170.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-06-06T06:46:54.317 INFO:teuthology.orchestra.run.smithi170.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-06-06T06:46:54.342 INFO:teuthology.orchestra.run.smithi170.stdout:(142/159): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-06-06T06:46:54.367 INFO:teuthology.orchestra.run.smithi170.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-06-06T06:46:54.393 INFO:teuthology.orchestra.run.smithi170.stdout:(144/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-06T06:46:54.418 INFO:teuthology.orchestra.run.smithi170.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 396 kB/s | 20 kB 00:00 2024-06-06T06:46:54.443 INFO:teuthology.orchestra.run.smithi170.stdout:(146/159): re2-20211101-3.el9.x86_64.rpm 3.7 MB/s | 192 kB 00:00 2024-06-06T06:46:54.469 INFO:teuthology.orchestra.run.smithi170.stdout:(147/159): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-06-06T06:46:54.484 INFO:teuthology.orchestra.run.smithi073.stdout:(148/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 9.2 MB/s | 3.1 MB 00:00 2024-06-06T06:46:54.536 INFO:teuthology.orchestra.run.smithi170.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-06T06:46:54.668 INFO:teuthology.orchestra.run.smithi073.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-06-06T06:46:54.735 INFO:teuthology.orchestra.run.smithi073.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T06:46:54.736 INFO:teuthology.orchestra.run.smithi170.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-06T06:46:54.777 INFO:teuthology.orchestra.run.smithi073.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 993 kB/s | 41 kB 00:00 2024-06-06T06:46:54.803 INFO:teuthology.orchestra.run.smithi170.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-06T06:46:54.827 INFO:teuthology.orchestra.run.smithi073.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.9 MB/s | 248 kB 00:00 2024-06-06T06:46:54.844 INFO:teuthology.orchestra.run.smithi170.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.0 MB/s | 41 kB 00:00 2024-06-06T06:46:54.863 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction check 2024-06-06T06:46:54.869 INFO:teuthology.orchestra.run.smithi073.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 885 kB/s | 37 kB 00:00 2024-06-06T06:46:54.886 INFO:teuthology.orchestra.run.smithi170.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-06T06:46:54.911 INFO:teuthology.orchestra.run.smithi073.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.8 MB/s | 207 kB 00:00 2024-06-06T06:46:54.929 INFO:teuthology.orchestra.run.smithi170.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 875 kB/s | 37 kB 00:00 2024-06-06T06:46:54.961 INFO:teuthology.orchestra.run.smithi073.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 5.1 MB/s | 260 kB 00:00 2024-06-06T06:46:54.979 INFO:teuthology.orchestra.run.smithi170.stdout:(154/159): librados2-19.0.0-2946.g3d89accf.el9. 6.3 MB/s | 3.4 MB 00:00 2024-06-06T06:46:55.005 INFO:teuthology.orchestra.run.smithi170.stdout:(155/159): elfutils-libelf-0.191-4.el9.x86_64.r 2.7 MB/s | 207 kB 00:00 2024-06-06T06:46:55.023 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction check succeeded. 2024-06-06T06:46:55.023 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction test 2024-06-06T06:46:55.071 INFO:teuthology.orchestra.run.smithi170.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-06-06T06:46:55.112 INFO:teuthology.orchestra.run.smithi073.stdout:(156/159): librados2-19.0.0-2946.g3d89accf.el9. 3.2 MB/s | 3.4 MB 00:01 2024-06-06T06:46:55.262 INFO:teuthology.orchestra.run.smithi073.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 4.5 MB/s | 1.3 MB 00:00 2024-06-06T06:46:55.305 INFO:teuthology.orchestra.run.smithi170.stdout:(157/159): elfutils-libs-0.191-4.el9.x86_64.rpm 798 kB/s | 260 kB 00:00 2024-06-06T06:46:55.355 INFO:teuthology.orchestra.run.smithi170.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.4 MB/s | 2.4 MB 00:00 2024-06-06T06:46:55.413 INFO:teuthology.orchestra.run.smithi073.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 8.0 MB/s | 2.4 MB 00:00 2024-06-06T06:46:55.439 INFO:teuthology.orchestra.run.smithi170.stdout:(159/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.2 MB/s | 3.1 MB 00:00 2024-06-06T06:46:55.445 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:46:55.445 INFO:teuthology.orchestra.run.smithi170.stdout:Total 11 MB/s | 225 MB 00:20 2024-06-06T06:46:56.633 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-06-06T06:46:56.790 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-06-06T06:46:56.790 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-06-06T06:46:57.194 INFO:teuthology.orchestra.run.smithi136.stdout:Transaction test succeeded. 2024-06-06T06:46:57.194 INFO:teuthology.orchestra.run.smithi136.stdout:Running transaction 2024-06-06T06:46:58.021 INFO:teuthology.orchestra.run.smithi073.stdout:(159/159): ceph-test-19.0.0-2946.g3d89accf.el9. 3.4 MB/s | 48 MB 00:14 2024-06-06T06:46:58.027 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:46:58.027 INFO:teuthology.orchestra.run.smithi073.stdout:Total 9.9 MB/s | 225 MB 00:22 2024-06-06T06:46:58.940 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-06-06T06:46:58.941 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-06-06T06:46:59.279 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-06-06T06:46:59.434 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-06-06T06:46:59.434 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-06-06T06:47:01.570 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-06-06T06:47:01.570 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-06-06T06:47:03.596 INFO:teuthology.orchestra.run.smithi136.stdout: Preparing : 1/1 2024-06-06T06:47:03.703 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T06:47:03.744 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T06:47:03.813 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T06:47:03.845 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T06:47:03.871 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T06:47:03.920 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T06:47:04.254 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T06:47:04.364 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:04.485 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:04.515 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:04.607 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:04.636 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:04.677 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T06:47:04.877 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T06:47:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T06:47:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:05.076 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:05.097 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:05.160 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:05.182 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:05.203 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:05.311 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-06-06T06:47:05.392 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:05.393 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T06:47:05.431 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T06:47:05.509 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T06:47:05.543 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T06:47:05.561 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:05.569 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T06:47:05.615 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T06:47:05.729 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:05.790 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:05.963 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T06:47:05.974 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:06.070 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:06.192 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:06.221 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:06.288 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T06:47:06.305 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:06.335 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:06.384 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T06:47:06.422 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T06:47:06.457 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T06:47:06.465 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T06:47:06.530 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T06:47:06.567 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T06:47:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T06:47:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T06:47:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T06:47:06.748 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T06:47:06.800 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T06:47:06.819 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T06:47:06.840 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T06:47:06.874 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T06:47:06.875 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T06:47:06.902 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:06.950 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T06:47:06.985 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T06:47:07.006 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:07.028 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:07.029 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T06:47:07.058 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T06:47:07.116 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:07.127 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T06:47:07.138 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:07.160 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:07.172 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T06:47:07.246 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T06:47:07.340 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:07.379 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T06:47:07.433 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T06:47:07.514 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:07.702 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:07.762 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:07.867 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:07.939 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:07.948 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T06:47:07.971 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-06-06T06:47:08.027 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T06:47:08.057 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T06:47:08.076 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-06T06:47:08.105 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T06:47:08.120 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-06T06:47:08.151 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T06:47:08.153 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:08.185 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-06T06:47:08.189 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T06:47:08.216 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-06T06:47:08.230 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T06:47:08.242 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-06T06:47:08.269 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T06:47:08.297 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-06T06:47:08.374 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T06:47:08.387 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T06:47:08.411 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T06:47:08.446 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T06:47:08.450 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T06:47:08.504 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T06:47:08.519 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T06:47:08.553 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T06:47:08.577 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T06:47:08.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T06:47:08.609 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T06:47:08.627 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-06T06:47:08.630 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T06:47:08.645 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T06:47:08.658 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T06:47:08.691 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T06:47:08.727 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T06:47:08.736 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T06:47:08.752 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:08.779 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T06:47:08.781 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T06:47:08.819 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T06:47:08.821 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T06:47:08.862 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T06:47:08.873 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:08.895 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:08.938 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T06:47:08.956 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T06:47:08.973 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T06:47:08.988 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:09.006 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T06:47:09.016 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:09.016 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T06:47:09.045 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T06:47:09.047 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-06T06:47:09.047 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T06:47:09.115 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T06:47:09.141 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T06:47:09.161 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T06:47:09.181 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T06:47:09.222 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T06:47:09.235 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T06:47:09.242 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-06T06:47:09.255 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T06:47:09.280 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T06:47:09.297 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-06T06:47:09.325 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:09.365 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T06:47:09.378 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T06:47:09.429 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:09.437 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T06:47:09.451 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:09.488 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T06:47:09.513 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-06T06:47:09.544 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:09.565 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-06T06:47:09.570 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T06:47:09.598 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T06:47:09.633 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T06:47:09.671 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T06:47:09.703 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T06:47:09.733 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T06:47:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:09.767 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T06:47:09.809 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T06:47:09.850 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T06:47:09.885 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T06:47:09.932 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-06T06:47:09.970 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T06:47:09.971 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T06:47:10.015 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T06:47:10.046 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T06:47:10.059 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T06:47:10.074 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:10.087 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T06:47:10.093 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T06:47:10.125 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T06:47:10.134 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-06T06:47:10.164 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T06:47:10.199 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T06:47:10.202 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T06:47:10.244 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T06:47:10.247 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T06:47:10.254 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:10.344 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:10.394 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T06:47:10.447 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T06:47:10.494 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T06:47:10.555 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-06T06:47:10.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T06:47:10.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T06:47:10.634 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T06:47:10.663 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-06T06:47:10.681 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T06:47:10.717 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T06:47:10.760 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T06:47:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-06T06:47:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-06T06:47:10.802 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T06:47:10.834 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-06T06:47:10.900 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-06T06:47:10.934 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-06T06:47:10.963 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T06:47:10.969 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T06:47:10.977 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-06T06:47:11.010 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-06T06:47:11.011 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T06:47:11.038 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-06T06:47:11.044 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T06:47:11.096 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T06:47:11.106 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-06T06:47:11.138 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T06:47:11.159 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T06:47:11.160 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-06T06:47:11.191 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-06T06:47:11.208 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T06:47:11.232 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T06:47:11.235 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-06T06:47:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T06:47:11.263 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T06:47:11.306 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T06:47:11.309 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-06T06:47:11.336 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T06:47:11.339 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T06:47:11.345 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-06T06:47:11.361 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T06:47:11.389 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-06T06:47:11.389 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T06:47:11.417 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-06T06:47:11.458 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T06:47:11.486 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-06T06:47:11.531 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-06T06:47:11.565 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T06:47:11.587 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T06:47:11.595 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T06:47:11.605 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-06T06:47:11.635 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T06:47:11.663 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T06:47:11.674 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T06:47:11.698 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T06:47:11.736 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T06:47:11.737 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-06T06:47:11.768 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T06:47:11.792 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-06T06:47:11.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T06:47:11.833 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T06:47:11.866 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T06:47:11.907 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T06:47:11.942 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T06:47:12.029 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T06:47:12.087 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T06:47:12.121 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T06:47:12.168 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T06:47:12.265 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T06:47:12.308 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-06T06:47:12.322 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T06:47:12.326 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T06:47:12.387 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-06T06:47:12.417 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-06T06:47:12.445 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-06T06:47:12.450 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T06:47:12.494 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-06T06:47:12.532 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-06T06:47:12.573 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-06T06:47:12.716 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-06T06:47:12.776 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-06T06:47:12.830 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-06T06:47:12.894 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-06T06:47:12.927 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-06T06:47:12.963 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-06T06:47:13.009 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-06T06:47:13.049 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T06:47:13.063 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-06T06:47:13.096 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T06:47:13.105 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-06T06:47:13.130 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T06:47:13.147 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-06T06:47:13.232 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T06:47:13.281 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-06T06:47:13.285 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T06:47:13.330 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T06:47:13.332 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-06T06:47:13.374 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-06T06:47:13.418 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T06:47:13.458 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T06:47:13.467 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-06T06:47:13.508 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-06T06:47:13.558 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-06T06:47:13.589 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-06T06:47:13.614 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-06T06:47:13.633 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T06:47:13.672 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T06:47:13.693 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T06:47:13.719 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-06T06:47:13.750 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T06:47:13.773 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T06:47:13.800 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T06:47:13.834 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T06:47:13.839 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-06T06:47:13.871 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-06T06:47:13.891 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-06T06:47:13.935 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-06T06:47:13.973 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-06T06:47:14.004 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-06T06:47:14.034 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-06T06:47:14.069 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-06T06:47:14.102 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-06T06:47:14.143 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-06T06:47:14.145 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T06:47:14.178 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-06T06:47:14.188 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:14.264 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-06T06:47:14.279 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:14.309 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T06:47:14.317 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-06T06:47:14.346 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T06:47:14.348 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-06T06:47:14.395 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-06T06:47:14.410 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T06:47:14.491 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-06T06:47:14.528 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T06:47:14.540 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-06T06:47:14.821 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T06:47:14.849 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:14.920 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:14.942 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T06:47:15.250 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-06T06:47:15.296 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-06T06:47:15.330 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-06T06:47:15.428 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-06T06:47:15.486 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-06T06:47:15.538 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-06T06:47:15.633 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-06T06:47:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-06T06:47:15.789 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T06:47:15.841 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-06T06:47:15.872 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-06T06:47:15.875 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T06:47:15.904 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T06:47:15.937 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T06:47:15.951 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-06T06:47:16.253 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T06:47:16.300 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:16.381 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:16.413 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T06:47:16.458 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T06:47:16.588 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-06T06:47:16.702 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-06T06:47:16.931 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T06:47:16.961 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:17.031 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:17.054 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T06:47:17.232 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:17.272 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:17.833 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:17.875 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:17.945 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-06T06:47:17.965 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:17.965 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T06:47:17.965 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T06:47:17.965 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:17.993 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:18.027 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-06T06:47:18.055 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-06T06:47:18.080 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-06T06:47:18.399 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-06T06:47:18.443 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-06T06:47:18.565 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-06T06:47:18.601 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-06T06:47:19.072 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-06T06:47:19.096 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:19.159 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-06T06:47:19.181 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-06T06:47:19.298 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:19.334 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:19.888 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:19.929 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:20.020 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:20.020 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T06:47:20.020 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T06:47:20.020 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:20.047 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:21.466 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:21.503 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:22.046 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-06T06:47:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:22.174 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-06T06:47:22.174 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-06T06:47:22.174 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-06T06:47:22.174 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:22.208 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /sys 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /proc 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /mnt 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /var/tmp 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /home 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /root 2024-06-06T06:47:29.740 INFO:teuthology.orchestra.run.smithi136.stdout:skipping the directory /tmp 2024-06-06T06:47:29.741 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:29.920 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:29.920 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T06:47:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:31.416 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:31.443 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:31.444 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:31.444 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T06:47:31.444 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:31.444 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:31.444 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:31.667 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:31.690 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:31.690 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:31.690 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T06:47:31.690 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:31.690 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:31.691 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /sys 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /proc 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /mnt 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /var/tmp 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /home 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /root 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout:skipping the directory /tmp 2024-06-06T06:47:32.052 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:32.142 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:32.165 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:32.165 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:32.165 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T06:47:32.166 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:32.166 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:32.166 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:32.219 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:32.234 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:32.234 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:32.234 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T06:47:32.234 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:32.273 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T06:47:32.329 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T06:47:32.386 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T06:47:32.570 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T06:47:32.618 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T06:47:32.696 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T06:47:32.783 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T06:47:32.932 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T06:47:32.985 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T06:47:33.016 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T06:47:33.044 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T06:47:33.455 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:33.492 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:33.524 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T06:47:33.586 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T06:47:33.620 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:33.724 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:33.750 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:33.947 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-06-06T06:47:33.948 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:33.974 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:33.996 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:33.997 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:33.997 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T06:47:33.997 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:33.997 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:33.997 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:34.109 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:34.125 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-06T06:47:34.125 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:34.125 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-06T06:47:34.125 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:34.441 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:34.478 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T06:47:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:34.479 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:34.479 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:34.571 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T06:47:34.628 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T06:47:34.643 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:34.684 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T06:47:34.685 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:34.869 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T06:47:34.917 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T06:47:34.995 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T06:47:35.083 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T06:47:35.223 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T06:47:35.275 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T06:47:35.307 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T06:47:35.335 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T06:47:35.587 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:35.613 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-06T06:47:35.614 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:35.614 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-06T06:47:35.614 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:35.614 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-06T06:47:35.614 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:35.648 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:35.685 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:35.738 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:35.767 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:35.786 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T06:47:35.837 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:35.860 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-06T06:47:35.861 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:35.890 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:35.956 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T06:47:35.987 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:36.012 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:36.063 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:36.075 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:36.172 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T06:47:36.228 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T06:47:36.314 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:36.337 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-06T06:47:36.337 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:36.338 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-06T06:47:36.338 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:36.338 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-06T06:47:36.338 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:36.435 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-06T06:47:36.483 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-06T06:47:36.539 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-06T06:47:36.722 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-06T06:47:36.772 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-06T06:47:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-06T06:47:36.917 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:36.929 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-06T06:47:36.950 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:37.070 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-06T06:47:37.122 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T06:47:37.131 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-06T06:47:37.149 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:37.163 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:37.173 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:37.191 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-06T06:47:37.242 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:37.264 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:37.264 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:37.265 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T06:47:37.265 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:37.309 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:37.332 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:37.635 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:37.653 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:37.658 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:37.681 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-06T06:47:37.721 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-06T06:47:37.782 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-06T06:47:37.813 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:37.909 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:37.933 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:38.035 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:38.196 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T06:47:38.227 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:38.250 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T06:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:38.251 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:38.295 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:38.306 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:38.412 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T06:47:38.472 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T06:47:38.834 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-06T06:47:38.865 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:39.368 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T06:47:39.400 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:39.422 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:39.422 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:39.422 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T06:47:39.422 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:39.422 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:39.423 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:39.475 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:39.497 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:39.498 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:39.498 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T06:47:39.498 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:39.543 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:39.565 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:39.565 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:39.565 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T06:47:39.565 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:39.566 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:39.566 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:39.823 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-06T06:47:39.857 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:39.868 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:39.891 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:39.892 INFO:teuthology.orchestra.run.smithi170.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:39.892 INFO:teuthology.orchestra.run.smithi170.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T06:47:39.892 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:39.892 INFO:teuthology.orchestra.run.smithi170.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:39.892 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:39.958 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-06T06:47:40.111 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-06T06:47:40.151 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:40.174 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-06T06:47:40.174 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:40.175 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-06T06:47:40.175 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:40.175 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-06T06:47:40.175 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:40.227 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:40.238 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-06T06:47:40.336 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-06T06:47:40.417 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-06T06:47:41.312 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-06T06:47:41.344 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:41.367 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-06T06:47:41.367 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:41.368 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-06T06:47:41.368 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:41.368 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-06T06:47:41.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:41.417 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:41.440 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-06T06:47:41.441 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:41.441 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-06T06:47:41.441 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:41.485 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:41.508 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-06T06:47:41.508 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:41.508 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-06T06:47:41.508 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:41.508 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-06T06:47:41.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:41.819 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:41.842 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-06T06:47:41.842 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-06T06:47:41.842 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-06T06:47:41.843 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:41.843 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-06T06:47:41.843 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:42.172 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T06:47:42.213 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T06:47:42.254 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T06:47:42.286 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T06:47:42.313 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T06:47:42.506 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T06:47:42.552 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T06:47:42.585 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T06:47:42.616 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T06:47:42.646 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T06:47:42.692 INFO:teuthology.orchestra.run.smithi136.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:43.332 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:43.374 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:43.394 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:44.221 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T06:47:44.313 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T06:47:44.354 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T06:47:44.406 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T06:47:44.448 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T06:47:44.456 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T06:47:44.483 INFO:teuthology.orchestra.run.smithi136.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T06:47:44.505 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T06:47:44.508 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:44.533 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T06:47:44.537 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T06:47:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:44.560 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:44.564 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T06:47:44.645 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:44.677 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T06:47:44.678 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:44.751 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T06:47:44.762 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:44.787 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T06:47:44.794 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T06:47:44.800 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:44.800 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:44.827 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T06:47:44.867 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T06:47:44.897 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T06:47:44.958 INFO:teuthology.orchestra.run.smithi170.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:44.997 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:44.997 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:45.091 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:45.115 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T06:47:45.180 INFO:teuthology.orchestra.run.smithi136.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:45.526 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:45.566 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:45.585 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:46.347 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-06T06:47:46.389 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-06T06:47:46.414 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T06:47:46.429 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-06T06:47:46.461 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-06T06:47:46.489 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-06T06:47:46.504 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T06:47:46.546 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T06:47:46.631 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T06:47:46.666 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T06:47:46.667 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-06T06:47:46.691 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:46.716 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T06:47:46.718 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-06T06:47:46.751 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:46.751 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:46.752 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-06T06:47:46.791 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-06T06:47:46.821 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-06T06:47:46.844 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:46.868 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:46.877 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T06:47:46.877 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:46.961 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:46.986 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T06:47:46.992 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:46.992 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:47.172 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:47.172 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:47.265 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:47.298 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T06:47:47.364 INFO:teuthology.orchestra.run.smithi170.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:47.495 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:47.532 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:47.552 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-06T06:47:48.375 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-06T06:47:48.462 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-06T06:47:48.504 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-06T06:47:48.594 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-06T06:47:48.624 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-06T06:47:48.657 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:48.682 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-06T06:47:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:48.709 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:48.793 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-06T06:47:48.827 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-06T06:47:48.827 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:48.894 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-06T06:47:48.919 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-06T06:47:48.927 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:48.927 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:49.121 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-06T06:47:49.122 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:49.205 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-06T06:47:49.230 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-06T06:47:49.296 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T06:47:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T06:47:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T06:47:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T06:47:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T06:47:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T06:47:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T06:47:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T06:47:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T06:47:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T06:47:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T06:47:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T06:47:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T06:47:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T06:47:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T06:47:52.927 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T06:47:52.928 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T06:47:52.929 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T06:47:52.942 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T06:47:52.943 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T06:47:52.944 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T06:47:52.945 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T06:47:52.946 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T06:47:52.947 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T06:47:52.948 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T06:47:52.949 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T06:47:52.950 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T06:47:52.951 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:52.952 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T06:47:52.953 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T06:47:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T06:47:53.484 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:53.484 INFO:teuthology.orchestra.run.smithi136.stdout:Upgraded: 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout:Installed: 2024-06-06T06:47:53.485 INFO:teuthology.orchestra.run.smithi136.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.486 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.487 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T06:47:53.488 INFO:teuthology.orchestra.run.smithi136.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T06:47:53.489 INFO:teuthology.orchestra.run.smithi136.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T06:47:53.490 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.491 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T06:47:53.492 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T06:47:53.493 INFO:teuthology.orchestra.run.smithi136.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T06:47:53.494 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T06:47:53.495 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T06:47:53.496 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T06:47:53.497 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-06-06T06:47:53.498 INFO:teuthology.orchestra.run.smithi136.stdout:Complete! 2024-06-06T06:47:53.928 DEBUG:teuthology.parallel:result is None 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-06T06:47:54.870 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-06T06:47:54.871 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-06T06:47:54.872 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-06T06:47:54.884 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-06T06:47:54.884 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-06T06:47:54.884 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-06T06:47:54.885 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-06T06:47:54.886 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-06T06:47:54.887 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-06T06:47:54.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-06T06:47:54.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-06T06:47:54.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-06T06:47:54.891 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-06T06:47:54.892 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-06T06:47:54.893 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-06T06:47:54.894 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-06T06:47:54.895 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-06T06:47:55.763 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T06:47:55.763 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:55.763 INFO:teuthology.orchestra.run.smithi170.stdout:Upgraded: 2024-06-06T06:47:55.763 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T06:47:55.764 INFO:teuthology.orchestra.run.smithi170.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.765 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.766 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.767 INFO:teuthology.orchestra.run.smithi170.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T06:47:55.768 INFO:teuthology.orchestra.run.smithi170.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T06:47:55.769 INFO:teuthology.orchestra.run.smithi170.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T06:47:55.770 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T06:47:55.771 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:55.772 INFO:teuthology.orchestra.run.smithi170.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.773 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T06:47:55.774 INFO:teuthology.orchestra.run.smithi170.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T06:47:55.775 INFO:teuthology.orchestra.run.smithi170.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T06:47:55.776 INFO:teuthology.orchestra.run.smithi170.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T06:47:55.776 INFO:teuthology.orchestra.run.smithi170.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T06:47:55.776 INFO:teuthology.orchestra.run.smithi170.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T06:47:55.776 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:47:55.776 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-06-06T06:47:56.186 DEBUG:teuthology.parallel:result is None 2024-06-06T06:47:57.823 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-06T06:47:57.823 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.824 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.825 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.826 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-06T06:47:57.827 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-06T06:47:57.828 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-06T06:47:57.829 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.14-1.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-06T06:47:57.830 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-06T06:47:57.831 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-06T06:47:57.832 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:57.833 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-06T06:47:57.834 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-06T06:47:57.835 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.836 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-3.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-06T06:47:57.837 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:47:57.838 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-06-06T06:47:58.254 DEBUG:teuthology.parallel:result is None 2024-06-06T06:47:58.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:47:58.395 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T06:47:58.417 INFO:teuthology.orchestra.run.smithi073.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.418 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.418 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T06:47:58.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:47:58.555 DEBUG:teuthology.orchestra.run.smithi136:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T06:47:58.582 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.582 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.582 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T06:47:58.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:47:58.802 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-06T06:47:58.828 INFO:teuthology.orchestra.run.smithi170.stdout:19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.829 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-06T06:47:58.829 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-06T06:47:58.830 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-06T06:47:58.831 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:47:58.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T06:47:58.859 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:47:58.859 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T06:47:58.891 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:47:58.892 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-06T06:47:58.923 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-06T06:47:58.923 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:47:58.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T06:47:58.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T06:47:59.013 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:47:59.013 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T06:47:59.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T06:47:59.113 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:47:59.113 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/daemon-helper 2024-06-06T06:47:59.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-06T06:47:59.210 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-06T06:47:59.210 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:47:59.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T06:47:59.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T06:47:59.302 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:47:59.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T06:47:59.330 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T06:47:59.399 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:47:59.399 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-06T06:47:59.428 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-06T06:47:59.495 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-06T06:47:59.496 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:47:59.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T06:47:59.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T06:47:59.589 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:47:59.589 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T06:47:59.618 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T06:47:59.686 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:47:59.687 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/usr/bin/stdin-killer 2024-06-06T06:47:59.716 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-06T06:47:59.785 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-06T06:47:59.878 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T06:47:59.878 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:47:59.879 INFO:tasks.cephadm:Cluster fsid is b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:47:59.879 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-06T06:47:59.879 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.73', 'mon.b': '172.21.15.136', 'mon.c': '172.21.15.170'} 2024-06-06T06:47:59.879 INFO:tasks.cephadm:First mon is mon.a on smithi073 2024-06-06T06:47:59.880 INFO:tasks.cephadm:First mgr is x 2024-06-06T06:47:59.880 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-06T06:47:59.880 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname $(hostname -s) 2024-06-06T06:47:59.908 DEBUG:teuthology.orchestra.run.smithi136:> sudo hostname $(hostname -s) 2024-06-06T06:47:59.937 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-06-06T06:47:59.966 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-06T06:47:59.967 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:48:00.184 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-06T06:48:00.402 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:48:00.404 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-06T06:48:00.404 DEBUG:teuthology.orchestra.run.smithi073:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:00.820 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 06:48 /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:00.820 DEBUG:teuthology.orchestra.run.smithi136:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.225 INFO:teuthology.orchestra.run.smithi136.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 06:48 /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.225 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.852 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 6 06:48 /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.853 DEBUG:teuthology.orchestra.run.smithi073:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.872 DEBUG:teuthology.orchestra.run.smithi136:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.889 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-06T06:48:01.924 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-06T06:48:01.924 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T06:48:01.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T06:48:01.932 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-06T06:48:02.734 INFO:teuthology.orchestra.run.smithi073.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T06:48:02.835 INFO:teuthology.orchestra.run.smithi136.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T06:48:02.843 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "repo_digests": [ 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-06-06T06:48:33.850 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-06T06:48:35.710 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-06-06T06:48:35.711 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-06-06T06:48:38.338 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T06:48:38.338 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T06:48:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-06T06:48:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "repo_digests": [ 2024-06-06T06:48:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-06T06:48:38.339 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T06:48:38.339 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T06:48:38.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph 2024-06-06T06:48:38.411 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph 2024-06-06T06:48:38.445 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-06-06T06:48:38.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /etc/ceph 2024-06-06T06:48:38.503 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /etc/ceph 2024-06-06T06:48:38.531 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-06-06T06:48:38.558 INFO:tasks.cephadm:Writing seed config... 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-06T06:48:38.560 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-06T06:48:38.561 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-06T06:48:38.562 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-06T06:48:38.563 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-06T06:48:38.564 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:48:38.564 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-06T06:48:38.579 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b6cc6aaa-23d0-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-06T06:48:38.579 DEBUG:teuthology.orchestra.run.smithi073:mon.a> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service 2024-06-06T06:48:38.622 DEBUG:teuthology.orchestra.run.smithi073:mgr.x> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-06T06:48:38.664 INFO:tasks.cephadm:Bootstrapping... 2024-06-06T06:48:38.664 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.73 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-06T06:48:38.854 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-06-06T06:48:38.855 INFO:teuthology.orchestra.run.smithi073.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', 'b6cc6aaa-23d0-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.73', '--skip-admin-label'] 2024-06-06T06:48:38.855 INFO:teuthology.orchestra.run.smithi073.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-06T06:48:38.855 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying podman|docker is present... 2024-06-06T06:48:38.880 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stdout 5.0.2 2024-06-06T06:48:38.880 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying lvm2 is present... 2024-06-06T06:48:38.880 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying time synchronization is in place... 2024-06-06T06:48:38.888 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T06:48:38.888 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T06:48:38.895 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T06:48:38.895 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:48:38.901 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout enabled 2024-06-06T06:48:38.908 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout active 2024-06-06T06:48:38.908 INFO:teuthology.orchestra.run.smithi073.stdout:Unit chronyd.service is enabled and running 2024-06-06T06:48:38.908 INFO:teuthology.orchestra.run.smithi073.stdout:Repeating the final host check... 2024-06-06T06:48:38.932 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stdout 5.0.2 2024-06-06T06:48:38.932 INFO:teuthology.orchestra.run.smithi073.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-06T06:48:38.932 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl is present 2024-06-06T06:48:38.932 INFO:teuthology.orchestra.run.smithi073.stdout:lvcreate is present 2024-06-06T06:48:38.938 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-06T06:48:38.938 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-06T06:48:38.945 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-06T06:48:38.945 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:48:38.951 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout enabled 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout active 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Unit chronyd.service is enabled and running 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Host looks OK 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Cluster fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Acquiring lock 140091112646016 on /run/cephadm/b6cc6aaa-23d0-11ef-bc9b-c7b262605968.lock 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Lock 140091112646016 acquired on /run/cephadm/b6cc6aaa-23d0-11ef-bc9b-c7b262605968.lock 2024-06-06T06:48:38.958 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying IP 172.21.15.73 port 3300 ... 2024-06-06T06:48:38.959 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying IP 172.21.15.73 port 6789 ... 2024-06-06T06:48:38.959 INFO:teuthology.orchestra.run.smithi073.stdout:Base mon IP(s) is [172.21.15.73:3300, 172.21.15.73:6789], mon addrv is [v2:172.21.15.73:3300,v1:172.21.15.73:6789] 2024-06-06T06:48:38.962 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.73 metric 100 2024-06-06T06:48:38.962 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.73 metric 100 2024-06-06T06:48:38.964 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-06T06:48:38.965 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-06-06T06:48:38.965 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-06-06T06:48:38.967 INFO:teuthology.orchestra.run.smithi073.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-06T06:48:38.968 INFO:teuthology.orchestra.run.smithi073.stdout:Mon IP `172.21.15.73` is in CIDR network `172.21.0.0/20` 2024-06-06T06:48:38.968 INFO:teuthology.orchestra.run.smithi073.stdout:Mon IP `172.21.15.73` is in CIDR network `172.21.0.0/20` 2024-06-06T06:48:38.968 INFO:teuthology.orchestra.run.smithi073.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-06T06:48:38.968 INFO:teuthology.orchestra.run.smithi073.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-06T06:48:38.969 INFO:teuthology.orchestra.run.smithi073.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Getting image source signatures 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-06T06:48:39.441 INFO:teuthology.orchestra.run.smithi073.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-06T06:48:40.404 INFO:teuthology.orchestra.run.smithi073.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T06:48:40.404 INFO:teuthology.orchestra.run.smithi073.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T06:48:40.404 INFO:teuthology.orchestra.run.smithi073.stdout:Extracting ceph user uid/gid from container image... 2024-06-06T06:48:41.268 INFO:teuthology.orchestra.run.smithi073.stdout:stat: stdout 167 167 2024-06-06T06:48:41.268 INFO:teuthology.orchestra.run.smithi073.stdout:Creating initial keys... 2024-06-06T06:48:42.132 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph-authtool: stdout AQDJW2FmZOVeIxAABA09y/QQJYtycFiEy7yy7w== 2024-06-06T06:48:42.996 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph-authtool: stdout AQDKW2Fm5mXAGhAAN5eENcdFCSf5lREZu/+Taw== 2024-06-06T06:48:43.968 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph-authtool: stdout AQDLW2FmdTyrEhAALfGgKb465Vom5YuS1OTfZg== 2024-06-06T06:48:43.969 INFO:teuthology.orchestra.run.smithi073.stdout:Creating initial monmap... 2024-06-06T06:48:44.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T06:48:44.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool for a [v2:172.21.15.73:3300,v1:172.21.15.73:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = quincy 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: set fsid to b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:48:44.810 INFO:teuthology.orchestra.run.smithi073.stdout:Creating mon... 2024-06-06T06:48:45.748 INFO:teuthology.orchestra.run.smithi073.stdout:create mon.a on 2024-06-06T06:48:45.961 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-06T06:48:46.112 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-06T06:48:46.289 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target. 2024-06-06T06:48:46.289 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target. 2024-06-06T06:48:46.530 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a 2024-06-06T06:48:46.530 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-06T06:48:46.692 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target.wants/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@.service. 2024-06-06T06:48:47.315 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T06:48:47.315 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout disabled 2024-06-06T06:48:47.322 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T06:48:47.322 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:48:47.322 INFO:teuthology.orchestra.run.smithi073.stdout:firewalld.service is not enabled 2024-06-06T06:48:47.322 INFO:teuthology.orchestra.run.smithi073.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T06:48:47.323 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mon to start... 2024-06-06T06:48:47.323 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mon... 2024-06-06T06:48:48.358 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:48:48 smithi073 ceph-mon[31929]: from='client.? 172.21.15.73:0/779067742' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-06T06:48:48.360 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout cluster: 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout id: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout services: 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.546188s) 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-06T06:48:48.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout data: 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout pgs: 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.718+0000 7f97a17b4700 1 Processor -- start 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.718+0000 7f97a17b4700 1 -- start start 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.718+0000 7f97a17b4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.718+0000 7f97a17b4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f979c07bdf0 con 0x7f979c106600 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f979affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f979affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33178/0 (socket says 172.21.15.73:33178) 2024-06-06T06:48:48.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f979affd700 1 -- 172.21.15.73:0/3857235128 learned_addr learned my addr 172.21.15.73:0/3857235128 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f979affd700 1 -- 172.21.15.73:0/3857235128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c106f60 con 0x7f979c106600 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f979affd700 1 --2- 172.21.15.73:0/3857235128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1937ec84e832c8f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f9799ffb700 1 -- 172.21.15.73:0/3857235128 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c004160 con 0x7f979c106600 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f9799ffb700 1 -- 172.21.15.73:0/3857235128 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c004300 con 0x7f979c106600 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.719+0000 7f9799ffb700 1 -- 172.21.15.73:0/3857235128 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c0036a0 con 0x7f979c106600 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 -- 172.21.15.73:0/3857235128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 msgr2=0x7f979c106a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 --2- 172.21.15.73:0/3857235128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f978c0098a0 tx=0x7f978c009bb0 comp rx=0 tx=0).stop 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 -- 172.21.15.73:0/3857235128 shutdown_connections 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 --2- 172.21.15.73:0/3857235128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c106a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 -- 172.21.15.73:0/3857235128 >> 172.21.15.73:0/3857235128 conn(0x7f979c1021e0 msgr2=0x7f979c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 -- 172.21.15.73:0/3857235128 shutdown_connections 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 -- 172.21.15.73:0/3857235128 wait complete. 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.720+0000 7f97a17b4700 1 Processor -- start 2024-06-06T06:48:48.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f97a17b4700 1 -- start start 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f97a17b4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f97a17b4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f979c107490 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f979affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f979affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33180/0 (socket says 172.21.15.73:33180) 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f979affd700 1 -- 172.21.15.73:0/779067742 learned_addr learned my addr 172.21.15.73:0/779067742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f979affd700 1 -- 172.21.15.73:0/779067742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f979c19db20 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f979affd700 1 --2- 172.21.15.73:0/779067742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f978c000c00 tx=0x7f978c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c003db0 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f978c011630 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f978c011950 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f979c19ddb0 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.721+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f979c1a0aa0 con 0x7f979c106600 2024-06-06T06:48:48.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.722+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f978c011af0 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.722+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f978c01c450 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.722+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f979c063af0 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.725+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f978c004260 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.796+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f979c1a0f90 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.796+0000 7f9783fff700 1 -- 172.21.15.73:0/779067742 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f978c01b450 con 0x7f979c106600 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 msgr2=0x7f979c19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 --2- 172.21.15.73:0/779067742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f978c000c00 tx=0x7f978c004750 comp rx=0 tx=0).stop 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 shutdown_connections 2024-06-06T06:48:48.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 --2- 172.21.15.73:0/779067742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f979c106600 0x7f979c19d5e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:48:48.371 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 >> 172.21.15.73:0/779067742 conn(0x7f979c1021e0 msgr2=0x7f979c193c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:48:48.371 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 shutdown_connections 2024-06-06T06:48:48.371 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:47.797+0000 7f97a17b4700 1 -- 172.21.15.73:0/779067742 wait complete. 2024-06-06T06:48:48.371 INFO:teuthology.orchestra.run.smithi073.stdout:mon is available 2024-06-06T06:48:48.371 INFO:teuthology.orchestra.run.smithi073.stdout:Assimilating anything we can from ceph.conf... 2024-06-06T06:48:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:48:49 smithi073 ceph-mon[31929]: from='client.? 172.21.15.73:0/1012630488' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T06:48:49.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:48:49 smithi073 ceph-mon[31929]: from='client.? 172.21.15.73:0/1012630488' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-06T06:48:51.111 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:51.111 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T06:48:51.111 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout fsid = b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.73:3300,v1:172.21.15.73:6789] 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T06:48:51.112 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b23f13700 1 Processor -- start 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b23f13700 1 -- start start 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b23f13700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b23f13700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c074090 con 0x7f1b1c072990 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b21caf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b21caf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33192/0 (socket says 172.21.15.73:33192) 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.761+0000 7f1b21caf700 1 -- 172.21.15.73:0/3912194705 learned_addr learned my addr 172.21.15.73:0/3912194705 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.762+0000 7f1b21caf700 1 -- 172.21.15.73:0/3912194705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c0732f0 con 0x7f1b1c072990 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.762+0000 7f1b21caf700 1 --2- 172.21.15.73:0/3912194705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098a0 tx=0x7f1b18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d05979b20337968 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.762+0000 7f1b20cad700 1 -- 172.21.15.73:0/3912194705 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b18004160 con 0x7f1b1c072990 2024-06-06T06:48:51.113 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.762+0000 7f1b20cad700 1 -- 172.21.15.73:0/3912194705 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18004300 con 0x7f1b1c072990 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.762+0000 7f1b20cad700 1 -- 172.21.15.73:0/3912194705 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b180036a0 con 0x7f1b1c072990 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- 172.21.15.73:0/3912194705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 msgr2=0x7f1b1c072db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 --2- 172.21.15.73:0/3912194705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098a0 tx=0x7f1b18009bb0 comp rx=0 tx=0).stop 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- 172.21.15.73:0/3912194705 shutdown_connections 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 --2- 172.21.15.73:0/3912194705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c072db0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- 172.21.15.73:0/3912194705 >> 172.21.15.73:0/3912194705 conn(0x7f1b1c06e350 msgr2=0x7f1b1c070790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- 172.21.15.73:0/3912194705 shutdown_connections 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- 172.21.15.73:0/3912194705 wait complete. 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 Processor -- start 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.763+0000 7f1b23f13700 1 -- start start 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b23f13700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:48:51.114 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b23f13700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b1c073820 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b21caf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b21caf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33206/0 (socket says 172.21.15.73:33206) 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b21caf700 1 -- 172.21.15.73:0/1012630488 learned_addr learned my addr 172.21.15.73:0/1012630488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b21caf700 1 -- 172.21.15.73:0/1012630488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b1c1a4130 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b21caf700 1 --2- 172.21.15.73:0/1012630488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1b18000c00 tx=0x7f1b18004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b18003db0 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b18011630 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b18011950 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b1c1a43c0 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.764+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b1c1a70b0 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.765+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1b18011af0 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.765+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b1801c450 con 0x7f1b1c072990 2024-06-06T06:48:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.765+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b1c113fa0 con 0x7f1b1c072990 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.768+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f1b18004260 con 0x7f1b1c072990 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.834+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f1b1c02de20 con 0x7f1b1c072990 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.837+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f1b1801b450 con 0x7f1b1c072990 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.838+0000 7f1b12ffd700 1 -- 172.21.15.73:0/1012630488 <== mon.0 v2:172.21.15.73:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1b1801b630 con 0x7f1b1c072990 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 msgr2=0x7f1b1c1a3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 --2- 172.21.15.73:0/1012630488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1b18000c00 tx=0x7f1b18004750 comp rx=0 tx=0).stop 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 shutdown_connections 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 --2- 172.21.15.73:0/1012630488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b1c072990 0x7f1b1c1a3bf0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 >> 172.21.15.73:0/1012630488 conn(0x7f1b1c06e350 msgr2=0x7f1b1c070400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 shutdown_connections 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:48.839+0000 7f1b23f13700 1 -- 172.21.15.73:0/1012630488 wait complete. 2024-06-06T06:48:51.116 INFO:teuthology.orchestra.run.smithi073.stdout:Generating new minimal ceph.conf... 2024-06-06T06:48:54.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:48:54 smithi073 ceph-mon[31929]: from='client.? 172.21.15.73:0/1891621830' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7faca08f1700 1 Processor -- start 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7faca08f1700 1 -- start start 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7faca08f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7faca08f1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c07be10 con 0x7fac9c1087f0 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7fac9a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7fac9a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33220/0 (socket says 172.21.15.73:33220) 2024-06-06T06:49:00.362 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.176+0000 7fac9a59c700 1 -- 172.21.15.73:0/1238816779 learned_addr learned my addr 172.21.15.73:0/1238816779 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7fac9a59c700 1 -- 172.21.15.73:0/1238816779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c10b120 con 0x7fac9c1087f0 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7fac9a59c700 1 --2- 172.21.15.73:0/1238816779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0098a0 tx=0x7fac8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74af585d74e92376 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7fac9959a700 1 -- 172.21.15.73:0/1238816779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac8c004160 con 0x7fac9c1087f0 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7fac9959a700 1 -- 172.21.15.73:0/1238816779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac8c004300 con 0x7fac9c1087f0 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7fac9959a700 1 -- 172.21.15.73:0/1238816779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac8c011a80 con 0x7fac9c1087f0 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7faca08f1700 1 -- 172.21.15.73:0/1238816779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 msgr2=0x7fac9c10abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.177+0000 7faca08f1700 1 --2- 172.21.15.73:0/1238816779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fac8c0098a0 tx=0x7fac8c009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 -- 172.21.15.73:0/1238816779 shutdown_connections 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 --2- 172.21.15.73:0/1238816779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c10abe0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 -- 172.21.15.73:0/1238816779 >> 172.21.15.73:0/1238816779 conn(0x7fac9c1021e0 msgr2=0x7fac9c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 -- 172.21.15.73:0/1238816779 shutdown_connections 2024-06-06T06:49:00.363 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 -- 172.21.15.73:0/1238816779 wait complete. 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 Processor -- start 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.178+0000 7faca08f1700 1 -- start start 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7faca08f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7faca08f1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fac9c10c080 con 0x7fac9c1087f0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac9a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac9a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33232/0 (socket says 172.21.15.73:33232) 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac9a59c700 1 -- 172.21.15.73:0/1891621830 learned_addr learned my addr 172.21.15.73:0/1891621830 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac9a59c700 1 -- 172.21.15.73:0/1891621830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac9c19c1f0 con 0x7fac9c1087f0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac9a59c700 1 --2- 172.21.15.73:0/1891621830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac8c011a50 tx=0x7fac8c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac8c0039f0 con 0x7fac9c1087f0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fac8c003b90 con 0x7fac9c1087f0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac9c19c480 con 0x7fac9c1087f0 2024-06-06T06:49:00.364 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac8c01a600 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.179+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac9c19f170 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.180+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fac8c01acb0 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.180+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fac8c019700 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.180+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac9c063af0 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.183+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fac8c024a40 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.249+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fac9c19f340 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7fac8b7fe700 1 -- 172.21.15.73:0/1891621830 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fac8c024440 con 0x7fac9c1087f0 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 msgr2=0x7fac9c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 --2- 172.21.15.73:0/1891621830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fac8c011a50 tx=0x7fac8c004290 comp rx=0 tx=0).stop 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 shutdown_connections 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 --2- 172.21.15.73:0/1891621830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac9c1087f0 0x7fac9c19bcb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 >> 172.21.15.73:0/1891621830 conn(0x7fac9c1021e0 msgr2=0x7fac9c102d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:00.365 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 shutdown_connections 2024-06-06T06:49:00.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:48:54.250+0000 7faca08f1700 1 -- 172.21.15.73:0/1891621830 wait complete. 2024-06-06T06:49:00.366 INFO:teuthology.orchestra.run.smithi073.stdout:Restarting the monitor... 2024-06-06T06:49:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:00 smithi073 systemd[1]: Stopping Ceph mon.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:49:01.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[31925]: 2024-06-06T06:49:00.951+0000 7f4d3d02d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T06:49:01.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[31925]: 2024-06-06T06:49:00.951+0000 7f4d3d02d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-06T06:49:01.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 podman[32176]: 2024-06-06 06:49:01.104753974 +0000 UTC m=+0.566598316 container died ca6dd361017bf9584d5e7e32e0f878caca048ffbb0e7e7d06e7e76f079829dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T06:49:01.677 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 podman[32176]: 2024-06-06 06:49:01.467241181 +0000 UTC m=+0.929085514 container cleanup ca6dd361017bf9584d5e7e32e0f878caca048ffbb0e7e7d06e7e76f079829dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0) 2024-06-06T06:49:01.677 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 bash[32176]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a 2024-06-06T06:49:02.019 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 podman[32188]: 2024-06-06 06:49:01.676645148 +0000 UTC m=+0.565659741 container remove ca6dd361017bf9584d5e7e32e0f878caca048ffbb0e7e7d06e7e76f079829dda (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T06:49:02.019 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-06T06:49:02.019 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 systemd[1]: Stopped Ceph mon.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:49:02.019 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:01 smithi073 systemd[1]: Starting Ceph mon.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 podman[32284]: 2024-06-06 06:49:02.019166947 +0000 UTC m=+0.110780047 container create ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 podman[32284]: 2024-06-06 06:49:01.925231002 +0000 UTC m=+0.016844076 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 podman[32284]: 2024-06-06 06:49:02.221365162 +0000 UTC m=+0.312978216 container init ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 podman[32284]: 2024-06-06 06:49:02.224367293 +0000 UTC m=+0.315980350 container start ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: set uid:gid to 167:167 (ceph:ceph) 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: pidfile_write: ignore empty --pid-file 2024-06-06T06:49:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: load: jerasure load: lrc 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: RocksDB version: 7.9.2 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Git sha 0 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: DB SUMMARY 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: DB Session ID: BU36KI2Z8Y7CD8GXODUM 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: CURRENT file: CURRENT 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: IDENTITY file: IDENTITY 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104204 ; 2024-06-06T06:49:02.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.error_if_exists: 0 2024-06-06T06:49:02.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.create_if_missing: 0 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.paranoid_checks: 1 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.env: 0x561891816c60 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.fs: PosixFileSystem 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.info_log: 0x561893d89b20 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_file_opening_threads: 16 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.statistics: (nil) 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.use_fsync: 0 2024-06-06T06:49:02.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_log_file_size: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.keep_log_file_num: 1000 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.recycle_log_file_num: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_fallocate: 1 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_mmap_reads: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_mmap_writes: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.use_direct_reads: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.create_missing_column_families: 0 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.db_log_dir: 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.wal_dir: 2024-06-06T06:49:02.279 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.advise_random_on_open: 1 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.db_write_buffer_size: 0 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.write_buffer_manager: 0x561893d5a1e0 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.rate_limiter: (nil) 2024-06-06T06:49:02.280 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.wal_recovery_mode: 2 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enable_thread_tracking: 0 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enable_pipelined_write: 0 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.unordered_write: 0 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.row_cache: None 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.wal_filter: None 2024-06-06T06:49:02.281 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_ingest_behind: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.two_write_queues: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.manual_wal_flush: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.wal_compression: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.atomic_flush: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.log_readahead_size: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.best_efforts_recovery: 0 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-06T06:49:02.282 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.allow_data_in_errors: 0 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.db_host_id: __hostname__ 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_background_jobs: 2 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_background_compactions: -1 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_subcompactions: 1 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_total_wal_size: 0 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-06T06:49:02.283 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_open_files: -1 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bytes_per_sync: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_readahead_size: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_background_flushes: -1 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Compression algorithms supported: 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kZSTD supported: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kXpressCompression supported: 0 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kLZ4HCCompression supported: 1 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kZlibCompression supported: 1 2024-06-06T06:49:02.284 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kSnappyCompression supported: 1 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kLZ4Compression supported: 1 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: kBZip2Compression supported: 0 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.merge_operator: 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_filter: None 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_filter_factory: None 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.sst_partitioner_factory: None 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-06T06:49:02.285 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561893d89c40) 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: cache_index_and_filter_blocks: 1 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: pin_top_level_index_and_filter: 1 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: index_type: 0 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: data_block_index_type: 0 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: index_shortening: 1 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: checksum: 4 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: no_block_cache: 0 2024-06-06T06:49:02.286 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_cache: 0x561893d8b090 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_cache_name: BinnedLRUCache 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_cache_options: 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: capacity : 536870912 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: num_shard_bits : 4 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: strict_capacity_limit : 0 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: high_pri_pool_ratio: 0.000 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_cache_compressed: (nil) 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: persistent_cache: (nil) 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_size: 4096 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_size_deviation: 10 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_restart_interval: 16 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: index_block_restart_interval: 1 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: metadata_block_size: 4096 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: partition_filters: 0 2024-06-06T06:49:02.287 INFO:journalctl@ceph.mon.a.smithi073.stdout: use_delta_encoding: 1 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: filter_policy: bloomfilter 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: whole_key_filtering: 1 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: verify_compression: 0 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: read_amp_bytes_per_bit: 0 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: format_version: 5 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: enable_index_compression: 1 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: block_align: 0 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: max_auto_readahead_size: 262144 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: prepopulate_block_cache: 0 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: initial_auto_readahead_size: 8192 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout: num_file_reads_for_auto_readahead: 2 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.write_buffer_size: 33554432 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_write_buffer_number: 2 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression: NoCompression 2024-06-06T06:49:02.288 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression: Disabled 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.prefix_extractor: nullptr 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.num_levels: 7 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-06T06:49:02.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.level: 32767 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.strategy: 0 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.enabled: false 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-06T06:49:02.290 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.target_file_size_base: 67108864 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-06T06:49:02.291 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.arena_block_size: 1048576 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.disable_auto_compactions: 0 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-06T06:49:02.292 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.table_properties_collectors: 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.inplace_update_support: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.bloom_locality: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.max_successive_merges: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.paranoid_file_checks: 0 2024-06-06T06:49:02.293 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.force_consistency_checks: 1 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.report_bg_io_stats: 0 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.ttl: 2592000 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enable_blob_files: false 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.min_blob_size: 0 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_file_size: 268435456 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-06T06:49:02.294 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.blob_file_starting_level: 0 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 594688d7-801d-4892-82f3-236b33ce9940 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717656542253705, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717656542254692, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99425, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97345, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14030, "raw_average_key_size": 49, "raw_value_size": 89669, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717656542, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "594688d7-801d-4892-82f3-236b33ce9940", "db_session_id": "BU36KI2Z8Y7CD8GXODUM", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717656542254777, "job": 1, "event": "recovery_finished"} 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561893e7a000 2024-06-06T06:49:02.295 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: DB pointer 0x561893e64000 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: ** DB Stats ** 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: ** Compaction Stats [default] ** 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T06:49:02.296 INFO:journalctl@ceph.mon.a.smithi073.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: L0 2/0 98.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Sum 2/0 98.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 151.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: ** Compaction Stats [default] ** 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 151.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-06T06:49:02.297 INFO:journalctl@ceph.mon.a.smithi073.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: Cumulative compaction: 0.00 GB write, 31.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: Interval compaction: 0.00 GB write, 31.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: Block cache BinnedLRUCache@0x561893d8b090#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: starting mon.a rank 0 at public addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] at bind addrs [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:02.298 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???) e1 preinit fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).mds e1 new map 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).mds e1 print_map 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: e1 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: legacy client fscid: -1 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout: No filesystems configured 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:49:02.299 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-06T06:49:02.319 INFO:teuthology.orchestra.run.smithi073.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-06T06:49:02.632 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 bash[32284]: ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: monmap epoch 1 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: last_changed 2024-06-06T06:48:44.267203+0000 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: min_mon_release 19 (squid) 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: election_strategy: 1 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: fsmap 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: osdmap e1: 0 total, 0 up, 0 in 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 ceph-mon[32303]: mgrmap e1: no daemons active 2024-06-06T06:49:02.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:02 smithi073 systemd[1]: Started Ceph mon.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:49:03.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.733+0000 7efc8b1e6700 1 Processor -- start 2024-06-06T06:49:03.353 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.733+0000 7efc8b1e6700 1 -- start start 2024-06-06T06:49:03.353 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc8b1e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:03.354 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc8b1e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc84108780 con 0x7efc84107d90 2024-06-06T06:49:03.354 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc88f82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:03.354 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc88f82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37368/0 (socket says 172.21.15.73:37368) 2024-06-06T06:49:03.354 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc88f82700 1 -- 172.21.15.73:0/1304990390 learned_addr learned my addr 172.21.15.73:0/1304990390 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc88f82700 1 -- 172.21.15.73:0/1304990390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc84108f40 con 0x7efc84107d90 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc88f82700 1 --2- 172.21.15.73:0/1304990390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efc780098a0 tx=0x7efc78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e900659c90c329 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.734+0000 7efc837fe700 1 -- 172.21.15.73:0/1304990390 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc78004160 con 0x7efc84107d90 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc837fe700 1 -- 172.21.15.73:0/1304990390 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efc78004300 con 0x7efc84107d90 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc837fe700 1 -- 172.21.15.73:0/1304990390 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc78011a80 con 0x7efc84107d90 2024-06-06T06:49:03.355 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1304990390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 msgr2=0x7efc841081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 --2- 172.21.15.73:0/1304990390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7efc780098a0 tx=0x7efc78009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1304990390 shutdown_connections 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 --2- 172.21.15.73:0/1304990390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc841081b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1304990390 >> 172.21.15.73:0/1304990390 conn(0x7efc84103370 msgr2=0x7efc84105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1304990390 shutdown_connections 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.735+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1304990390 wait complete. 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc8b1e6700 1 Processor -- start 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc8b1e6700 1 -- start start 2024-06-06T06:49:03.356 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc8b1e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc8b1e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc8410c2f0 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc88f82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc88f82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37372/0 (socket says 172.21.15.73:37372) 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc88f82700 1 -- 172.21.15.73:0/1258772077 learned_addr learned my addr 172.21.15.73:0/1258772077 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc88f82700 1 -- 172.21.15.73:0/1258772077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc8419c1e0 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc88f82700 1 --2- 172.21.15.73:0/1258772077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efc78006af0 tx=0x7efc78011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.736+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc780039e0 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efc78003b80 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc780196a0 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc8419c470 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc8419f160 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efc78019ce0 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.737+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efc78024b60 con 0x7efc84107d90 2024-06-06T06:49:03.357 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.738+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc84063af0 con 0x7efc84107d90 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.740+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efc78043dd0 con 0x7efc84107d90 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.806+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7efc8419f330 con 0x7efc84107d90 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.810+0000 7efc81ffb700 1 -- 172.21.15.73:0/1258772077 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7efc78032020 con 0x7efc84107d90 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 msgr2=0x7efc8419bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 --2- 172.21.15.73:0/1258772077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7efc78006af0 tx=0x7efc78011c40 comp rx=0 tx=0).stop 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 shutdown_connections 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 --2- 172.21.15.73:0/1258772077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc84107d90 0x7efc8419bca0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 >> 172.21.15.73:0/1258772077 conn(0x7efc84103370 msgr2=0x7efc84103ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 shutdown_connections 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:02.811+0000 7efc8b1e6700 1 -- 172.21.15.73:0/1258772077 wait complete. 2024-06-06T06:49:03.358 INFO:teuthology.orchestra.run.smithi073.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-06T06:49:03.369 INFO:teuthology.orchestra.run.smithi073.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-06T06:49:03.369 INFO:teuthology.orchestra.run.smithi073.stdout:Creating mgr... 2024-06-06T06:49:03.369 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-06T06:49:03.370 INFO:teuthology.orchestra.run.smithi073.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-06T06:49:03.608 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T06:49:03.608 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-06T06:49:03.758 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968.target.wants/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@.service. 2024-06-06T06:49:04.127 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:03 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1258772077' entity='client.admin' 2024-06-06T06:49:04.320 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T06:49:04.320 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout disabled 2024-06-06T06:49:04.327 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T06:49:04.328 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:49:04.328 INFO:teuthology.orchestra.run.smithi073.stdout:firewalld.service is not enabled 2024-06-06T06:49:04.328 INFO:teuthology.orchestra.run.smithi073.stdout:Not possible to enable service . firewalld.service is not available 2024-06-06T06:49:04.336 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T06:49:04.336 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout disabled 2024-06-06T06:49:04.344 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T06:49:04.344 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:49:04.344 INFO:teuthology.orchestra.run.smithi073.stdout:firewalld.service is not enabled 2024-06-06T06:49:04.345 INFO:teuthology.orchestra.run.smithi073.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-06T06:49:04.345 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mgr to start... 2024-06-06T06:49:04.345 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mgr... 2024-06-06T06:49:05.104 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:04 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/250068657' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T06:49:05.104 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:05.102+0000 7f25bc12b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T06:49:05.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:05.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:05.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T06:49:05.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 0 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-06T06:49:05.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T06:49:05.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T06:49:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T06:49:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T06:49:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T06:49:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T06:49:05.488 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T06:49:05.489 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T06:48:47.254721+0000", 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.793+0000 7f647f19c700 1 Processor -- start 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.793+0000 7f647f19c700 1 -- start start 2024-06-06T06:49:05.490 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.793+0000 7f647f19c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.793+0000 7f647f19c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6478107660 con 0x7f6478106c70 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f647cf38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f647cf38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37394/0 (socket says 172.21.15.73:37394) 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f647cf38700 1 -- 172.21.15.73:0/1956238735 learned_addr learned my addr 172.21.15.73:0/1956238735 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f647cf38700 1 -- 172.21.15.73:0/1956238735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6478107e20 con 0x7f6478106c70 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f647cf38700 1 --2- 172.21.15.73:0/1956238735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5b4a1b134f7fd82 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f646f7fe700 1 -- 172.21.15.73:0/1956238735 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6474004160 con 0x7f6478106c70 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f646f7fe700 1 -- 172.21.15.73:0/1956238735 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6474004300 con 0x7f6478106c70 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.794+0000 7f646f7fe700 1 -- 172.21.15.73:0/1956238735 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6474011a80 con 0x7f6478106c70 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 -- 172.21.15.73:0/1956238735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 msgr2=0x7f6478107090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 --2- 172.21.15.73:0/1956238735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:05.491 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 -- 172.21.15.73:0/1956238735 shutdown_connections 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 --2- 172.21.15.73:0/1956238735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f6478107090 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 -- 172.21.15.73:0/1956238735 >> 172.21.15.73:0/1956238735 conn(0x7f64781021f0 msgr2=0x7f6478104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 -- 172.21.15.73:0/1956238735 shutdown_connections 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.795+0000 7f647f19c700 1 -- 172.21.15.73:0/1956238735 wait complete. 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647f19c700 1 Processor -- start 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647f19c700 1 -- start start 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647f19c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647f19c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f647810b1d0 con 0x7f6478106c70 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647cf38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647cf38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37408/0 (socket says 172.21.15.73:37408) 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647cf38700 1 -- 172.21.15.73:0/250068657 learned_addr learned my addr 172.21.15.73:0/250068657 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.796+0000 7f647cf38700 1 -- 172.21.15.73:0/250068657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647819ea20 con 0x7f6478106c70 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f647cf38700 1 --2- 172.21.15.73:0/250068657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f6474011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:05.492 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64740039f0 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6474003b90 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f647401a740 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647819ebf0 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647819ef90 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f647402d3f0 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.797+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6474019700 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.798+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6478063af0 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.800+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6474024af0 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.872+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f647819f290 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.872+0000 7f646dffb700 1 -- 172.21.15.73:0/250068657 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6474024440 con 0x7f6478106c70 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 msgr2=0x7f647807c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:05.493 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 --2- 172.21.15.73:0/250068657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6474000c00 tx=0x7f6474011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 shutdown_connections 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 --2- 172.21.15.73:0/250068657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6478106c70 0x7f647807c2c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 >> 172.21.15.73:0/250068657 conn(0x7f64781021f0 msgr2=0x7f6478102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 shutdown_connections 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:04.874+0000 7f647f19c700 1 -- 172.21.15.73:0/250068657 wait complete. 2024-06-06T06:49:05.494 INFO:teuthology.orchestra.run.smithi073.stdout:mgr not available, waiting (1/15)... 2024-06-06T06:49:06.665 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:06.320+0000 7f25bc12b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T06:49:06.665 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:06.492+0000 7f25bc12b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:06.665 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:06.664+0000 7f25bc12b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T06:49:07.263 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:06.961+0000 7f25bc12b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T06:49:07.266 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:07.059+0000 7f25bc12b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T06:49:07.266 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:07.262+0000 7f25bc12b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T06:49:07.524 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:07.453+0000 7f25bc12b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:49:08.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:08 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1098603546' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T06:49:08.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:08.071+0000 7f25bc12b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T06:49:08.630 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:08.630 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 0 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T06:49:08.631 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T06:49:08.632 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T06:49:08.633 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:08.634 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T06:48:47.254721+0000", 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.025+0000 7fe22d621700 1 Processor -- start 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.025+0000 7fe22d621700 1 -- start start 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.025+0000 7fe22d621700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.025+0000 7fe22d621700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe22807be70 con 0x7fe228108790 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe226ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:08.635 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe226ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46720/0 (socket says 172.21.15.73:46720) 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe226ffd700 1 -- 172.21.15.73:0/1062728437 learned_addr learned my addr 172.21.15.73:0/1062728437 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe226ffd700 1 -- 172.21.15.73:0/1062728437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22810b0c0 con 0x7fe228108790 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe226ffd700 1 --2- 172.21.15.73:0/1062728437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2100098a0 tx=0x7fe210009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72128e80788acd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe225ffb700 1 -- 172.21.15.73:0/1062728437 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe210004160 con 0x7fe228108790 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe225ffb700 1 -- 172.21.15.73:0/1062728437 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe210004300 con 0x7fe228108790 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.026+0000 7fe225ffb700 1 -- 172.21.15.73:0/1062728437 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe210011a80 con 0x7fe228108790 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 -- 172.21.15.73:0/1062728437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 msgr2=0x7fe22810ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 --2- 172.21.15.73:0/1062728437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fe2100098a0 tx=0x7fe210009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 -- 172.21.15.73:0/1062728437 shutdown_connections 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 --2- 172.21.15.73:0/1062728437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22810ab80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 -- 172.21.15.73:0/1062728437 >> 172.21.15.73:0/1062728437 conn(0x7fe2281021e0 msgr2=0x7fe228104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 -- 172.21.15.73:0/1062728437 shutdown_connections 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.027+0000 7fe22d621700 1 -- 172.21.15.73:0/1062728437 wait complete. 2024-06-06T06:49:08.636 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe22d621700 1 Processor -- start 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe22d621700 1 -- start start 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe22d621700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe22d621700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe22810c010 con 0x7fe228108790 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe226ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe226ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46736/0 (socket says 172.21.15.73:46736) 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.028+0000 7fe226ffd700 1 -- 172.21.15.73:0/1098603546 learned_addr learned my addr 172.21.15.73:0/1098603546 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe226ffd700 1 -- 172.21.15.73:0/1098603546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe22819c1a0 con 0x7fe228108790 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe226ffd700 1 --2- 172.21.15.73:0/1098603546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe210006af0 tx=0x7fe210011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe210003930 con 0x7fe228108790 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe210003ad0 con 0x7fe228108790 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe22819c490 con 0x7fe228108790 2024-06-06T06:49:08.637 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe210019590 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe22819f120 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe210019c20 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.029+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe210024730 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.030+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe228063af0 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.034+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fe210024930 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.106+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fe22819f770 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.107+0000 7fe21ffff700 1 -- 172.21.15.73:0/1098603546 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fe21002d440 con 0x7fe228108790 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.108+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 msgr2=0x7fe22819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.108+0000 7fe22d621700 1 --2- 172.21.15.73:0/1098603546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe210006af0 tx=0x7fe210011b70 comp rx=0 tx=0).stop 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.108+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 shutdown_connections 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.108+0000 7fe22d621700 1 --2- 172.21.15.73:0/1098603546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe228108790 0x7fe22819bc60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.108+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 >> 172.21.15.73:0/1098603546 conn(0x7fe2281021e0 msgr2=0x7fe228102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.109+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 shutdown_connections 2024-06-06T06:49:08.638 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:08.109+0000 7fe22d621700 1 -- 172.21.15.73:0/1098603546 wait complete. 2024-06-06T06:49:08.639 INFO:teuthology.orchestra.run.smithi073.stdout:mgr not available, waiting (2/15)... 2024-06-06T06:49:08.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:08.502+0000 7f25bc12b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T06:49:08.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:08.600+0000 7f25bc12b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T06:49:09.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:08.804+0000 7f25bc12b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T06:49:09.656 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.319+0000 7f25bc12b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T06:49:09.656 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.412+0000 7f25bc12b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T06:49:10.000 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.655+0000 7f25bc12b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T06:49:10.000 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.789+0000 7f25bc12b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T06:49:10.000 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.883+0000 7f25bc12b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T06:49:10.000 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:09.999+0000 7f25bc12b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:10.751 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:10.322+0000 7f25bc12b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T06:49:11.024 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:10.932+0000 7f25bc12b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:11.161+0000 7f25bc12b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1503612523' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: Activating manager daemon x 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: mgrmap e2: x(active, starting, since 0.00297023s) 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T06:49:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:49:11.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T06:49:11.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:11 smithi073 ceph-mon[32303]: Manager daemon x is now available 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: from='mgr.14100 172.21.15.73:0/1297241484' entity='mgr.x' 2024-06-06T06:49:12.517 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:12 smithi073 ceph-mon[32303]: mgrmap e3: x(active, since 1.0074s) 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T06:49:12.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 0 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T06:49:12.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:12.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T06:49:12.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T06:49:12.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T06:49:12.531 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.532 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T06:48:47.254721+0000", 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:12.533 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ffbd39700 1 Processor -- start 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ffbd39700 1 -- start start 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ffbd39700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ffbd39700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4109cc0 con 0x7f0ff41092d0 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ff9ad5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ff9ad5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46748/0 (socket says 172.21.15.73:46748) 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.044+0000 7f0ff9ad5700 1 -- 172.21.15.73:0/1052175221 learned_addr learned my addr 172.21.15.73:0/1052175221 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.045+0000 7f0ff9ad5700 1 -- 172.21.15.73:0/1052175221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff4109e40 con 0x7f0ff41092d0 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.045+0000 7f0ff9ad5700 1 --2- 172.21.15.73:0/1052175221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00098a0 tx=0x7f0ff0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5ea5b7562a8700 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.045+0000 7f0ff8ad3700 1 -- 172.21.15.73:0/1052175221 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ff0004160 con 0x7f0ff41092d0 2024-06-06T06:49:12.534 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.045+0000 7f0ff8ad3700 1 -- 172.21.15.73:0/1052175221 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ff0004300 con 0x7f0ff41092d0 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.045+0000 7f0ff8ad3700 1 -- 172.21.15.73:0/1052175221 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ff0011a80 con 0x7f0ff41092d0 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1052175221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 msgr2=0x7f0ff41096f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 --2- 172.21.15.73:0/1052175221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00098a0 tx=0x7f0ff0009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1052175221 shutdown_connections 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 --2- 172.21.15.73:0/1052175221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff41096f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1052175221 >> 172.21.15.73:0/1052175221 conn(0x7f0ff4101910 msgr2=0x7f0ff4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1052175221 shutdown_connections 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1052175221 wait complete. 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 Processor -- start 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.046+0000 7f0ffbd39700 1 -- start start 2024-06-06T06:49:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ffbd39700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ffbd39700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff4106ad0 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ff9ad5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ff9ad5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46754/0 (socket says 172.21.15.73:46754) 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ff9ad5700 1 -- 172.21.15.73:0/1503612523 learned_addr learned my addr 172.21.15.73:0/1503612523 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ff9ad5700 1 -- 172.21.15.73:0/1503612523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff419c190 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ff9ad5700 1 --2- 172.21.15.73:0/1503612523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0006af0 tx=0x7f0ff0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ff0003930 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff419c480 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff419f110 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.047+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0ff0003ad0 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.048+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ff0004300 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.048+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0ff0019970 con 0x7f0ff41092d0 2024-06-06T06:49:12.536 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.048+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0ff0024730 con 0x7f0ff41092d0 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.048+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd8005360 con 0x7f0ff41092d0 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.052+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f0ff0043dd0 con 0x7f0ff41092d0 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.125+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f0fd8005650 con 0x7f0ff41092d0 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.125+0000 7f0feaffd700 1 -- 172.21.15.73:0/1503612523 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f0ff0032020 con 0x7f0ff41092d0 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 msgr2=0x7f0ff419bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 --2- 172.21.15.73:0/1503612523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0006af0 tx=0x7f0ff0011b70 comp rx=0 tx=0).stop 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 shutdown_connections 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 --2- 172.21.15.73:0/1503612523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff41092d0 0x7f0ff419bc50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 >> 172.21.15.73:0/1503612523 conn(0x7f0ff4101910 msgr2=0x7f0ff4102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 shutdown_connections 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:11.126+0000 7f0ffbd39700 1 -- 172.21.15.73:0/1503612523 wait complete. 2024-06-06T06:49:12.537 INFO:teuthology.orchestra.run.smithi073.stdout:mgr not available, waiting (3/15)... 2024-06-06T06:49:14.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:14 smithi073 ceph-mon[32303]: mgrmap e4: x(active, since 2s) 2024-06-06T06:49:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:15 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3117902121' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-06T06:49:15.792 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "health": { 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-06T06:49:15.795 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 0 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "a" 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-06T06:49:15.796 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-06T06:49:15.797 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-06T06:49:15.804 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-06T06:49:15.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "restful" 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ], 2024-06-06T06:49:15.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "modified": "2024-06-06T06:48:47.254721+0000", 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout }, 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.935+0000 7fb683145700 1 Processor -- start 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.935+0000 7fb683145700 1 -- start start 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb683145700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb683145700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c07be30 con 0x7fb67c1087d0 2024-06-06T06:49:15.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb680ee1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb680ee1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46838/0 (socket says 172.21.15.73:46838) 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb680ee1700 1 -- 172.21.15.73:0/2619650171 learned_addr learned my addr 172.21.15.73:0/2619650171 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb680ee1700 1 -- 172.21.15.73:0/2619650171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c10b100 con 0x7fb67c1087d0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.936+0000 7fb680ee1700 1 --2- 172.21.15.73:0/2619650171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6780098a0 tx=0x7fb678009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a6f90c7fb7b5b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb6737fe700 1 -- 172.21.15.73:0/2619650171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb678004160 con 0x7fb67c1087d0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb6737fe700 1 -- 172.21.15.73:0/2619650171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb678004300 con 0x7fb67c1087d0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb6737fe700 1 -- 172.21.15.73:0/2619650171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb678011a80 con 0x7fb67c1087d0 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 -- 172.21.15.73:0/2619650171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 msgr2=0x7fb67c10abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 --2- 172.21.15.73:0/2619650171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb6780098a0 tx=0x7fb678009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 -- 172.21.15.73:0/2619650171 shutdown_connections 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 --2- 172.21.15.73:0/2619650171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c10abc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 -- 172.21.15.73:0/2619650171 >> 172.21.15.73:0/2619650171 conn(0x7fb67c1021e0 msgr2=0x7fb67c104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 -- 172.21.15.73:0/2619650171 shutdown_connections 2024-06-06T06:49:15.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.937+0000 7fb683145700 1 -- 172.21.15.73:0/2619650171 wait complete. 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb683145700 1 Processor -- start 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb683145700 1 -- start start 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb683145700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb683145700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb67c107360 con 0x7fb67c1087d0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb680ee1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb680ee1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46850/0 (socket says 172.21.15.73:46850) 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb680ee1700 1 -- 172.21.15.73:0/3117902121 learned_addr learned my addr 172.21.15.73:0/3117902121 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.938+0000 7fb680ee1700 1 -- 172.21.15.73:0/3117902121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67c19e410 con 0x7fb67c1087d0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb680ee1700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb678006af0 tx=0x7fb678011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb678003930 con 0x7fb67c1087d0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb678003ad0 con 0x7fb67c1087d0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb67c19e6a0 con 0x7fb67c1087d0 2024-06-06T06:49:15.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6780195e0 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.939+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb67c1a1390 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.940+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fb678019890 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.940+0000 7fb671ffb700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 0x7fb66403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.940+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb67804d4f0 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.940+0000 7fb673fff700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 0x7fb66403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.940+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb67c063af0 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.941+0000 7fb673fff700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 0x7fb66403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006fd0 tx=0x7fb66c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:14.950+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb67804cf10 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.239+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fb67c02ded0 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.240+0000 7fb671ffb700 1 -- 172.21.15.73:0/3117902121 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fb67800fda0 con 0x7fb67c1087d0 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 msgr2=0x7fb66403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 0x7fb66403b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb66c006fd0 tx=0x7fb66c006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:15.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 msgr2=0x7fb67c19ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb678006af0 tx=0x7fb678011b70 comp rx=0 tx=0).stop 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 shutdown_connections 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb664039000 0x7fb66403b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 --2- 172.21.15.73:0/3117902121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb67c1087d0 0x7fb67c19ded0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 >> 172.21.15.73:0/3117902121 conn(0x7fb67c1021e0 msgr2=0x7fb67c102aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 shutdown_connections 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:15.243+0000 7fb683145700 1 -- 172.21.15.73:0/3117902121 wait complete. 2024-06-06T06:49:15.811 INFO:teuthology.orchestra.run.smithi073.stdout:mgr is available 2024-06-06T06:49:16.771 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:16 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3028852777' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [global] 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout fsid = b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-06T06:49:17.022 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout [osd] 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efccb95f700 1 Processor -- start 2024-06-06T06:49:17.023 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efccb95f700 1 -- start start 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efccb95f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efccb95f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcc407d140 con 0x7efcc407e6c0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efcc96fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efcc96fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45740/0 (socket says 172.21.15.73:45740) 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.217+0000 7efcc96fb700 1 -- 172.21.15.73:0/2556792369 learned_addr learned my addr 172.21.15.73:0/2556792369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efcc96fb700 1 -- 172.21.15.73:0/2556792369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc407d2c0 con 0x7efcc407e6c0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efcc96fb700 1 --2- 172.21.15.73:0/2556792369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2b54f747040339 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efcbbfff700 1 -- 172.21.15.73:0/2556792369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0004160 con 0x7efcc407e6c0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efcbbfff700 1 -- 172.21.15.73:0/2556792369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efcc0004300 con 0x7efcc407e6c0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efcbbfff700 1 -- 172.21.15.73:0/2556792369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0011a80 con 0x7efcc407e6c0 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efccb95f700 1 -- 172.21.15.73:0/2556792369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 msgr2=0x7efcc407cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:17.024 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.218+0000 7efccb95f700 1 --2- 172.21.15.73:0/2556792369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc0009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- 172.21.15.73:0/2556792369 shutdown_connections 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 --2- 172.21.15.73:0/2556792369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc407cb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- 172.21.15.73:0/2556792369 >> 172.21.15.73:0/2556792369 conn(0x7efcc4103370 msgr2=0x7efcc4105790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- 172.21.15.73:0/2556792369 shutdown_connections 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- 172.21.15.73:0/2556792369 wait complete. 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 Processor -- start 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- start start 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efccb95f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcc410e2e0 con 0x7efcc407e6c0 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.219+0000 7efcc96fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcc96fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45756/0 (socket says 172.21.15.73:45756) 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcc96fb700 1 -- 172.21.15.73:0/3028852777 learned_addr learned my addr 172.21.15.73:0/3028852777 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:17.025 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcc96fb700 1 -- 172.21.15.73:0/3028852777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcc419e380 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcc96fb700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcc0011a50 tx=0x7efcc0011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc0003a80 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7efcc0003c20 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcc419e610 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efcc001a650 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.220+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcc41a1300 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7efcc001a950 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efcba7fc700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 0x7efcb003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcc4063af0 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efcc004d750 con 0x7efcc407e6c0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efcc8efa700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 0x7efcb003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.221+0000 7efcc8efa700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 0x7efcb003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcb4006fd0 tx=0x7efcb4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:17.026 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.226+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efcc0032020 con 0x7efcc407e6c0 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.417+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7efcc41a2a50 con 0x7efcc407e6c0 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.418+0000 7efcba7fc700 1 -- 172.21.15.73:0/3028852777 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7efcc0019c70 con 0x7efcc407e6c0 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 msgr2=0x7efcb003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 0x7efcb003b4c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7efcb4006fd0 tx=0x7efcb4006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 msgr2=0x7efcc419de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7efcc0011a50 tx=0x7efcc0011b30 comp rx=0 tx=0).stop 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 shutdown_connections 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7efcb0039000 0x7efcb003b4c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 --2- 172.21.15.73:0/3028852777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcc407e6c0 0x7efcc419de40 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 >> 172.21.15.73:0/3028852777 conn(0x7efcc4103370 msgr2=0x7efcc4103ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.420+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 shutdown_connections 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:16.421+0000 7efccb95f700 1 -- 172.21.15.73:0/3028852777 wait complete. 2024-06-06T06:49:17.027 INFO:teuthology.orchestra.run.smithi073.stdout:Enabling cephadm module... 2024-06-06T06:49:18.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:17 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/467436717' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-06T06:49:18.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setuser ceph since I am not root 2024-06-06T06:49:18.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setgroup ceph since I am not root 2024-06-06T06:49:18.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:17.929+0000 7fe5f4705200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T06:49:18.338 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:18.115+0000 7fe5f4705200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:49:18.338 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:18.218+0000 7fe5f4705200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd319f50700 1 Processor -- start 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd319f50700 1 -- start start 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd319f50700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd319f50700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410adf0 con 0x7fd31407c560 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.441+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45758/0 (socket says 172.21.15.73:45758) 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3137fe700 1 -- 172.21.15.73:0/3633126702 learned_addr learned my addr 172.21.15.73:0/3633126702 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3137fe700 1 -- 172.21.15.73:0/3633126702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd31410af70 con 0x7fd31407c560 2024-06-06T06:49:18.346 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3137fe700 1 --2- 172.21.15.73:0/3633126702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3040098a0 tx=0x7fd304009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18048f044680ff17 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3127fc700 1 -- 172.21.15.73:0/3633126702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd304004160 con 0x7fd31407c560 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3127fc700 1 -- 172.21.15.73:0/3633126702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd304004300 con 0x7fd31407c560 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.442+0000 7fd3127fc700 1 -- 172.21.15.73:0/3633126702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd304011a80 con 0x7fd31407c560 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 -- 172.21.15.73:0/3633126702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 msgr2=0x7fd31410a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 --2- 172.21.15.73:0/3633126702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd3040098a0 tx=0x7fd304009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 -- 172.21.15.73:0/3633126702 shutdown_connections 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 --2- 172.21.15.73:0/3633126702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd31407c560 0x7fd31410a820 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 -- 172.21.15.73:0/3633126702 >> 172.21.15.73:0/3633126702 conn(0x7fd314102220 msgr2=0x7fd314104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 -- 172.21.15.73:0/3633126702 shutdown_connections 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 -- 172.21.15.73:0/3633126702 wait complete. 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.443+0000 7fd319f50700 1 Processor -- start 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd319f50700 1 -- start start 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd319f50700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:18.347 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd319f50700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd31410bff0 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd3137fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45764/0 (socket says 172.21.15.73:45764) 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd3137fe700 1 -- 172.21.15.73:0/467436717 learned_addr learned my addr 172.21.15.73:0/467436717 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd3137fe700 1 -- 172.21.15.73:0/467436717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd314108470 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd3137fe700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd304006af0 tx=0x7fd304011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3040039e0 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd304003b80 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd314106d00 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3040196a0 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.444+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3141071e0 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.445+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314108640 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.445+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7fd30401ab50 con 0x7fd314108050 2024-06-06T06:49:18.348 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.446+0000 7fd310ff9700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 0x7fd2fc03b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.446+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd30404d590 con 0x7fd314108050 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.446+0000 7fd312ffd700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 0x7fd2fc03b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.446+0000 7fd312ffd700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 0x7fd2fc03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd308006fd0 tx=0x7fd308006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.451+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd304023e00 con 0x7fd314108050 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.703+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7fd31407c600 con 0x7fd314108050 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.755+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7fd30404cfb0 con 0x7fd314108050 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.756+0000 7fd310ff9700 1 -- 172.21.15.73:0/467436717 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd30401a430 con 0x7fd314108050 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 msgr2=0x7fd2fc03b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 0x7fd2fc03b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd308006fd0 tx=0x7fd308006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 msgr2=0x7fd3141067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd304006af0 tx=0x7fd304011c40 comp rx=0 tx=0).stop 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 shutdown_connections 2024-06-06T06:49:18.349 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fd2fc038fb0 0x7fd2fc03b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 --2- 172.21.15.73:0/467436717 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd314108050 0x7fd3141067c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.757+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 >> 172.21.15.73:0/467436717 conn(0x7fd314102220 msgr2=0x7fd314102d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.758+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 shutdown_connections 2024-06-06T06:49:18.350 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:17.758+0000 7fd319f50700 1 -- 172.21.15.73:0/467436717 wait complete. 2024-06-06T06:49:18.591 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:18.539+0000 7fe5f4705200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T06:49:18.992 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/467436717' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-06T06:49:18.992 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-mon[32303]: mgrmap e5: x(active, since 6s) 2024-06-06T06:49:18.992 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:18.628+0000 7fe5f4705200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T06:49:19.666 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:19.667 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-06T06:49:19.667 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T06:49:19.667 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T06:49:19.667 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T06:49:19.667 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:19.668 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb48603f700 1 Processor -- start 2024-06-06T06:49:19.668 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb48603f700 1 -- start start 2024-06-06T06:49:19.668 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb48603f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:19.668 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb48603f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4801075f0 con 0x7fb480106c00 2024-06-06T06:49:19.668 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.785+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45790/0 (socket says 172.21.15.73:45790) 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47f7fe700 1 -- 172.21.15.73:0/1298085451 learned_addr learned my addr 172.21.15.73:0/1298085451 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47f7fe700 1 -- 172.21.15.73:0/1298085451 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb480107e00 con 0x7fb480106c00 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47f7fe700 1 --2- 172.21.15.73:0/1298085451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4680098a0 tx=0x7fb468009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42f2cd7699545306 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47e7fc700 1 -- 172.21.15.73:0/1298085451 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb468004160 con 0x7fb480106c00 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47e7fc700 1 -- 172.21.15.73:0/1298085451 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb468004300 con 0x7fb480106c00 2024-06-06T06:49:19.669 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.786+0000 7fb47e7fc700 1 -- 172.21.15.73:0/1298085451 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb468011a80 con 0x7fb480106c00 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 -- 172.21.15.73:0/1298085451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 msgr2=0x7fb480107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 --2- 172.21.15.73:0/1298085451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb4680098a0 tx=0x7fb468009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 -- 172.21.15.73:0/1298085451 shutdown_connections 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 --2- 172.21.15.73:0/1298085451 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb480107020 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 -- 172.21.15.73:0/1298085451 >> 172.21.15.73:0/1298085451 conn(0x7fb4801021e0 msgr2=0x7fb480104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 -- 172.21.15.73:0/1298085451 shutdown_connections 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.787+0000 7fb48603f700 1 -- 172.21.15.73:0/1298085451 wait complete. 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb48603f700 1 Processor -- start 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb48603f700 1 -- start start 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb48603f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb48603f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb48010b1b0 con 0x7fb480106c00 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:19.670 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb47f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45792/0 (socket says 172.21.15.73:45792) 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.788+0000 7fb47f7fe700 1 -- 172.21.15.73:0/370922078 learned_addr learned my addr 172.21.15.73:0/370922078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb47f7fe700 1 -- 172.21.15.73:0/370922078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48019ea60 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb47f7fe700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb4680039c0 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48019ec30 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb468003b60 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb46801a6f0 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.789+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48019efd0 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fb46801a9a0 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb47cff9700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 0x7fb46c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb46804d390 con 0x7fb480106c00 2024-06-06T06:49:19.671 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb47effd700 1 -- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 msgr2=0x7fb46c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb47effd700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 0x7fb46c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.790+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb480063af0 con 0x7fb480106c00 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.799+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb468020240 con 0x7fb480106c00 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.991+0000 7fb47effd700 1 -- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 msgr2=0x7fb46c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:18.991+0000 7fb47effd700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 0x7fb46c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.098+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb48019f740 con 0x7fb480106c00 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.098+0000 7fb47cff9700 1 -- 172.21.15.73:0/370922078 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb46804ce50 con 0x7fb480106c00 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 msgr2=0x7fb46c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 0x7fb46c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 msgr2=0x7fb48007ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb468000c00 tx=0x7fb468004290 comp rx=0 tx=0).stop 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 shutdown_connections 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7fb46c039050 0x7fb46c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:19.672 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 --2- 172.21.15.73:0/370922078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb480106c00 0x7fb48007ba30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 >> 172.21.15.73:0/370922078 conn(0x7fb4801021e0 msgr2=0x7fb480102e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.100+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 shutdown_connections 2024-06-06T06:49:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:19.101+0000 7fb48603f700 1 -- 172.21.15.73:0/370922078 wait complete. 2024-06-06T06:49:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for the mgr to restart... 2024-06-06T06:49:19.673 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mgr epoch 5... 2024-06-06T06:49:19.998 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:19 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/370922078' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T06:49:19.999 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:19.864+0000 7fe5f4705200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T06:49:20.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.039+0000 7fe5f4705200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:20.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.214+0000 7fe5f4705200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T06:49:20.596 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.503+0000 7fe5f4705200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T06:49:20.596 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.595+0000 7fe5f4705200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T06:49:20.993 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.798+0000 7fe5f4705200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T06:49:21.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:20.992+0000 7fe5f4705200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:49:21.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:21.591+0000 7fe5f4705200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T06:49:22.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:22 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:22.014+0000 7fe5f4705200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T06:49:22.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:22 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:22.112+0000 7fe5f4705200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T06:49:22.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:22 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:22.314+0000 7fe5f4705200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T06:49:23.112 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:22 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:22.841+0000 7fe5f4705200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T06:49:23.113 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:22 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:22.932+0000 7fe5f4705200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T06:49:23.403 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:23 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:23.175+0000 7fe5f4705200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T06:49:23.403 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:23 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:23.307+0000 7fe5f4705200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T06:49:23.403 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:23 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:23.402+0000 7fe5f4705200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T06:49:23.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:23 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:23.517+0000 7fe5f4705200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:24.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:23 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:23.816+0000 7fe5f4705200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T06:49:24.689 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:24.417+0000 7fe5f4705200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T06:49:24.689 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:24.635+0000 7fe5f4705200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T06:49:25.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: Active manager daemon x restarted 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: Activating manager daemon x 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: osdmap e2: 0 total, 0 up, 0 in 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: mgrmap e6: x(active, starting, since 0.00357151s) 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: Manager daemon x is now available 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:25.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T06:49:25.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:24 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T06:49:25.951 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:25 smithi073 ceph-mon[32303]: Found migration_current of "None". Setting to last migration. 2024-06-06T06:49:25.952 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:25 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:25.952 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:25 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:25.952 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:25 smithi073 ceph-mon[32303]: mgrmap e7: x(active, since 1.00542s) 2024-06-06T06:49:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-06T06:49:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T06:49:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:26.261 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f8057461700 1 Processor -- start 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f8057461700 1 -- start start 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f8057461700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f8057461700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f805007c690 con 0x7f80501087d0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f80551fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f80551fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45796/0 (socket says 172.21.15.73:45796) 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.102+0000 7f80551fd700 1 -- 172.21.15.73:0/496392843 learned_addr learned my addr 172.21.15.73:0/496392843 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.103+0000 7f80551fd700 1 -- 172.21.15.73:0/496392843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f805010b100 con 0x7f80501087d0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.103+0000 7f80551fd700 1 --2- 172.21.15.73:0/496392843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098a0 tx=0x7f804c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969d700119d54953 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.103+0000 7f8047fff700 1 -- 172.21.15.73:0/496392843 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c004160 con 0x7f80501087d0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.103+0000 7f8047fff700 1 -- 172.21.15.73:0/496392843 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f804c004300 con 0x7f80501087d0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.103+0000 7f8047fff700 1 -- 172.21.15.73:0/496392843 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c011a80 con 0x7f80501087d0 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 -- 172.21.15.73:0/496392843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 msgr2=0x7f805010abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:26.262 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 --2- 172.21.15.73:0/496392843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f804c0098a0 tx=0x7f804c009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 -- 172.21.15.73:0/496392843 shutdown_connections 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 --2- 172.21.15.73:0/496392843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f805010abc0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 -- 172.21.15.73:0/496392843 >> 172.21.15.73:0/496392843 conn(0x7f8050102280 msgr2=0x7f80501046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 -- 172.21.15.73:0/496392843 shutdown_connections 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.104+0000 7f8057461700 1 -- 172.21.15.73:0/496392843 wait complete. 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f8057461700 1 Processor -- start 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f8057461700 1 -- start start 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f8057461700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f8057461700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f805010c060 con 0x7f80501087d0 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f80551fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f80551fd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45802/0 (socket says 172.21.15.73:45802) 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f80551fd700 1 -- 172.21.15.73:0/1104387122 learned_addr learned my addr 172.21.15.73:0/1104387122 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f80551fd700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8050108590 con 0x7f80501087d0 2024-06-06T06:49:26.263 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.105+0000 7f80551fd700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f804c011a50 tx=0x7f804c011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.106+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c003a80 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.106+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f804c003c20 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.106+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f804c01a650 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.106+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8050106780 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.106+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8050106cc0 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f804c01a900 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f80467fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8050063af0 con 0x7f803c0390a0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f804c04e2a0 con 0x7f80501087d0 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f80549fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.107+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.308+0000 7f80549fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:26.264 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.308+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.709+0000 7f80549fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:20.709+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:21.510+0000 7f80549fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:21.510+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:23.111+0000 7f80549fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/961923295 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:23.111+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:24.639+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f804c024ad0 con 0x7f80501087d0 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:24.639+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 msgr2=0x7f803c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:24.639+0000 7f80467fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.640+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f804c028c00 con 0x7f80501087d0 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.640+0000 7f80467fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 0x7f803c03ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.640+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8050063af0 con 0x7f803c03c7e0 2024-06-06T06:49:26.265 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.640+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 0x7f803c03ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.641+0000 7f80549fc700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 0x7f803c03ebd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8040003a10 tx=0x7f80400092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.642+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f8050063af0 con 0x7f803c03c7e0 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.648+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f8050103470 con 0x7f803c03c7e0 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.648+0000 7f80467fc700 1 -- 172.21.15.73:0/1104387122 <== mgr.14120 v2:172.21.15.73:6800/3054677757 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f8050103470 con 0x7f803c03c7e0 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 msgr2=0x7f803c03ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 0x7f803c03ebd0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f8040003a10 tx=0x7f80400092b0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 msgr2=0x7f8050108050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f804c011a50 tx=0x7f804c011b30 comp rx=0 tx=0).stop 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 shutdown_connections 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f803c03c7e0 0x7f803c03ebd0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80501087d0 0x7f8050108050 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 --2- 172.21.15.73:0/1104387122 >> [v2:172.21.15.73:6800/961923295,v1:172.21.15.73:6801/961923295] conn(0x7f803c0390a0 0x7f803c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 >> 172.21.15.73:0/1104387122 conn(0x7f8050102280 msgr2=0x7f8050102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 shutdown_connections 2024-06-06T06:49:26.267 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:25.649+0000 7f8057461700 1 -- 172.21.15.73:0/1104387122 wait complete. 2024-06-06T06:49:26.267 INFO:teuthology.orchestra.run.smithi073.stdout:mgr epoch 5 is available 2024-06-06T06:49:26.267 INFO:teuthology.orchestra.run.smithi073.stdout:Setting orchestrator backend to cephadm... 2024-06-06T06:49:26.895 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:26 smithi073 ceph-mon[32303]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T06:49:26.895 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:26 smithi073 ceph-mon[32303]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T06:49:26.896 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:26 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:27.549 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.671+0000 7fab5b51a700 1 Processor -- start 2024-06-06T06:49:27.549 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.671+0000 7fab5b51a700 1 -- start start 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.671+0000 7fab5b51a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.671+0000 7fab5b51a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab541069a0 con 0x7fab54105fb0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab592b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab592b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40504/0 (socket says 172.21.15.73:40504) 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab592b6700 1 -- 172.21.15.73:0/2860385920 learned_addr learned my addr 172.21.15.73:0/2860385920 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab592b6700 1 -- 172.21.15.73:0/2860385920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab54106b20 con 0x7fab54105fb0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab592b6700 1 --2- 172.21.15.73:0/2860385920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e80 tx=0x7fab5000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24ac2f8cb648b0d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab4bfff700 1 -- 172.21.15.73:0/2860385920 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab50017070 con 0x7fab54105fb0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab4bfff700 1 -- 172.21.15.73:0/2860385920 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fab5000b8c0 con 0x7fab54105fb0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.672+0000 7fab4bfff700 1 -- 172.21.15.73:0/2860385920 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab50004520 con 0x7fab54105fb0 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 -- 172.21.15.73:0/2860385920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 msgr2=0x7fab541063d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:27.550 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2860385920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e80 tx=0x7fab5000b330 comp rx=0 tx=0).stop 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 -- 172.21.15.73:0/2860385920 shutdown_connections 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2860385920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab541063d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 -- 172.21.15.73:0/2860385920 >> 172.21.15.73:0/2860385920 conn(0x7fab54101910 msgr2=0x7fab54103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 -- 172.21.15.73:0/2860385920 shutdown_connections 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.673+0000 7fab5b51a700 1 -- 172.21.15.73:0/2860385920 wait complete. 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab5b51a700 1 Processor -- start 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab5b51a700 1 -- start start 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab5b51a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab5b51a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab54109e10 con 0x7fab54105fb0 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab592b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab592b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40518/0 (socket says 172.21.15.73:40518) 2024-06-06T06:49:27.551 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab592b6700 1 -- 172.21.15.73:0/2946515066 learned_addr learned my addr 172.21.15.73:0/2946515066 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab592b6700 1 -- 172.21.15.73:0/2946515066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab5419c180 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab592b6700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e50 tx=0x7fab50004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.674+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab50020020 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.675+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab5419c410 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.675+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fab5000bba0 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.675+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fab50017040 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.675+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab5419f100 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7fab50023020 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab5410c260 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab4a7fc700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 0x7fab4003b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fab5000f050 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab58ab5700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 0x7fab4003b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.676+0000 7fab58ab5700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 0x7fab4003b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.681+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fab5002a020 con 0x7fab54105fb0 2024-06-06T06:49:27.552 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.694+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fab50025400 con 0x7fab54105fb0 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.894+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fab54102bd0 con 0x7fab40038e80 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.900+0000 7fab4a7fc700 1 -- 172.21.15.73:0/2946515066 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fab54102bd0 con 0x7fab40038e80 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 msgr2=0x7fab4003b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 0x7fab4003b340 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fab44006fd0 tx=0x7fab44006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 msgr2=0x7fab5419bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fab50009e50 tx=0x7fab50004670 comp rx=0 tx=0).stop 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 shutdown_connections 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fab40038e80 0x7fab4003b340 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 --2- 172.21.15.73:0/2946515066 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab54105fb0 0x7fab5419bc40 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 >> 172.21.15.73:0/2946515066 conn(0x7fab54101910 msgr2=0x7fab541024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 shutdown_connections 2024-06-06T06:49:27.553 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:26.903+0000 7fab5b51a700 1 -- 172.21.15.73:0/2946515066 wait complete. 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:25] ENGINE Bus STARTING 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:25] ENGINE Serving on http://172.21.15.73:8765 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:26] ENGINE Serving on https://172.21.15.73:7150 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:26] ENGINE Bus STARTED 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: mgrmap e8: x(active, since 2s) 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:27.862 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:27 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:28.740 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:28 smithi073 ceph-mon[32303]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62fe47700 1 Processor -- start 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62fe47700 1 -- start start 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62fe47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62fe47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6281087b0 con 0x7fa628107dc0 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62dbe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:28.747 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62dbe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40524/0 (socket says 172.21.15.73:40524) 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.961+0000 7fa62dbe3700 1 -- 172.21.15.73:0/4170160149 learned_addr learned my addr 172.21.15.73:0/4170160149 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.962+0000 7fa62dbe3700 1 -- 172.21.15.73:0/4170160149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa628108f20 con 0x7fa628107dc0 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.962+0000 7fa62dbe3700 1 --2- 172.21.15.73:0/4170160149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6240098a0 tx=0x7fa624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab8c9efe8c0de7f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.962+0000 7fa62cbe1700 1 -- 172.21.15.73:0/4170160149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa624004160 con 0x7fa628107dc0 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.962+0000 7fa62cbe1700 1 -- 172.21.15.73:0/4170160149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa624004300 con 0x7fa628107dc0 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.962+0000 7fa62cbe1700 1 -- 172.21.15.73:0/4170160149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa624011a80 con 0x7fa628107dc0 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- 172.21.15.73:0/4170160149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 msgr2=0x7fa6281081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 --2- 172.21.15.73:0/4170160149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa6240098a0 tx=0x7fa624009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- 172.21.15.73:0/4170160149 shutdown_connections 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 --2- 172.21.15.73:0/4170160149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa6281081e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- 172.21.15.73:0/4170160149 >> 172.21.15.73:0/4170160149 conn(0x7fa628103360 msgr2=0x7fa6281057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- 172.21.15.73:0/4170160149 shutdown_connections 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- 172.21.15.73:0/4170160149 wait complete. 2024-06-06T06:49:28.748 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 Processor -- start 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- start start 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.963+0000 7fa62fe47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa62810c2d0 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62dbe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62dbe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40526/0 (socket says 172.21.15.73:40526) 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62dbe3700 1 -- 172.21.15.73:0/1182722890 learned_addr learned my addr 172.21.15.73:0/1182722890 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62dbe3700 1 -- 172.21.15.73:0/1182722890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62819c260 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62dbe3700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa624006af0 tx=0x7fa624011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa624003930 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa624003ad0 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62819c4f0 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6240195e0 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.964+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62819f1d0 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.965+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa624019940 con 0x7fa628107dc0 2024-06-06T06:49:28.749 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.965+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa628063af0 con 0x7fa628107dc0 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.965+0000 7fa61effd700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 0x7fa61403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.965+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa62404d550 con 0x7fa628107dc0 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.965+0000 7fa62d3e2700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 0x7fa61403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.966+0000 7fa62d3e2700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 0x7fa61403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa618006fd0 tx=0x7fa618006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:27.970+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa624016df0 con 0x7fa628107dc0 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.170+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fa628104600 con 0x7fa614039000 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.170+0000 7fa61effd700 1 -- 172.21.15.73:0/1182722890 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fa628104600 con 0x7fa614039000 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 msgr2=0x7fa61403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 0x7fa61403b4c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa618006fd0 tx=0x7fa618006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 msgr2=0x7fa62819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa624006af0 tx=0x7fa624011b70 comp rx=0 tx=0).stop 2024-06-06T06:49:28.750 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 shutdown_connections 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fa614039000 0x7fa61403b4c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 --2- 172.21.15.73:0/1182722890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa628107dc0 0x7fa62819bd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 >> 172.21.15.73:0/1182722890 conn(0x7fa628103360 msgr2=0x7fa628103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 shutdown_connections 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:28.173+0000 7fa62fe47700 1 -- 172.21.15.73:0/1182722890 wait complete. 2024-06-06T06:49:28.751 INFO:teuthology.orchestra.run.smithi073.stdout:Generating ssh key... 2024-06-06T06:49:29.696 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: Generating public/private rsa key pair. 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: Your identification has been saved in /tmp/tmpp7j7bm6s/key. 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: Your public key has been saved in /tmp/tmpp7j7bm6s/key.pub. 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: The key fingerprint is: 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: SHA256:pzt9UU98vPjn8jWV7g2B8LMkgzG5mC/qVwU/DsfL4o8 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: The key's randomart image is: 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: +---[RSA 3072]----+ 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | | 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | .. | 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | ++. o | 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | o.=*o .. *| 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | o S*+o=.o++| 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | .oo++.+.oo| 2024-06-06T06:49:29.697 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | .ooo ..oo.| 2024-06-06T06:49:29.698 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | ...oo. . oo=| 2024-06-06T06:49:29.698 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: | .o. Eo.. ==| 2024-06-06T06:49:29.698 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: +----[SHA256]-----+ 2024-06-06T06:49:29.951 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-mon[32303]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:29.951 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:29.951 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:29 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:30.081 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de9791700 1 Processor -- start 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de9791700 1 -- start start 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de9791700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de9791700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0de4108ce0 con 0x7f0de41082f0 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40542/0 (socket says 172.21.15.73:40542) 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.167+0000 7f0de2ffd700 1 -- 172.21.15.73:0/139155605 learned_addr learned my addr 172.21.15.73:0/139155605 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.168+0000 7f0de2ffd700 1 -- 172.21.15.73:0/139155605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de4106ec0 con 0x7f0de41082f0 2024-06-06T06:49:30.082 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.168+0000 7f0de2ffd700 1 --2- 172.21.15.73:0/139155605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e80 tx=0x7f0dd400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296a19154381acd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.168+0000 7f0de1ffb700 1 -- 172.21.15.73:0/139155605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dd4017070 con 0x7f0de41082f0 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.168+0000 7f0de1ffb700 1 -- 172.21.15.73:0/139155605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0dd400b8c0 con 0x7f0de41082f0 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.168+0000 7f0de1ffb700 1 -- 172.21.15.73:0/139155605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dd4004520 con 0x7f0de41082f0 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- 172.21.15.73:0/139155605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 msgr2=0x7f0de4108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 --2- 172.21.15.73:0/139155605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e80 tx=0x7f0dd400b330 comp rx=0 tx=0).stop 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- 172.21.15.73:0/139155605 shutdown_connections 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 --2- 172.21.15.73:0/139155605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de4108710 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- 172.21.15.73:0/139155605 >> 172.21.15.73:0/139155605 conn(0x7f0de41019f0 msgr2=0x7f0de4103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- 172.21.15.73:0/139155605 shutdown_connections 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- 172.21.15.73:0/139155605 wait complete. 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 Processor -- start 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.169+0000 7f0de9791700 1 -- start start 2024-06-06T06:49:30.083 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de9791700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de9791700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0de410b280 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40550/0 (socket says 172.21.15.73:40550) 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de2ffd700 1 -- 172.21.15.73:0/1993725092 learned_addr learned my addr 172.21.15.73:0/1993725092 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de2ffd700 1 -- 172.21.15.73:0/1993725092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0de419c200 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de2ffd700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e50 tx=0x7f0dd4004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dd4020020 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f0dd400bba0 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0de419c490 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dd4017040 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.170+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0de419f180 con 0x7f0de41082f0 2024-06-06T06:49:30.084 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.171+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f0dd4023020 con 0x7f0de41082f0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.171+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de4063af0 con 0x7f0de41082f0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.171+0000 7f0ddbfff700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 0x7f0dd003b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.171+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0dd400f050 con 0x7f0de41082f0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.171+0000 7f0de27fc700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 0x7f0dd003b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.172+0000 7f0de27fc700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 0x7f0dd003b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006fd0 tx=0x7f0dcc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.176+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0dd402a020 con 0x7f0de41082f0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.374+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f0de402e080 con 0x7f0dd0038fb0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.474+0000 7f0ddbfff700 1 -- 172.21.15.73:0/1993725092 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0de402e080 con 0x7f0dd0038fb0 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.476+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 msgr2=0x7f0dd003b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.476+0000 7f0de9791700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 0x7f0dd003b470 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc006fd0 tx=0x7f0dcc006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.476+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 msgr2=0x7f0de419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:30.085 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.476+0000 7f0de9791700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0dd4009e50 tx=0x7f0dd4004670 comp rx=0 tx=0).stop 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 shutdown_connections 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f0dd0038fb0 0x7f0dd003b470 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 --2- 172.21.15.73:0/1993725092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de41082f0 0x7f0de419bcc0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 >> 172.21.15.73:0/1993725092 conn(0x7f0de41019f0 msgr2=0x7f0de4102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 shutdown_connections 2024-06-06T06:49:30.086 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:29.477+0000 7f0de9791700 1 -- 172.21.15.73:0/1993725092 wait complete. 2024-06-06T06:49:31.003 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:30 smithi073 ceph-mon[32303]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:31.003 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:30 smithi073 ceph-mon[32303]: Generating ssh key... 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.500+0000 7f40b50b7700 1 Processor -- start 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.500+0000 7f40b50b7700 1 -- start start 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40b50b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40b50b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01069f0 con 0x7f40b0106000 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40554/0 (socket says 172.21.15.73:40554) 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40aed9d700 1 -- 172.21.15.73:0/221367757 learned_addr learned my addr 172.21.15.73:0/221367757 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:31.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40aed9d700 1 -- 172.21.15.73:0/221367757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b0108f90 con 0x7f40b0106000 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40aed9d700 1 --2- 172.21.15.73:0/221367757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40a40098a0 tx=0x7f40a4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b30e0555eef03f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40add9b700 1 -- 172.21.15.73:0/221367757 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a4004160 con 0x7f40b0106000 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40add9b700 1 -- 172.21.15.73:0/221367757 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f40a4004300 con 0x7f40b0106000 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.501+0000 7f40add9b700 1 -- 172.21.15.73:0/221367757 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a4011a80 con 0x7f40b0106000 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 -- 172.21.15.73:0/221367757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 msgr2=0x7f40b0106420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 --2- 172.21.15.73:0/221367757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40a40098a0 tx=0x7f40a4009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 -- 172.21.15.73:0/221367757 shutdown_connections 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 --2- 172.21.15.73:0/221367757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b0106420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 -- 172.21.15.73:0/221367757 >> 172.21.15.73:0/221367757 conn(0x7f40b0101990 msgr2=0x7f40b0103db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 -- 172.21.15.73:0/221367757 shutdown_connections 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.502+0000 7f40b50b7700 1 -- 172.21.15.73:0/221367757 wait complete. 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40b50b7700 1 Processor -- start 2024-06-06T06:49:31.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40b50b7700 1 -- start start 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40b50b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40b50b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40b010c340 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40566/0 (socket says 172.21.15.73:40566) 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40aed9d700 1 -- 172.21.15.73:0/1183540354 learned_addr learned my addr 172.21.15.73:0/1183540354 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.503+0000 7f40aed9d700 1 -- 172.21.15.73:0/1183540354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b019c230 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f40aed9d700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a40039c0 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f40a4003b60 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b019c4c0 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40a401a6f0 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.504+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b019f1a0 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f40a401a890 con 0x7f40b0106000 2024-06-06T06:49:31.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b0063af0 con 0x7f40b0106000 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f4097fff700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 0x7f409803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f40a404d380 con 0x7f40b0106000 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f40ae59c700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 0x7f409803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.505+0000 7f40ae59c700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 0x7f409803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.510+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40a4016df0 con 0x7f40b0106000 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.706+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f40b0102bd0 con 0x7f40980390a0 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.707+0000 7f4097fff700 1 -- 172.21.15.73:0/1183540354 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f40b0102bd0 con 0x7f40980390a0 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 msgr2=0x7f409803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 0x7f409803b560 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f40a0006fd0 tx=0x7f40a0006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 msgr2=0x7f40b019bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4004290 comp rx=0 tx=0).stop 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 shutdown_connections 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f40980390a0 0x7f409803b560 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:31.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 --2- 172.21.15.73:0/1183540354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40b0106000 0x7f40b019bcf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 >> 172.21.15.73:0/1183540354 conn(0x7f40b0101990 msgr2=0x7f40b01024c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 shutdown_connections 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:30.709+0000 7f40b50b7700 1 -- 172.21.15.73:0/1183540354 wait complete. 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:Adding key to root@localhost authorized_keys... 2024-06-06T06:49:31.274 INFO:teuthology.orchestra.run.smithi073.stdout:Adding host smithi073... 2024-06-06T06:49:31.919 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:31 smithi073 ceph-mon[32303]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:32 smithi073 ceph-mon[32303]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi073", "addr": "172.21.15.73", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:32 smithi073 ceph-mon[32303]: Deploying cephadm binary to smithi073 2024-06-06T06:49:36.604 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout Added host 'smithi073' with addr '172.21.15.73' 2024-06-06T06:49:36.604 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.710+0000 7fe7f1853700 1 Processor -- start 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7f1853700 1 -- start start 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7f1853700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7f1853700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec07c670 con 0x7fe7ec108830 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40576/0 (socket says 172.21.15.73:40576) 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7eaffd700 1 -- 172.21.15.73:0/2992338276 learned_addr learned my addr 172.21.15.73:0/2992338276 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7eaffd700 1 -- 172.21.15.73:0/2992338276 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec10b160 con 0x7fe7ec108830 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.711+0000 7fe7eaffd700 1 --2- 172.21.15.73:0/2992338276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098a0 tx=0x7fe7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488931ccac255b9d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7ea7fc700 1 -- 172.21.15.73:0/2992338276 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc004160 con 0x7fe7ec108830 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7ea7fc700 1 -- 172.21.15.73:0/2992338276 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe7dc004300 con 0x7fe7ec108830 2024-06-06T06:49:36.605 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7ea7fc700 1 -- 172.21.15.73:0/2992338276 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc011a80 con 0x7fe7ec108830 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 -- 172.21.15.73:0/2992338276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 msgr2=0x7fe7ec10ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2992338276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc0098a0 tx=0x7fe7dc009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 -- 172.21.15.73:0/2992338276 shutdown_connections 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2992338276 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec10ac20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 -- 172.21.15.73:0/2992338276 >> 172.21.15.73:0/2992338276 conn(0x7fe7ec102280 msgr2=0x7fe7ec1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 -- 172.21.15.73:0/2992338276 shutdown_connections 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.712+0000 7fe7f1853700 1 -- 172.21.15.73:0/2992338276 wait complete. 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7f1853700 1 Processor -- start 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7f1853700 1 -- start start 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7f1853700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:36.606 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7f1853700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec10c0c0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40582/0 (socket says 172.21.15.73:40582) 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7eaffd700 1 -- 172.21.15.73:0/2428995081 learned_addr learned my addr 172.21.15.73:0/2428995081 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.713+0000 7fe7eaffd700 1 -- 172.21.15.73:0/2428995081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ec19c1a0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7eaffd700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006af0 tx=0x7fe7dc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc0039e0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fe7dc003b80 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ec19c430 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe7dc0196a0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.714+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ec19f0d0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe7dc019950 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ec063af0 con 0x7fe7ec108830 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7f0851700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 0x7fe7d803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:36.607 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe7dc04d730 con 0x7fe7ec108830 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7e3fff700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 0x7fe7d803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.715+0000 7fe7e3fff700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 0x7fe7d803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.720+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe7dc023c90 con 0x7fe7ec108830 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:31.918+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi073", "addr": "172.21.15.73", "target": ["mon-mgr", ""]}) -- 0x7fe7ec1034d0 con 0x7fe7d8039050 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.044+0000 7fe7f0851700 1 -- 172.21.15.73:0/2428995081 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe7ec1034d0 con 0x7fe7d8039050 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 msgr2=0x7fe7d803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 0x7fe7d803b510 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe7d4006fd0 tx=0x7fe7d4006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 msgr2=0x7fe7ec19bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc006af0 tx=0x7fe7dc011c40 comp rx=0 tx=0).stop 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 shutdown_connections 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe7d8039050 0x7fe7d803b510 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:36.608 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 --2- 172.21.15.73:0/2428995081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7ec108830 0x7fe7ec19bc60 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:36.609 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 >> 172.21.15.73:0/2428995081 conn(0x7fe7ec102280 msgr2=0x7fe7ec102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:36.609 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 shutdown_connections 2024-06-06T06:49:36.609 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:36.047+0000 7fe7f1853700 1 -- 172.21.15.73:0/2428995081 wait complete. 2024-06-06T06:49:36.609 INFO:teuthology.orchestra.run.smithi073.stdout:Deploying unmanaged mon service... 2024-06-06T06:49:37.171 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:37 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:37.171 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:37 smithi073 ceph-mon[32303]: Added host smithi073 2024-06-06T06:49:37.171 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:37 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:38.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:38 smithi073 ceph-mon[32303]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:38.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:38 smithi073 ceph-mon[32303]: Saving service mon spec with placement count:5 2024-06-06T06:49:38.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:38 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:38.870 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-06T06:49:38.870 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bfc6c700 1 Processor -- start 2024-06-06T06:49:38.870 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bfc6c700 1 -- start start 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bfc6c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bfc6c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55b8108770 con 0x7f55b8107d80 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bda08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bda08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33222/0 (socket says 172.21.15.73:33222) 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.170+0000 7f55bda08700 1 -- 172.21.15.73:0/2094573285 learned_addr learned my addr 172.21.15.73:0/2094573285 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bda08700 1 -- 172.21.15.73:0/2094573285 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b8108f30 con 0x7f55b8107d80 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bda08700 1 --2- 172.21.15.73:0/2094573285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55b40098a0 tx=0x7f55b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2474e68a5bf7b7a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bca06700 1 -- 172.21.15.73:0/2094573285 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4004160 con 0x7f55b8107d80 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bca06700 1 -- 172.21.15.73:0/2094573285 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f55b4004300 con 0x7f55b8107d80 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bca06700 1 -- 172.21.15.73:0/2094573285 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b4011a80 con 0x7f55b8107d80 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2094573285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 msgr2=0x7f55b81081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2094573285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f55b40098a0 tx=0x7f55b4009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:38.871 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2094573285 shutdown_connections 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2094573285 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b81081a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.171+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2094573285 >> 172.21.15.73:0/2094573285 conn(0x7f55b81033a0 msgr2=0x7f55b81057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2094573285 shutdown_connections 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2094573285 wait complete. 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 Processor -- start 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 -- start start 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bfc6c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55b810c2e0 con 0x7f55b8107d80 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bda08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bda08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33234/0 (socket says 172.21.15.73:33234) 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.172+0000 7f55bda08700 1 -- 172.21.15.73:0/2829268775 learned_addr learned my addr 172.21.15.73:0/2829268775 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55bda08700 1 -- 172.21.15.73:0/2829268775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55b819c1e0 con 0x7f55b8107d80 2024-06-06T06:49:38.872 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55bda08700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b40039f0 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f55b4003b90 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55b819c470 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55b401a740 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.173+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55b819f160 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f55b4024af0 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55b8063af0 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55aeffd700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 0x7f55a403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f55b404d380 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55bd207700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 0x7f55a403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.174+0000 7f55bd207700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 0x7f55a403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.179+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55b4020db0 con 0x7f55b8107d80 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.398+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f55b81045a0 con 0x7f55a40390a0 2024-06-06T06:49:38.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.403+0000 7f55aeffd700 1 -- 172.21.15.73:0/2829268775 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f55b81045a0 con 0x7f55a40390a0 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 msgr2=0x7f55a403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 0x7f55a403b560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f55a8006fd0 tx=0x7f55a8006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 msgr2=0x7f55b819bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f55b4000c00 tx=0x7f55b4011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 shutdown_connections 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f55a40390a0 0x7f55a403b560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 --2- 172.21.15.73:0/2829268775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b8107d80 0x7f55b819bca0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 >> 172.21.15.73:0/2829268775 conn(0x7f55b81033a0 msgr2=0x7f55b8103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 shutdown_connections 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:37.405+0000 7f55bfc6c700 1 -- 172.21.15.73:0/2829268775 wait complete. 2024-06-06T06:49:38.874 INFO:teuthology.orchestra.run.smithi073.stdout:Deploying unmanaged mgr service... 2024-06-06T06:49:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:39 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:39.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:39 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.311+0000 7fe736a46700 1 Processor -- start 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.311+0000 7fe736a46700 1 -- start start 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe736a46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe736a46700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe730108ba0 con 0x7fe7301081b0 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33248/0 (socket says 172.21.15.73:33248) 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe72ffff700 1 -- 172.21.15.73:0/653325698 learned_addr learned my addr 172.21.15.73:0/653325698 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe72ffff700 1 -- 172.21.15.73:0/653325698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe730106e10 con 0x7fe7301081b0 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.312+0000 7fe72ffff700 1 --2- 172.21.15.73:0/653325698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098a0 tx=0x7fe720009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e76c68e962815b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:40.127 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.313+0000 7fe72effd700 1 -- 172.21.15.73:0/653325698 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe720004160 con 0x7fe7301081b0 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.313+0000 7fe72effd700 1 -- 172.21.15.73:0/653325698 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe720004300 con 0x7fe7301081b0 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.313+0000 7fe72effd700 1 -- 172.21.15.73:0/653325698 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe720011a50 con 0x7fe7301081b0 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.313+0000 7fe736a46700 1 -- 172.21.15.73:0/653325698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 msgr2=0x7fe7301085d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.313+0000 7fe736a46700 1 --2- 172.21.15.73:0/653325698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe7200098a0 tx=0x7fe720009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 -- 172.21.15.73:0/653325698 shutdown_connections 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 --2- 172.21.15.73:0/653325698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe7301085d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 -- 172.21.15.73:0/653325698 >> 172.21.15.73:0/653325698 conn(0x7fe730101960 msgr2=0x7fe730103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 -- 172.21.15.73:0/653325698 shutdown_connections 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 -- 172.21.15.73:0/653325698 wait complete. 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 Processor -- start 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.314+0000 7fe736a46700 1 -- start start 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe736a46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:40.128 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe736a46700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe73010b1d0 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33262/0 (socket says 172.21.15.73:33262) 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe72ffff700 1 -- 172.21.15.73:0/4251012084 learned_addr learned my addr 172.21.15.73:0/4251012084 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe72ffff700 1 -- 172.21.15.73:0/4251012084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe73019b0b0 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.315+0000 7fe72ffff700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe720011a00 tx=0x7fe720011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe720003af0 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe72001a8e0 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe73019b340 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe720024840 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe73019e030 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.316+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fe720022020 con 0x7fe7301081b0 2024-06-06T06:49:40.129 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.317+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe730063af0 con 0x7fe7301081b0 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.317+0000 7fe72d7fa700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 0x7fe71803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.317+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe72004d380 con 0x7fe7301081b0 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.317+0000 7fe72f7fe700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 0x7fe71803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.317+0000 7fe72f7fe700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 0x7fe71803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe724006fd0 tx=0x7fe724006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.326+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe72001adb0 con 0x7fe7301081b0 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.547+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fe730102b80 con 0x7fe718039050 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.554+0000 7fe72d7fa700 1 -- 172.21.15.73:0/4251012084 <== mgr.14120 v2:172.21.15.73:6800/3054677757 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fe730102b80 con 0x7fe718039050 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 msgr2=0x7fe71803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 0x7fe71803b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe724006fd0 tx=0x7fe724006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 msgr2=0x7fe73019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe720011a00 tx=0x7fe720011ae0 comp rx=0 tx=0).stop 2024-06-06T06:49:40.130 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 shutdown_connections 2024-06-06T06:49:40.131 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fe718039050 0x7fe71803b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:40.131 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 --2- 172.21.15.73:0/4251012084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7301081b0 0x7fe73019ab70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:40.131 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 >> 172.21.15.73:0/4251012084 conn(0x7fe730101960 msgr2=0x7fe730102470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:40.131 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 shutdown_connections 2024-06-06T06:49:40.131 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:39.557+0000 7fe736a46700 1 -- 172.21.15.73:0/4251012084 wait complete. 2024-06-06T06:49:40.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:40 smithi073 ceph-mon[32303]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:40.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:40 smithi073 ceph-mon[32303]: Saving service mgr spec with placement count:2 2024-06-06T06:49:40.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:40 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:41.873 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.542+0000 7f75a0551700 1 Processor -- start 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.542+0000 7f75a0551700 1 -- start start 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f75a0551700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f75a0551700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7598108770 con 0x7f7598107d80 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759e2ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759e2ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33270/0 (socket says 172.21.15.73:33270) 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759e2ed700 1 -- 172.21.15.73:0/3162277592 learned_addr learned my addr 172.21.15.73:0/3162277592 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759e2ed700 1 -- 172.21.15.73:0/3162277592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7598108f30 con 0x7f7598107d80 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759e2ed700 1 --2- 172.21.15.73:0/3162277592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098a0 tx=0x7f758c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=306a103844f64ecb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.543+0000 7f759d2eb700 1 -- 172.21.15.73:0/3162277592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c004160 con 0x7f7598107d80 2024-06-06T06:49:41.874 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f759d2eb700 1 -- 172.21.15.73:0/3162277592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f758c004300 con 0x7f7598107d80 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f759d2eb700 1 -- 172.21.15.73:0/3162277592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c011a50 con 0x7f7598107d80 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 -- 172.21.15.73:0/3162277592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 msgr2=0x7f75981081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 --2- 172.21.15.73:0/3162277592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f758c0098a0 tx=0x7f758c009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 -- 172.21.15.73:0/3162277592 shutdown_connections 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 --2- 172.21.15.73:0/3162277592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f75981081a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 -- 172.21.15.73:0/3162277592 >> 172.21.15.73:0/3162277592 conn(0x7f75981033a0 msgr2=0x7f75981057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 -- 172.21.15.73:0/3162277592 shutdown_connections 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.544+0000 7f75a0551700 1 -- 172.21.15.73:0/3162277592 wait complete. 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f75a0551700 1 Processor -- start 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f75a0551700 1 -- start start 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f75a0551700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:41.875 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f75a0551700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f759810c2e0 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f759e2ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f759e2ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33280/0 (socket says 172.21.15.73:33280) 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f759e2ed700 1 -- 172.21.15.73:0/4060625588 learned_addr learned my addr 172.21.15.73:0/4060625588 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f759e2ed700 1 -- 172.21.15.73:0/4060625588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759807e920 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.545+0000 7f759e2ed700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f758c006b20 tx=0x7f758c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c0039e0 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f758c01a8e0 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f758c0247e0 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759807cb10 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f759807d050 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.546+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f758c022020 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.547+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7598063af0 con 0x7f7598107d80 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.547+0000 7f758b7fe700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 0x7f758403b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:41.876 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.547+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f758c04d410 con 0x7f7598107d80 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.547+0000 7f759daec700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 0x7f758403b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.547+0000 7f759daec700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 0x7f758403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7594006fd0 tx=0x7f7594006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.552+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f758c00f2e0 con 0x7f7598107d80 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.758+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f759807d8f0 con 0x7f7598107d80 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.764+0000 7f758b7fe700 1 -- 172.21.15.73:0/4060625588 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f758c02d3f0 con 0x7f7598107d80 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 msgr2=0x7f758403b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 0x7f758403b560 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7594006fd0 tx=0x7f7594006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 msgr2=0x7f759807e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f758c006b20 tx=0x7f758c011ad0 comp rx=0 tx=0).stop 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 shutdown_connections 2024-06-06T06:49:41.877 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f75840390a0 0x7f758403b560 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:41.878 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 --2- 172.21.15.73:0/4060625588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7598107d80 0x7f759807e3e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:41.878 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 >> 172.21.15.73:0/4060625588 conn(0x7f75981033a0 msgr2=0x7f7598103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:41.878 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 shutdown_connections 2024-06-06T06:49:41.878 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:40.766+0000 7f75a0551700 1 -- 172.21.15.73:0/4060625588 wait complete. 2024-06-06T06:49:41.992 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:41 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4060625588' entity='client.admin' 2024-06-06T06:49:43.263 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:42 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:43.263 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:43 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3135588101' entity='client.admin' 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.279+0000 7fb77146c700 1 Processor -- start 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.279+0000 7fb77146c700 1 -- start start 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.279+0000 7fb77146c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.279+0000 7fb77146c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c1069e0 con 0x7fb76c105ff0 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb76affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:43.270 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb76affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33282/0 (socket says 172.21.15.73:33282) 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb76affd700 1 -- 172.21.15.73:0/1156791308 learned_addr learned my addr 172.21.15.73:0/1156791308 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb76affd700 1 -- 172.21.15.73:0/1156791308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c108f80 con 0x7fb76c105ff0 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb76affd700 1 --2- 172.21.15.73:0/1156791308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201968eec80b04aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb769ffb700 1 -- 172.21.15.73:0/1156791308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754004160 con 0x7fb76c105ff0 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb769ffb700 1 -- 172.21.15.73:0/1156791308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb754004300 con 0x7fb76c105ff0 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.280+0000 7fb769ffb700 1 -- 172.21.15.73:0/1156791308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb754011a50 con 0x7fb76c105ff0 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 -- 172.21.15.73:0/1156791308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 msgr2=0x7fb76c106410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 --2- 172.21.15.73:0/1156791308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb7540098a0 tx=0x7fb754009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 -- 172.21.15.73:0/1156791308 shutdown_connections 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 --2- 172.21.15.73:0/1156791308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c106410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 -- 172.21.15.73:0/1156791308 >> 172.21.15.73:0/1156791308 conn(0x7fb76c101960 msgr2=0x7fb76c103d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:43.271 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 -- 172.21.15.73:0/1156791308 shutdown_connections 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.281+0000 7fb77146c700 1 -- 172.21.15.73:0/1156791308 wait complete. 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb77146c700 1 Processor -- start 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb77146c700 1 -- start start 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb77146c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb77146c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb76c10c330 con 0x7fb76c105ff0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb76affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb76affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33286/0 (socket says 172.21.15.73:33286) 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb76affd700 1 -- 172.21.15.73:0/3135588101 learned_addr learned my addr 172.21.15.73:0/3135588101 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.282+0000 7fb76affd700 1 -- 172.21.15.73:0/3135588101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb76c19c1f0 con 0x7fb76c105ff0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb76affd700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb754011710 tx=0x7fb754011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb7540039c0 con 0x7fb76c105ff0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb76c19c480 con 0x7fb76c105ff0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb754003b60 con 0x7fb76c105ff0 2024-06-06T06:49:43.272 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb76c19f170 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.283+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb75401a690 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.284+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb76c063af0 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.284+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb75401aa10 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.284+0000 7fb763fff700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 0x7fb75803b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.284+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb75404d6a0 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.284+0000 7fb76a7fc700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 0x7fb75803b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.285+0000 7fb76a7fc700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 0x7fb75803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb75c006fd0 tx=0x7fb75c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.294+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb75401adb0 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.495+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb76c19f3e0 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.497+0000 7fb763fff700 1 -- 172.21.15.73:0/3135588101 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb75400bc40 con 0x7fb76c105ff0 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 msgr2=0x7fb75803b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 0x7fb75803b4c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb75c006fd0 tx=0x7fb75c006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:43.273 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 msgr2=0x7fb76c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb754011710 tx=0x7fb754011ad0 comp rx=0 tx=0).stop 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 shutdown_connections 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fb758039000 0x7fb75803b4c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 --2- 172.21.15.73:0/3135588101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb76c105ff0 0x7fb76c19bcb0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 >> 172.21.15.73:0/3135588101 conn(0x7fb76c101960 msgr2=0x7fb76c102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 shutdown_connections 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:42.500+0000 7fb77146c700 1 -- 172.21.15.73:0/3135588101 wait complete. 2024-06-06T06:49:43.274 INFO:teuthology.orchestra.run.smithi073.stdout:Enabling the dashboard module... 2024-06-06T06:49:44.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-mon[32303]: from='mgr.14120 172.21.15.73:0/627143311' entity='mgr.x' 2024-06-06T06:49:44.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/815553196' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-06T06:49:44.525 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setuser ceph since I am not root 2024-06-06T06:49:44.525 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setgroup ceph since I am not root 2024-06-06T06:49:44.525 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:44.432+0000 7fac2bf7c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T06:49:44.915 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:44.622+0000 7fac2bf7c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:49:44.915 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:44.725+0000 7fac2bf7c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T06:49:45.243 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:45.036+0000 7fac2bf7c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T06:49:45.243 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:45.122+0000 7fac2bf7c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T06:49:45.510 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:45 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/815553196' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-06T06:49:45.510 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:45 smithi073 ceph-mon[32303]: mgrmap e9: x(active, since 19s) 2024-06-06T06:49:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d0c06a700 1 Processor -- start 2024-06-06T06:49:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d0c06a700 1 -- start start 2024-06-06T06:49:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d0c06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d0c06a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0407c6d0 con 0x7f1d041087c0 2024-06-06T06:49:45.520 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d09e06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d09e06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33298/0 (socket says 172.21.15.73:33298) 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.738+0000 7f1d09e06700 1 -- 172.21.15.73:0/3276392021 learned_addr learned my addr 172.21.15.73:0/3276392021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d09e06700 1 -- 172.21.15.73:0/3276392021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0410b0f0 con 0x7f1d041087c0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d09e06700 1 --2- 172.21.15.73:0/3276392021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098a0 tx=0x7f1cf8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a074a7181345821 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d08e04700 1 -- 172.21.15.73:0/3276392021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf8004160 con 0x7f1d041087c0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d08e04700 1 -- 172.21.15.73:0/3276392021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf8004300 con 0x7f1d041087c0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d08e04700 1 -- 172.21.15.73:0/3276392021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf8011a50 con 0x7f1d041087c0 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d0c06a700 1 -- 172.21.15.73:0/3276392021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 msgr2=0x7f1d0410abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/3276392021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80098a0 tx=0x7f1cf8009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d0c06a700 1 -- 172.21.15.73:0/3276392021 shutdown_connections 2024-06-06T06:49:45.521 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/3276392021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0410abb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.739+0000 7f1d0c06a700 1 -- 172.21.15.73:0/3276392021 >> 172.21.15.73:0/3276392021 conn(0x7f1d041022b0 msgr2=0x7f1d041046d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 -- 172.21.15.73:0/3276392021 shutdown_connections 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 -- 172.21.15.73:0/3276392021 wait complete. 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 Processor -- start 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 -- start start 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d0c06a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d0410c0f0 con 0x7f1d041087c0 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d09e06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d09e06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33300/0 (socket says 172.21.15.73:33300) 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.740+0000 7f1d09e06700 1 -- 172.21.15.73:0/815553196 learned_addr learned my addr 172.21.15.73:0/815553196 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1d09e06700 1 -- 172.21.15.73:0/815553196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0419c110 con 0x7f1d041087c0 2024-06-06T06:49:45.522 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1d09e06700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8011a20 tx=0x7f1cf8011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf8003b70 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1cf801a8e0 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0419c3a0 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1cf8024990 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.741+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0419f090 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.742+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f1cf8022020 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.742+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d04063af0 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.742+0000 7f1cf6ffd700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 0x7f1cf003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.742+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1cf804d350 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.742+0000 7f1d09605700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 0x7f1cf003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.743+0000 7f1d09605700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 0x7f1cf003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006fd0 tx=0x7f1d00006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:43.747+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1cf800f2e0 con 0x7f1d041087c0 2024-06-06T06:49:45.523 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.012+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f1d0419f640 con 0x7f1d041087c0 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.253+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f1cf802d3f0 con 0x7f1d041087c0 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.255+0000 7f1cf6ffd700 1 -- 172.21.15.73:0/815553196 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f1cf801aa90 con 0x7f1d041087c0 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.256+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 msgr2=0x7f1cf003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.256+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 0x7f1cf003b510 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006fd0 tx=0x7f1d00006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 msgr2=0x7f1d0419bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8011a20 tx=0x7f1cf8011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 shutdown_connections 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f1cf0039050 0x7f1cf003b510 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 --2- 172.21.15.73:0/815553196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d041087c0 0x7f1d0419bbd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 >> 172.21.15.73:0/815553196 conn(0x7f1d041022b0 msgr2=0x7f1d04102cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 shutdown_connections 2024-06-06T06:49:45.524 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:44.257+0000 7f1d0c06a700 1 -- 172.21.15.73:0/815553196 wait complete. 2024-06-06T06:49:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1526822750' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-06T06:49:46.525 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:46.346+0000 7fac2bf7c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T06:49:46.800 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:46.525+0000 7fac2bf7c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:46.801 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:46.705+0000 7fac2bf7c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T06:49:46.803 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.995+0000 7fd845cf2700 1 Processor -- start 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.995+0000 7fd845cf2700 1 -- start start 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.995+0000 7fd845cf2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.995+0000 7fd845cf2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8401075f0 con 0x7fd840106c00 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37416/0 (socket says 172.21.15.73:37416) 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83f7fe700 1 -- 172.21.15.73:0/961366007 learned_addr learned my addr 172.21.15.73:0/961366007 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83f7fe700 1 -- 172.21.15.73:0/961366007 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd840107e00 con 0x7fd840106c00 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83f7fe700 1 --2- 172.21.15.73:0/961366007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098a0 tx=0x7fd828009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185fa6b37df4650a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83e7fc700 1 -- 172.21.15.73:0/961366007 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd828004160 con 0x7fd840106c00 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83e7fc700 1 -- 172.21.15.73:0/961366007 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd828004300 con 0x7fd840106c00 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.996+0000 7fd83e7fc700 1 -- 172.21.15.73:0/961366007 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd828011a50 con 0x7fd840106c00 2024-06-06T06:49:46.811 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 -- 172.21.15.73:0/961366007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 msgr2=0x7fd840107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 --2- 172.21.15.73:0/961366007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd8280098a0 tx=0x7fd828009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 -- 172.21.15.73:0/961366007 shutdown_connections 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 --2- 172.21.15.73:0/961366007 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd840107020 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 -- 172.21.15.73:0/961366007 >> 172.21.15.73:0/961366007 conn(0x7fd8401021e0 msgr2=0x7fd840104600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 -- 172.21.15.73:0/961366007 shutdown_connections 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.997+0000 7fd845cf2700 1 -- 172.21.15.73:0/961366007 wait complete. 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd845cf2700 1 Processor -- start 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd845cf2700 1 -- start start 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd845cf2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd845cf2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd84010b1b0 con 0x7fd840106c00 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd83f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:46.812 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd83f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37426/0 (socket says 172.21.15.73:37426) 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.998+0000 7fd83f7fe700 1 -- 172.21.15.73:0/1526822750 learned_addr learned my addr 172.21.15.73:0/1526822750 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd83f7fe700 1 -- 172.21.15.73:0/1526822750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84019b0f0 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd83f7fe700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd828011a20 tx=0x7fd828011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd828003af0 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd82801a8e0 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84019b380 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd828024860 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:45.999+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd84019e070 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7fd828022020 con 0x7fd840106c00 2024-06-06T06:49:46.813 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd83cff9700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 0x7fd82c03b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd840063af0 con 0x7fd840106c00 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd83effd700 1 -- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 msgr2=0x7fd82c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd82804e230 con 0x7fd840106c00 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.000+0000 7fd83effd700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 0x7fd82c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.005+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd82800b3e0 con 0x7fd840106c00 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.201+0000 7fd83effd700 1 -- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 msgr2=0x7fd82c03b560 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.201+0000 7fd83effd700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 0x7fd82c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.264+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd84019e5f0 con 0x7fd840106c00 2024-06-06T06:49:46.814 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.264+0000 7fd83cff9700 1 -- 172.21.15.73:0/1526822750 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd828021d80 con 0x7fd840106c00 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 msgr2=0x7fd82c03b560 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 0x7fd82c03b560 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 msgr2=0x7fd84019abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd828011a20 tx=0x7fd828011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 shutdown_connections 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7fd82c0390a0 0x7fd82c03b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 --2- 172.21.15.73:0/1526822750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd840106c00 0x7fd84019abb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.267+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 >> 172.21.15.73:0/1526822750 conn(0x7fd8401021e0 msgr2=0x7fd840102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.268+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 shutdown_connections 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:46.268+0000 7fd845cf2700 1 -- 172.21.15.73:0/1526822750 wait complete. 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for the mgr to restart... 2024-06-06T06:49:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:Waiting for mgr epoch 9... 2024-06-06T06:49:47.074 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:47.020+0000 7fac2bf7c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T06:49:47.341 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:47.118+0000 7fac2bf7c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T06:49:47.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:47.339+0000 7fac2bf7c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T06:49:47.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:47.537+0000 7fac2bf7c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:49:48.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:48.136+0000 7fac2bf7c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T06:49:48.867 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:48.563+0000 7fac2bf7c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T06:49:48.867 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:48.660+0000 7fac2bf7c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T06:49:48.867 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:48.864+0000 7fac2bf7c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T06:49:49.734 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:49.397+0000 7fac2bf7c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T06:49:49.735 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:49.489+0000 7fac2bf7c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T06:49:50.024 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:49.732+0000 7fac2bf7c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T06:49:50.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:49.866+0000 7fac2bf7c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T06:49:50.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:49.963+0000 7fac2bf7c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T06:49:50.391 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:50.078+0000 7fac2bf7c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T06:49:50.392 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:50.389+0000 7fac2bf7c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T06:49:51.268 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:50.994+0000 7fac2bf7c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T06:49:51.268 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:49:51.214+0000 7fac2bf7c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: Active manager daemon x restarted 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: Activating manager daemon x 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: osdmap e3: 0 total, 0 up, 0 in 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: mgrmap e10: x(active, starting, since 0.00328371s) 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: Manager daemon x is now available 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:49:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T06:49:51.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout { 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout } 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.292+0000 7f7c6fca2700 1 Processor -- start 2024-06-06T06:49:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6fca2700 1 -- start start 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6fca2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6fca2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68107600 con 0x7f7c68106c10 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6da3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6da3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37428/0 (socket says 172.21.15.73:37428) 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6da3e700 1 -- 172.21.15.73:0/1156228456 learned_addr learned my addr 172.21.15.73:0/1156228456 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.293+0000 7f7c6da3e700 1 -- 172.21.15.73:0/1156228456 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c68107dc0 con 0x7f7c68106c10 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6da3e700 1 --2- 172.21.15.73:0/1156228456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098a0 tx=0x7f7c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82a8a0e18f143740 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6ca3c700 1 -- 172.21.15.73:0/1156228456 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c64004160 con 0x7f7c68106c10 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6ca3c700 1 -- 172.21.15.73:0/1156228456 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c64004300 con 0x7f7c68106c10 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6ca3c700 1 -- 172.21.15.73:0/1156228456 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c64011a50 con 0x7f7c68106c10 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6fca2700 1 -- 172.21.15.73:0/1156228456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 msgr2=0x7f7c68107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:53.070 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.294+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/1156228456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7c640098a0 tx=0x7f7c64009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- 172.21.15.73:0/1156228456 shutdown_connections 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/1156228456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c68107030 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- 172.21.15.73:0/1156228456 >> 172.21.15.73:0/1156228456 conn(0x7f7c68102250 msgr2=0x7f7c68104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- 172.21.15.73:0/1156228456 shutdown_connections 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- 172.21.15.73:0/1156228456 wait complete. 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 Processor -- start 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- start start 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.295+0000 7f7c6fca2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c6810b170 con 0x7f7c68106c10 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6da3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6da3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37432/0 (socket says 172.21.15.73:37432) 2024-06-06T06:49:53.071 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6da3e700 1 -- 172.21.15.73:0/963084767 learned_addr learned my addr 172.21.15.73:0/963084767 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6da3e700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c6819c130 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6da3e700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006b20 tx=0x7f7c64011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c640039e0 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c6401a8e0 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7c640247e0 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c6819c3c0 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.296+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c6819f0b0 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.297+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f7c64022020 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.297+0000 7f7c5effd700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.297+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c5403bcc0 con 0x7f7c540390f0 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.298+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f7c6404d430 con 0x7f7c68106c10 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.298+0000 7f7c6d23d700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.298+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:49:53.072 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.499+0000 7f7c6d23d700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.499+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.900+0000 7f7c6d23d700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:47.900+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:48.701+0000 7f7c6d23d700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:48.701+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:50.302+0000 7f7c6d23d700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/3054677757 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:50.302+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:51.218+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f7c6401a5c0 con 0x7f7c68106c10 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:51.218+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 msgr2=0x7f7c5403b5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:51.218+0000 7f7c5effd700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.219+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f7c6404f620 con 0x7f7c68106c10 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.219+0000 7f7c5effd700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 0x7f7c5403efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:53.073 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.219+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7c5403bcc0 con 0x7f7c5403cbb0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.220+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 0x7f7c5403efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.220+0000 7f7c6d23d700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 0x7f7c5403efa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003a10 tx=0x7f7c58009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.221+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f7c5403bcc0 con 0x7f7c5403cbb0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f7c6810afc0 con 0x7f7c5403cbb0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c5effd700 1 -- 172.21.15.73:0/963084767 <== mgr.14152 v2:172.21.15.73:6800/793833713 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f7c6810afc0 con 0x7f7c5403cbb0 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 msgr2=0x7f7c5403efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 0x7f7c5403efa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7c58003a10 tx=0x7f7c58009270 comp rx=0 tx=0).stop 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 msgr2=0x7f7c6819bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c64006b20 tx=0x7f7c64011ad0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 shutdown_connections 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7c5403cbb0 0x7f7c5403efa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c68106c10 0x7f7c6819bbf0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.074 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 --2- 172.21.15.73:0/963084767 >> [v2:172.21.15.73:6800/3054677757,v1:172.21.15.73:6801/3054677757] conn(0x7f7c540390f0 0x7f7c5403b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 >> 172.21.15.73:0/963084767 conn(0x7f7c68102250 msgr2=0x7f7c68102cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 shutdown_connections 2024-06-06T06:49:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:52.228+0000 7f7c6fca2700 1 -- 172.21.15.73:0/963084767 wait complete. 2024-06-06T06:49:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:mgr epoch 9 is available 2024-06-06T06:49:53.075 INFO:teuthology.orchestra.run.smithi073.stdout:Generating a dashboard self-signed certificate... 2024-06-06T06:49:53.388 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:53 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:52] ENGINE Bus STARTING 2024-06-06T06:49:53.388 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:53 smithi073 ceph-mon[32303]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-06T06:49:53.389 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:53 smithi073 ceph-mon[32303]: mgrmap e11: x(active, since 1.00603s) 2024-06-06T06:49:53.389 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:53 smithi073 ceph-mon[32303]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-06T06:49:53.389 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:49:54.366 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.491+0000 7f6db8af1700 1 Processor -- start 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.491+0000 7f6db8af1700 1 -- start start 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.491+0000 7f6db8af1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.491+0000 7f6db8af1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6db4108bd0 con 0x7f6db41081e0 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.491+0000 7f6db259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37498/0 (socket says 172.21.15.73:37498) 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db259c700 1 -- 172.21.15.73:0/1327330373 learned_addr learned my addr 172.21.15.73:0/1327330373 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db259c700 1 -- 172.21.15.73:0/1327330373 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db4106e40 con 0x7f6db41081e0 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db259c700 1 --2- 172.21.15.73:0/1327330373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098a0 tx=0x7f6da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8cc9e76603fe0a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db159a700 1 -- 172.21.15.73:0/1327330373 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da4004160 con 0x7f6db41081e0 2024-06-06T06:49:54.367 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db159a700 1 -- 172.21.15.73:0/1327330373 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da4004300 con 0x7f6db41081e0 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.492+0000 7f6db159a700 1 -- 172.21.15.73:0/1327330373 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da4011a50 con 0x7f6db41081e0 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 -- 172.21.15.73:0/1327330373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 msgr2=0x7f6db4108600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 --2- 172.21.15.73:0/1327330373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098a0 tx=0x7f6da4009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 -- 172.21.15.73:0/1327330373 shutdown_connections 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 --2- 172.21.15.73:0/1327330373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db4108600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 -- 172.21.15.73:0/1327330373 >> 172.21.15.73:0/1327330373 conn(0x7f6db4101910 msgr2=0x7f6db4103d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 -- 172.21.15.73:0/1327330373 shutdown_connections 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.493+0000 7f6db8af1700 1 -- 172.21.15.73:0/1327330373 wait complete. 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db8af1700 1 Processor -- start 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db8af1700 1 -- start start 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db8af1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db8af1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6db410b200 con 0x7f6db41081e0 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:54.368 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37510/0 (socket says 172.21.15.73:37510) 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.494+0000 7f6db259c700 1 -- 172.21.15.73:0/2635184626 learned_addr learned my addr 172.21.15.73:0/2635184626 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6db259c700 1 -- 172.21.15.73:0/2635184626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6db419c200 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6db259c700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011a20 tx=0x7f6da4011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da4003af0 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6da401a8e0 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6db419c490 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6da4024860 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.495+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6db419f180 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.496+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f6da4022020 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.496+0000 7f6da37fe700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 0x7f6d9c03b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.496+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6da404e100 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.496+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6db4063af0 con 0x7f6db41081e0 2024-06-06T06:49:54.369 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.496+0000 7f6db1d9b700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 0x7f6d9c03b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.497+0000 7f6db1d9b700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 0x7f6d9c03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.502+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6da400b3e0 con 0x7f6db41081e0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.715+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f6db402de60 con 0x7f6d9c038ff0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.773+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f6da402dc90 con 0x7f6db41081e0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.815+0000 7f6da37fe700 1 -- 172.21.15.73:0/2635184626 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f6db402de60 con 0x7f6d9c038ff0 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 msgr2=0x7f6d9c03b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 0x7f6d9c03b4b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6da8006fd0 tx=0x7f6da8006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 msgr2=0x7f6db419bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6da4011a20 tx=0x7f6da4011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 shutdown_connections 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6d9c038ff0 0x7f6d9c03b4b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 --2- 172.21.15.73:0/2635184626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6db41081e0 0x7f6db419bcc0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:54.370 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 >> 172.21.15.73:0/2635184626 conn(0x7f6db4101910 msgr2=0x7f6db4102760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:54.371 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 shutdown_connections 2024-06-06T06:49:54.371 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:53.818+0000 7f6db8af1700 1 -- 172.21.15.73:0/2635184626 wait complete. 2024-06-06T06:49:54.371 INFO:teuthology.orchestra.run.smithi073.stdout:Creating initial admin user... 2024-06-06T06:49:54.476 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:52] ENGINE Serving on https://172.21.15.73:7150 2024-06-06T06:49:54.476 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:52] ENGINE Serving on http://172.21.15.73:8765 2024-06-06T06:49:54.477 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: [06/Jun/2024:06:49:52] ENGINE Bus STARTED 2024-06-06T06:49:54.477 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: mgrmap e12: x(active, since 2s) 2024-06-06T06:49:54.477 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:49:54.477 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:49:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:55 smithi073 ceph-mon[32303]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$X75DROC6EpsX27Bn0.k35.ooItNPhENGhU/KOX8SupYGoNw3CYBxK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717656595, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9ef8b700 1 Processor -- start 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9ef8b700 1 -- start start 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9ef8b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9ef8b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9807c6d0 con 0x7f1f98108760 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9cd27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:55.805 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9cd27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37514/0 (socket says 172.21.15.73:37514) 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.764+0000 7f1f9cd27700 1 -- 172.21.15.73:0/3792072023 learned_addr learned my addr 172.21.15.73:0/3792072023 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f9cd27700 1 -- 172.21.15.73:0/3792072023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9810b090 con 0x7f1f98108760 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f9cd27700 1 --2- 172.21.15.73:0/3792072023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0098a0 tx=0x7f1f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cca5c2d07a9e642d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f977fe700 1 -- 172.21.15.73:0/3792072023 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c004160 con 0x7f1f98108760 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f977fe700 1 -- 172.21.15.73:0/3792072023 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f8c004300 con 0x7f1f98108760 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f977fe700 1 -- 172.21.15.73:0/3792072023 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c011a50 con 0x7f1f98108760 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/3792072023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 msgr2=0x7f1f9810ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.765+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/3792072023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c0098a0 tx=0x7f1f8c009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/3792072023 shutdown_connections 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/3792072023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9810ab50 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/3792072023 >> 172.21.15.73:0/3792072023 conn(0x7f1f98102230 msgr2=0x7f1f98104650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/3792072023 shutdown_connections 2024-06-06T06:49:55.806 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/3792072023 wait complete. 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 Processor -- start 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- start start 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9ef8b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1f9810c090 con 0x7f1f98108760 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9cd27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9cd27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37516/0 (socket says 172.21.15.73:37516) 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.766+0000 7f1f9cd27700 1 -- 172.21.15.73:0/2988452462 learned_addr learned my addr 172.21.15.73:0/2988452462 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f9cd27700 1 -- 172.21.15.73:0/2988452462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9819c080 con 0x7f1f98108760 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f9cd27700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c011a20 tx=0x7f1f8c011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c003b70 con 0x7f1f98108760 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9819c310 con 0x7f1f98108760 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f8c01a8e0 con 0x7f1f98108760 2024-06-06T06:49:55.807 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1f8c024990 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.767+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f9819f000 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.768+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1f8c022020 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.768+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f98063af0 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.768+0000 7f1f95ffb700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 0x7f1f8803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.768+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1f8c04d4d0 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.768+0000 7f1f97fff700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 0x7f1f8803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.769+0000 7f1f97fff700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 0x7f1f8803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006fd0 tx=0x7f1f84006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:54.773+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1f8c0172b0 con 0x7f1f98108760 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.006+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f1f9819f1d0 con 0x7f1f88039050 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.230+0000 7f1f95ffb700 1 -- 172.21.15.73:0/2988452462 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f1f9819f1d0 con 0x7f1f88039050 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 msgr2=0x7f1f8803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 0x7f1f8803b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f84006fd0 tx=0x7f1f84006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 msgr2=0x7f1f9819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:55.808 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c011a20 tx=0x7f1f8c011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 shutdown_connections 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1f88039050 0x7f1f8803b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 --2- 172.21.15.73:0/2988452462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f98108760 0x7f1f9819bb40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 >> 172.21.15.73:0/2988452462 conn(0x7f1f98102230 msgr2=0x7f1f98102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 shutdown_connections 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:55.232+0000 7f1f9ef8b700 1 -- 172.21.15.73:0/2988452462 wait complete. 2024-06-06T06:49:55.809 INFO:teuthology.orchestra.run.smithi073.stdout:Fetching dashboard port number... 2024-06-06T06:49:56.473 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:56 smithi073 ceph-mon[32303]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:49:56.473 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stdout 8443 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.236+0000 7f4dd54ff700 1 Processor -- start 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.236+0000 7f4dd54ff700 1 -- start start 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dd54ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dd54ff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd007d0b0 con 0x7f4dd007e670 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dceffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dceffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57018/0 (socket says 172.21.15.73:57018) 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dceffd700 1 -- 172.21.15.73:0/2391987270 learned_addr learned my addr 172.21.15.73:0/2391987270 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dceffd700 1 -- 172.21.15.73:0/2391987270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd007d230 con 0x7f4dd007e670 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dceffd700 1 --2- 172.21.15.73:0/2391987270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00098a0 tx=0x7f4dc0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b34e34b2d9a165b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:57.165 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dcdffb700 1 -- 172.21.15.73:0/2391987270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0004160 con 0x7f4dd007e670 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dcdffb700 1 -- 172.21.15.73:0/2391987270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dc0004300 con 0x7f4dd007e670 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.237+0000 7f4dcdffb700 1 -- 172.21.15.73:0/2391987270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0011a50 con 0x7f4dd007e670 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2391987270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 msgr2=0x7f4dd007cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2391987270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00098a0 tx=0x7f4dc0009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2391987270 shutdown_connections 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2391987270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd007cb70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2391987270 >> 172.21.15.73:0/2391987270 conn(0x7f4dd0103380 msgr2=0x7f4dd01057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2391987270 shutdown_connections 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2391987270 wait complete. 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.238+0000 7f4dd54ff700 1 Processor -- start 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dd54ff700 1 -- start start 2024-06-06T06:49:57.166 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dd54ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dd54ff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4dd010e300 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dceffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dceffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57034/0 (socket says 172.21.15.73:57034) 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dceffd700 1 -- 172.21.15.73:0/2697869602 learned_addr learned my addr 172.21.15.73:0/2697869602 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dceffd700 1 -- 172.21.15.73:0/2697869602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dd019e300 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4dceffd700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0011a20 tx=0x7f4dc0011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0003b70 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.239+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dc001a8e0 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.240+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4dc0024990 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.240+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dd019e590 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.240+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dd01a1280 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.240+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f4dc0022020 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.240+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4dd0063af0 con 0x7f4dd007e670 2024-06-06T06:49:57.167 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.241+0000 7f4db7fff700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 0x7f4db803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.241+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4dc004d8e0 con 0x7f4dd007e670 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.241+0000 7f4dce7fc700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 0x7f4db803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.241+0000 7f4dce7fc700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 0x7f4db803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4006fd0 tx=0x7f4dc4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.246+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4dc000b3e0 con 0x7f4dd007e670 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.487+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f4dd01a1830 con 0x7f4dd007e670 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.487+0000 7f4db7fff700 1 -- 172.21.15.73:0/2697869602 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f4dc0029c40 con 0x7f4dd007e670 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 msgr2=0x7f4db803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 0x7f4db803b510 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4006fd0 tx=0x7f4dc4006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 msgr2=0x7f4dd019ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0011a20 tx=0x7f4dc0011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 shutdown_connections 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4db8039050 0x7f4db803b510 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 --2- 172.21.15.73:0/2697869602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dd007e670 0x7f4dd019ddc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:57.168 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.489+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 >> 172.21.15.73:0/2697869602 conn(0x7f4dd0103380 msgr2=0x7f4dd0103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:57.169 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.490+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 shutdown_connections 2024-06-06T06:49:57.169 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:56.490+0000 7f4dd54ff700 1 -- 172.21.15.73:0/2697869602 wait complete. 2024-06-06T06:49:57.172 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-06T06:49:57.173 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout disabled 2024-06-06T06:49:57.180 INFO:teuthology.orchestra.run.smithi073.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-06T06:49:57.180 INFO:teuthology.orchestra.run.smithi073.stdout:systemctl: stdout inactive 2024-06-06T06:49:57.180 INFO:teuthology.orchestra.run.smithi073.stdout:firewalld.service is not enabled 2024-06-06T06:49:57.180 INFO:teuthology.orchestra.run.smithi073.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout:Ceph Dashboard is now available at: 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: URL: https://smithi073.front.sepia.ceph.com:8443/ 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: User: admin 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: Password: xah7gscmsk 2024-06-06T06:49:57.182 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:49:57.183 INFO:teuthology.orchestra.run.smithi073.stdout:Saving cluster configuration to /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config directory 2024-06-06T06:49:57.183 INFO:teuthology.orchestra.run.smithi073.stdout:Enabling autotune for osd_memory_target 2024-06-06T06:49:57.305 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:57 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2697869602' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-06T06:49:58.499 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:49:58 smithi073 ceph-mon[32303]: mgrmap e13: x(active, since 6s) 2024-06-06T06:49:59.060 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29369e700 1 Processor -- start 2024-06-06T06:49:59.060 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29369e700 1 -- start start 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29369e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29369e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c07c610 con 0x7fe28c108870 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29143a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29143a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57044/0 (socket says 172.21.15.73:57044) 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.083+0000 7fe29143a700 1 -- 172.21.15.73:0/4051233702 learned_addr learned my addr 172.21.15.73:0/4051233702 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.084+0000 7fe29143a700 1 -- 172.21.15.73:0/4051233702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c10b1a0 con 0x7fe28c108870 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.084+0000 7fe29143a700 1 --2- 172.21.15.73:0/4051233702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2880098a0 tx=0x7fe288009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e048d8e41e2ae5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.084+0000 7fe283fff700 1 -- 172.21.15.73:0/4051233702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288004160 con 0x7fe28c108870 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.084+0000 7fe283fff700 1 -- 172.21.15.73:0/4051233702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe288004300 con 0x7fe28c108870 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.084+0000 7fe283fff700 1 -- 172.21.15.73:0/4051233702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288011a50 con 0x7fe28c108870 2024-06-06T06:49:59.061 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- 172.21.15.73:0/4051233702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 msgr2=0x7fe28c10ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 --2- 172.21.15.73:0/4051233702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe2880098a0 tx=0x7fe288009bb0 comp rx=0 tx=0).stop 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- 172.21.15.73:0/4051233702 shutdown_connections 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 --2- 172.21.15.73:0/4051233702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c10ac60 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- 172.21.15.73:0/4051233702 >> 172.21.15.73:0/4051233702 conn(0x7fe28c102280 msgr2=0x7fe28c1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- 172.21.15.73:0/4051233702 shutdown_connections 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- 172.21.15.73:0/4051233702 wait complete. 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 Processor -- start 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- start start 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.085+0000 7fe29369e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe28c10c0e0 con 0x7fe28c108870 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29143a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29143a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57058/0 (socket says 172.21.15.73:57058) 2024-06-06T06:49:59.062 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29143a700 1 -- 172.21.15.73:0/3478544192 learned_addr learned my addr 172.21.15.73:0/3478544192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29143a700 1 -- 172.21.15.73:0/3478544192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28c19c0d0 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29143a700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe288011a20 tx=0x7fe288011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288003b70 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe28801a8e0 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe28c19c360 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe288024990 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.086+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe28c19f050 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.087+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe288022020 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.087+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe28c063af0 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.087+0000 7fe2827fc700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 0x7fe27803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.087+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe28804d540 con 0x7fe28c108870 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.087+0000 7fe290c39700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 0x7fe27803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.088+0000 7fe290c39700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 0x7fe27803b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe27c006fd0 tx=0x7fe27c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:49:59.063 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.092+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe28800b3e0 con 0x7fe28c108870 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.295+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fe28c19f220 con 0x7fe28c108870 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.296+0000 7fe2827fc700 1 -- 172.21.15.73:0/3478544192 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fe28802ad90 con 0x7fe28c108870 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 msgr2=0x7fe27803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 0x7fe27803b610 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe27c006fd0 tx=0x7fe27c006e40 comp rx=0 tx=0).stop 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 msgr2=0x7fe28c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe288011a20 tx=0x7fe288011b00 comp rx=0 tx=0).stop 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 shutdown_connections 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe278039150 0x7fe27803b610 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 --2- 172.21.15.73:0/3478544192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe28c108870 0x7fe28c19bb90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 >> 172.21.15.73:0/3478544192 conn(0x7fe28c102280 msgr2=0x7fe28c102ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 shutdown_connections 2024-06-06T06:49:59.064 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:58.298+0000 7fe29369e700 1 -- 172.21.15.73:0/3478544192 wait complete. 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.541+0000 7f6a08773700 1 Processor -- start 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.541+0000 7f6a08773700 1 -- start start 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a08773700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a08773700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0007c650 con 0x7f6a00108800 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0650f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:00.477 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0650f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57070/0 (socket says 172.21.15.73:57070) 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0650f700 1 -- 172.21.15.73:0/443429316 learned_addr learned my addr 172.21.15.73:0/443429316 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0650f700 1 -- 172.21.15.73:0/443429316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a0010b130 con 0x7f6a00108800 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0650f700 1 --2- 172.21.15.73:0/443429316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009e80 tx=0x7f69fc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49019aac6c9e4cd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0550d700 1 -- 172.21.15.73:0/443429316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69fc017070 con 0x7f6a00108800 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0550d700 1 -- 172.21.15.73:0/443429316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69fc00b8c0 con 0x7f6a00108800 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.542+0000 7f6a0550d700 1 -- 172.21.15.73:0/443429316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69fc004490 con 0x7f6a00108800 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 -- 172.21.15.73:0/443429316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 msgr2=0x7f6a0010abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 --2- 172.21.15.73:0/443429316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f69fc009e80 tx=0x7f69fc00b330 comp rx=0 tx=0).stop 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 -- 172.21.15.73:0/443429316 shutdown_connections 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 --2- 172.21.15.73:0/443429316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a0010abf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 -- 172.21.15.73:0/443429316 >> 172.21.15.73:0/443429316 conn(0x7f6a00102250 msgr2=0x7f6a00104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:00.478 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 -- 172.21.15.73:0/443429316 shutdown_connections 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.543+0000 7f6a08773700 1 -- 172.21.15.73:0/443429316 wait complete. 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a08773700 1 Processor -- start 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a08773700 1 -- start start 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a08773700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a08773700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a0010c070 con 0x7f6a00108800 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a0650f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a0650f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57076/0 (socket says 172.21.15.73:57076) 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a0650f700 1 -- 172.21.15.73:0/3454074329 learned_addr learned my addr 172.21.15.73:0/3454074329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:00.479 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a0650f700 1 -- 172.21.15.73:0/3454074329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a001085c0 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f6a0650f700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0072a0 tx=0x7f69fc003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.544+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69fc020020 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69fc00bba0 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a001067b0 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69fc017040 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a00106cf0 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f69fc023020 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.545+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a00063af0 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.546+0000 7f69f77fe700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 0x7f69ec03b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.546+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f69fc00f050 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.546+0000 7f6a05d0e700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 0x7f69ec03b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.546+0000 7f6a05d0e700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 0x7f69ec03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.551+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f69fc02a020 con 0x7f6a00108800 2024-06-06T06:50:00.480 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.857+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f6a00107030 con 0x7f6a00108800 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.860+0000 7f69f77fe700 1 -- 172.21.15.73:0/3454074329 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f6a00107030 con 0x7f6a00108800 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 msgr2=0x7f69ec03b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 0x7f69ec03b610 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 msgr2=0x7f6a00108080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0072a0 tx=0x7f69fc003d00 comp rx=0 tx=0).stop 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 shutdown_connections 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f69ec039150 0x7f69ec03b610 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 --2- 172.21.15.73:0/3454074329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a00108800 0x7f6a00108080 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 >> 172.21.15.73:0/3454074329 conn(0x7f6a00102250 msgr2=0x7f6a00102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 shutdown_connections 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr 2024-06-06T06:49:59.862+0000 7f6a08773700 1 -- 172.21.15.73:0/3454074329 wait complete. 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-06T06:50:00.481 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout:Or, if you are only running a single cluster on this host: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: ceph telemetry on 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout:For more information see: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.482 INFO:teuthology.orchestra.run.smithi073.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-06T06:50:00.483 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:00.483 INFO:teuthology.orchestra.run.smithi073.stdout:Bootstrap complete. 2024-06-06T06:50:00.517 INFO:tasks.cephadm:Fetching config... 2024-06-06T06:50:00.518 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:00.518 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-06T06:50:00.534 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-06T06:50:00.534 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:00.535 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-06T06:50:00.590 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-06T06:50:00.591 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:00.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-06T06:50:00.655 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-06T06:50:00.655 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:00.655 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-06T06:50:00.713 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-06T06:50:00.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T06:50:00.794 INFO:teuthology.orchestra.run.smithi073.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:00.806 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T06:50:00.850 INFO:teuthology.orchestra.run.smithi136.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:00.865 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-06T06:50:00.922 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC1SRqo+avVoB0+F7l8zw+mZOhgva/RHnhCSdw0FjzZkCehbsRfUZHb74NEwO0jsH3Qh0l0cwBCm2+YU8BRGzxBDZcTQj17+Vw1ViZYOcXKPJjJjibY31+kYKms5sM14EQS18FdFvHKC8f1UOkN433rYQWsbvhmNH9FigRv2NOrblabc6Uiwi3vvg5rCcQwj25Phf4FBFqz3zmaeXTLxe//j5t8XN3qH/+nKlIf5YuMKi78S4P4C4ropNSeHYuzA6wakS3X4bhQTDplaVGYjyD83JU9DbLHmAf7z6K1hSKUCIouWIitxCyC2MMALFQDVnXGlw5rhPp4yweQmuBS2PnngbaHZX0/ibmFbBZANR45RUE++0StZPGNg4TZanB3yW6GJDaKlLQgaPRNpDu0Fta+EE/KcVmNPiaMiFLh8E+kXg4PV+vCGIO5Tszw5/2leSp4Z1LTGgjKz8Js1/JzLcIkuRdVTFHhtoeD7cS6ZiAFf7erBWxZ6DEadkaKQZ7+rlE= ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:00.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-06T06:50:01.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:00 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3454074329' entity='client.admin' 2024-06-06T06:50:01.173 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 -- 172.21.15.73:0/3309977929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 msgr2=0x7fa22c074920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 --2- 172.21.15.73:0/3309977929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c074920 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa228009910 tx=0x7fa228009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 -- 172.21.15.73:0/3309977929 shutdown_connections 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 --2- 172.21.15.73:0/3309977929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c074920 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 -- 172.21.15.73:0/3309977929 >> 172.21.15.73:0/3309977929 conn(0x7fa22c0fd4e0 msgr2=0x7fa22c0ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 -- 172.21.15.73:0/3309977929 shutdown_connections 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.836+0000 7fa233ffc700 1 -- 172.21.15.73:0/3309977929 wait complete. 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa233ffc700 1 Processor -- start 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa233ffc700 1 -- start start 2024-06-06T06:50:03.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa233ffc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa233ffc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa22c10a060 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa231d98700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa231d98700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57098/0 (socket says 172.21.15.73:57098) 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.837+0000 7fa231d98700 1 -- 172.21.15.73:0/1026652939 learned_addr learned my addr 172.21.15.73:0/1026652939 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa231d98700 1 -- 172.21.15.73:0/1026652939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa22c199cd0 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa231d98700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa228005800 tx=0x7fa228009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa228027070 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa22c199f60 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa22800be10 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa22800f460 con 0x7fa22c0764c0 2024-06-06T06:50:03.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.838+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa22c19cc50 con 0x7fa22c0764c0 2024-06-06T06:50:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.839+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa22802a070 con 0x7fa22c0764c0 2024-06-06T06:50:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.839+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa22c10a7e0 con 0x7fa22c0764c0 2024-06-06T06:50:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.839+0000 7fa222ffd700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 0x7fa21803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.839+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa22801e030 con 0x7fa22c0764c0 2024-06-06T06:50:03.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.839+0000 7fa231597700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 0x7fa21803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:03.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.840+0000 7fa231597700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 0x7fa21803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006fd0 tx=0x7fa21c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:03.845 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:03.844+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa228013070 con 0x7fa22c0764c0 2024-06-06T06:50:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.059+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa22c02ddb0 con 0x7fa22c0764c0 2024-06-06T06:50:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.061+0000 7fa222ffd700 1 -- 172.21.15.73:0/1026652939 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa228031070 con 0x7fa22c0764c0 2024-06-06T06:50:04.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 msgr2=0x7fa21803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:04.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 0x7fa21803b610 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa21c006fd0 tx=0x7fa21c006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:04.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 msgr2=0x7fa22c199790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa228005800 tx=0x7fa228009f60 comp rx=0 tx=0).stop 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 shutdown_connections 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa218039150 0x7fa21803b610 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 --2- 172.21.15.73:0/1026652939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa22c0764c0 0x7fa22c199790 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.063+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 >> 172.21.15.73:0/1026652939 conn(0x7fa22c0fd4e0 msgr2=0x7fa22c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.064+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 shutdown_connections 2024-06-06T06:50:04.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:04.064+0000 7fa233ffc700 1 -- 172.21.15.73:0/1026652939 wait complete. 2024-06-06T06:50:05.198 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1026652939' entity='client.admin' 2024-06-06T06:50:05.198 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:05.198 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:05.198 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:50:05.198 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:05.199 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:05.199 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:05.225 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-06T06:50:05.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-06T06:50:05.458 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:06.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 -- 172.21.15.73:0/4073404886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 msgr2=0x7fc33c101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:06.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 --2- 172.21.15.73:0/4073404886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c101260 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc338009910 tx=0x7fc338009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:06.857 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 -- 172.21.15.73:0/4073404886 shutdown_connections 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 --2- 172.21.15.73:0/4073404886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c101260 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 -- 172.21.15.73:0/4073404886 >> 172.21.15.73:0/4073404886 conn(0x7fc33c0fc3a0 msgr2=0x7fc33c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 -- 172.21.15.73:0/4073404886 shutdown_connections 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.856+0000 7fc343ffa700 1 -- 172.21.15.73:0/4073404886 wait complete. 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc343ffa700 1 Processor -- start 2024-06-06T06:50:06.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc343ffa700 1 -- start start 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc343ffa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc343ffa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc33c106d00 con 0x7fc33c100e40 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc341d96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc341d96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48194/0 (socket says 172.21.15.73:48194) 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc341d96700 1 -- 172.21.15.73:0/535987715 learned_addr learned my addr 172.21.15.73:0/535987715 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc341d96700 1 -- 172.21.15.73:0/535987715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc33c19a280 con 0x7fc33c100e40 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc341d96700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc338005b40 tx=0x7fc338004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.857+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc338027070 con 0x7fc33c100e40 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.858+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc33800be10 con 0x7fc33c100e40 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.858+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc33c19a450 con 0x7fc33c100e40 2024-06-06T06:50:06.860 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.858+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc338022de0 con 0x7fc33c100e40 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.858+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc33c19a7f0 con 0x7fc33c100e40 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.858+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fc33802a070 con 0x7fc33c100e40 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.859+0000 7fc332ffd700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 0x7fc32803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.859+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc33801e030 con 0x7fc33c100e40 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.859+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc33c107480 con 0x7fc33c100e40 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.859+0000 7fc341595700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 0x7fc32803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:06.861 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.859+0000 7fc341595700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 0x7fc32803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc32c006fd0 tx=0x7fc32c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:06.864 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:06.864+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc338013070 con 0x7fc33c100e40 2024-06-06T06:50:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.062+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7fc33c02dea0 con 0x7fc3280390f0 2024-06-06T06:50:07.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.065+0000 7fc332ffd700 1 -- 172.21.15.73:0/535987715 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fc33c02dea0 con 0x7fc3280390f0 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.067+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 msgr2=0x7fc32803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.067+0000 7fc343ffa700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 0x7fc32803b5b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc32c006fd0 tx=0x7fc32c006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 msgr2=0x7fc33c06ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc338005b40 tx=0x7fc338004f80 comp rx=0 tx=0).stop 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 shutdown_connections 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc3280390f0 0x7fc32803b5b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 --2- 172.21.15.73:0/535987715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc33c100e40 0x7fc33c06ad80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 >> 172.21.15.73:0/535987715 conn(0x7fc33c0fc3a0 msgr2=0x7fc33c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 shutdown_connections 2024-06-06T06:50:07.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:07.068+0000 7fc343ffa700 1 -- 172.21.15.73:0/535987715 wait complete. 2024-06-06T06:50:07.818 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi136 2024-06-06T06:50:07.819 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:50:07.819 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.conf 2024-06-06T06:50:07.838 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:50:07.838 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:07.897 INFO:tasks.cephadm:Adding host smithi136 to orchestrator... 2024-06-06T06:50:07.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch host add smithi136 2024-06-06T06:50:08.137 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T06:50:08.369 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:08.370 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:08.370 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:08.370 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:09.330 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:09 smithi073 ceph-mon[32303]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:09.330 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:09 smithi073 ceph-mon[32303]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T06:50:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4113245439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 msgr2=0x7fe568101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4113245439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe568101420 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe558009910 tx=0x7fe558009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4113245439 shutdown_connections 2024-06-06T06:50:09.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4113245439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe568101420 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4113245439 >> 172.21.15.73:0/4113245439 conn(0x7fe5680768f0 msgr2=0x7fe568076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4113245439 shutdown_connections 2024-06-06T06:50:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.619+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4113245439 wait complete. 2024-06-06T06:50:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe56dd8d700 1 Processor -- start 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe56dd8d700 1 -- start start 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe56dd8d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe56dd8d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe56810b270 con 0x7fe568101040 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe5677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48210/0 (socket says 172.21.15.73:48210) 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe5677fe700 1 -- 172.21.15.73:0/4013144551 learned_addr learned my addr 172.21.15.73:0/4013144551 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.620+0000 7fe5677fe700 1 -- 172.21.15.73:0/4013144551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56819fcb0 con 0x7fe568101040 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe5677fe700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5580056d0 tx=0x7fe5580057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe558026070 con 0x7fe568101040 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe55800bba0 con 0x7fe568101040 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56819ffa0 con 0x7fe568101040 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5580037f0 con 0x7fe568101040 2024-06-06T06:50:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.621+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe568199930 con 0x7fe568101040 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe558029070 con 0x7fe568101040 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5681024d0 con 0x7fe568101040 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe564ff9700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 0x7fe55003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe55801e070 con 0x7fe568101040 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe566ffd700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 0x7fe55003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.622+0000 7fe566ffd700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 0x7fe55003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006fd0 tx=0x7fe55c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.627+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe558013070 con 0x7fe568101040 2024-06-06T06:50:09.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:09.826+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}) -- 0x7fe56819ae60 con 0x7fe5500390f0 2024-06-06T06:50:10.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:10 smithi073 ceph-mon[32303]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi136", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:12 smithi073 ceph-mon[32303]: Deploying cephadm binary to smithi136 2024-06-06T06:50:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:12 smithi073 ceph-mon[32303]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:14.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.028+0000 7fe564ff9700 1 -- 172.21.15.73:0/4013144551 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fe56819ae60 con 0x7fe5500390f0 2024-06-06T06:50:14.029 INFO:teuthology.orchestra.run.smithi073.stdout:Added host 'smithi136' with addr '172.21.15.136' 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 msgr2=0x7fe55003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 0x7fe55003b5b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006fd0 tx=0x7fe55c006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 msgr2=0x7fe56819f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe5580056d0 tx=0x7fe5580057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 shutdown_connections 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5500390f0 0x7fe55003b5b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 --2- 172.21.15.73:0/4013144551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe568101040 0x7fe56819f770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 >> 172.21.15.73:0/4013144551 conn(0x7fe5680768f0 msgr2=0x7fe5680698b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 shutdown_connections 2024-06-06T06:50:14.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:14.030+0000 7fe56dd8d700 1 -- 172.21.15.73:0/4013144551 wait complete. 2024-06-06T06:50:14.344 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:14 smithi073 ceph-mon[32303]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:14.344 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:14.344 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:14.609 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T06:50:14.846 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:15 smithi073 ceph-mon[32303]: Added host smithi136 2024-06-06T06:50:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:16.312 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:16 smithi073 ceph-mon[32303]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 -- 172.21.15.73:0/265914419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 msgr2=0x7f8a181013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/265914419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181013f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c009910 tx=0x7f8a0c009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 -- 172.21.15.73:0/265914419 shutdown_connections 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/265914419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181013f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 -- 172.21.15.73:0/265914419 >> 172.21.15.73:0/265914419 conn(0x7f8a180fc8e0 msgr2=0x7f8a180fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 -- 172.21.15.73:0/265914419 shutdown_connections 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.417+0000 7f8a1ef79700 1 -- 172.21.15.73:0/265914419 wait complete. 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1ef79700 1 Processor -- start 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1ef79700 1 -- start start 2024-06-06T06:50:16.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1ef79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:16.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1ef79700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1810b290 con 0x7f8a18101010 2024-06-06T06:50:16.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1cd15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:16.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1cd15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45114/0 (socket says 172.21.15.73:45114) 2024-06-06T06:50:16.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1cd15700 1 -- 172.21.15.73:0/3674705588 learned_addr learned my addr 172.21.15.73:0/3674705588 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.418+0000 7f8a1cd15700 1 -- 172.21.15.73:0/3674705588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a18198c20 con 0x7f8a18101010 2024-06-06T06:50:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a1cd15700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0056d0 tx=0x7f8a0c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c026070 con 0x7f8a18101010 2024-06-06T06:50:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a0c00bba0 con 0x7f8a18101010 2024-06-06T06:50:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a18198f10 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8a0c0037f0 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.419+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a181993d0 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.420+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8a0c029070 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.420+0000 7f8a15ffb700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 0x7f8a0803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.420+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8a0c01e070 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.420+0000 7f8a17fff700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 0x7f8a0803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.420+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a181024f0 con 0x7f8a18101010 2024-06-06T06:50:16.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.421+0000 7f8a17fff700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 0x7f8a0803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:16.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.425+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a0c030070 con 0x7f8a18101010 2024-06-06T06:50:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.660+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8a18063af0 con 0x7f8a080390f0 2024-06-06T06:50:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.661+0000 7f8a15ffb700 1 -- 172.21.15.73:0/3674705588 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f8a18063af0 con 0x7f8a080390f0 2024-06-06T06:50:16.662 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:16.662 INFO:teuthology.orchestra.run.smithi073.stdout:[{"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}] 2024-06-06T06:50:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.663+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 msgr2=0x7f8a0803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.663+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 0x7f8a0803b5b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.663+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 msgr2=0x7f8a181986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.663+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0056d0 tx=0x7f8a0c0057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 shutdown_connections 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a080390f0 0x7f8a0803b5b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 --2- 172.21.15.73:0/3674705588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a18101010 0x7f8a181986e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 >> 172.21.15.73:0/3674705588 conn(0x7f8a180fc8e0 msgr2=0x7f8a181029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 shutdown_connections 2024-06-06T06:50:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:16.664+0000 7f8a1ef79700 1 -- 172.21.15.73:0/3674705588 wait complete. 2024-06-06T06:50:17.251 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-06-06T06:50:17.251 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:50:17.251 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-06-06T06:50:17.270 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:50:17.271 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:17.329 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-06-06T06:50:17.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch host add smithi170 2024-06-06T06:50:17.570 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:17.934 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:17 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:17.934 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:17 smithi073 ceph-mon[32303]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:50:17.934 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:17 smithi073 ceph-mon[32303]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:19.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 -- 172.21.15.73:0/752867562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 msgr2=0x7f389c101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 --2- 172.21.15.73:0/752867562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c101420 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f388c009910 tx=0x7f388c009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 -- 172.21.15.73:0/752867562 shutdown_connections 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 --2- 172.21.15.73:0/752867562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c101420 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 -- 172.21.15.73:0/752867562 >> 172.21.15.73:0/752867562 conn(0x7f389c0768f0 msgr2=0x7f389c076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 -- 172.21.15.73:0/752867562 shutdown_connections 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.176+0000 7f38a13fb700 1 -- 172.21.15.73:0/752867562 wait complete. 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f38a13fb700 1 Processor -- start 2024-06-06T06:50:19.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f38a13fb700 1 -- start start 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f38a13fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f38a13fb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f389c10b270 con 0x7f389c101040 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f389affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f389affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45136/0 (socket says 172.21.15.73:45136) 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.177+0000 7f389affd700 1 -- 172.21.15.73:0/2756371338 learned_addr learned my addr 172.21.15.73:0/2756371338 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:19.180 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f389affd700 1 -- 172.21.15.73:0/2756371338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f389c198c60 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f389affd700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f388c0056d0 tx=0x7f388c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c026070 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f388c00bba0 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f389c198f50 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f388c0037f0 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.178+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f389c199410 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.179+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f389c1024d0 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.179+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f388c029070 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.179+0000 7f3883fff700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 0x7f388403b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.179+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f388c01e070 con 0x7f389c101040 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.179+0000 7f389a7fc700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 0x7f388403b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:19.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.180+0000 7f389a7fc700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 0x7f388403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3890006fd0 tx=0x7f3890006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:19.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.184+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f388c013070 con 0x7f389c101040 2024-06-06T06:50:19.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:19.384+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}) -- 0x7f389c069ec0 con 0x7f3884039150 2024-06-06T06:50:20.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:20.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:20.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:21 smithi073 ceph-mon[32303]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:21 smithi073 ceph-mon[32303]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T06:50:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:21 smithi073 ceph-mon[32303]: Deploying cephadm binary to smithi170 2024-06-06T06:50:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:21 smithi073 ceph-mon[32303]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:22 smithi073 ceph-mon[32303]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:22 smithi073 ceph-mon[32303]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:23.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.538+0000 7f3883fff700 1 -- 172.21.15.73:0/2756371338 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f389c069ec0 con 0x7f3884039150 2024-06-06T06:50:23.539 INFO:teuthology.orchestra.run.smithi073.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-06-06T06:50:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 msgr2=0x7f388403b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 0x7f388403b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3890006fd0 tx=0x7f3890006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 msgr2=0x7f389c198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:23.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f388c0056d0 tx=0x7f388c0057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 shutdown_connections 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3884039150 0x7f388403b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 --2- 172.21.15.73:0/2756371338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f389c101040 0x7f389c198720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 >> 172.21.15.73:0/2756371338 conn(0x7f389c0768f0 msgr2=0x7f389c1029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 shutdown_connections 2024-06-06T06:50:23.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:23.540+0000 7f38a13fb700 1 -- 172.21.15.73:0/2756371338 wait complete. 2024-06-06T06:50:24.243 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:24.244 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:24.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-06T06:50:24.511 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:25.265 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:25 smithi073 ceph-mon[32303]: Added host smithi170 2024-06-06T06:50:25.265 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:25 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:26.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 -- 172.21.15.73:0/4279521756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 msgr2=0x7f30341013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:26.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 --2- 172.21.15.73:0/4279521756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341013a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f302400b0a0 tx=0x7f302400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:50:26.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 -- 172.21.15.73:0/4279521756 shutdown_connections 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 --2- 172.21.15.73:0/4279521756 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341013a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 -- 172.21.15.73:0/4279521756 >> 172.21.15.73:0/4279521756 conn(0x7f30340fc850 msgr2=0x7f30340fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 -- 172.21.15.73:0/4279521756 shutdown_connections 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 -- 172.21.15.73:0/4279521756 wait complete. 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.020+0000 7f3039e4b700 1 Processor -- start 2024-06-06T06:50:26.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3039e4b700 1 -- start start 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3039e4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3039e4b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f303410b240 con 0x7f3034100fc0 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f30337fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f30337fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34506/0 (socket says 172.21.15.73:34506) 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f30337fe700 1 -- 172.21.15.73:0/2086013709 learned_addr learned my addr 172.21.15.73:0/2086013709 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f30337fe700 1 -- 172.21.15.73:0/2086013709 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3034198c00 con 0x7f3034100fc0 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f30337fe700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f302400d010 tx=0x7f3024009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3024017040 con 0x7f3034100fc0 2024-06-06T06:50:26.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3034198e90 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3024022070 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.021+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f302401d5e0 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.022+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3034199350 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.022+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f302401d8e0 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.022+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30341024a0 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.023+0000 7f3030ff9700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 0x7f301c03b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.023+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f302400f050 con 0x7f3034100fc0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.023+0000 7f3032ffd700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 0x7f301c03b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:26.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.023+0000 7f3032ffd700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 0x7f301c03b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3028006fd0 tx=0x7f3028006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:26.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.028+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3024025c20 con 0x7f3034100fc0 2024-06-06T06:50:26.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.230+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3034063af0 con 0x7f301c039140 2024-06-06T06:50:26.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.231+0000 7f3030ff9700 1 -- 172.21.15.73:0/2086013709 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f3034063af0 con 0x7f301c039140 2024-06-06T06:50:26.232 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:50:26.232 INFO:teuthology.orchestra.run.smithi073.stdout:[{"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.136", "hostname": "smithi136", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-06-06T06:50:26.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 msgr2=0x7f301c03b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 0x7f301c03b600 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f3028006fd0 tx=0x7f3028006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 msgr2=0x7f30341986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f302400d010 tx=0x7f3024009790 comp rx=0 tx=0).stop 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 shutdown_connections 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f301c039140 0x7f301c03b600 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 --2- 172.21.15.73:0/2086013709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3034100fc0 0x7f30341986c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.233+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 >> 172.21.15.73:0/2086013709 conn(0x7f30340fc850 msgr2=0x7f3034102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.234+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 shutdown_connections 2024-06-06T06:50:26.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:26.234+0000 7f3039e4b700 1 -- 172.21.15.73:0/2086013709 wait complete. 2024-06-06T06:50:26.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:26 smithi073 ceph-mon[32303]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:26.816 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-06T06:50:26.817 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-06T06:50:27.056 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:27.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:27.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:27 smithi073 ceph-mon[32303]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:50:28.402 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:28 smithi073 ceph-mon[32303]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 -- 172.21.15.73:0/3897238774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 msgr2=0x7f1368108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 --2- 172.21.15.73:0/3897238774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f1368108400 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1350009910 tx=0x7f1350009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 -- 172.21.15.73:0/3897238774 shutdown_connections 2024-06-06T06:50:28.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 --2- 172.21.15.73:0/3897238774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f1368108400 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:28.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 -- 172.21.15.73:0/3897238774 >> 172.21.15.73:0/3897238774 conn(0x7f1368077510 msgr2=0x7f1368077920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:28.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 -- 172.21.15.73:0/3897238774 shutdown_connections 2024-06-06T06:50:28.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.611+0000 7f136d0e9700 1 -- 172.21.15.73:0/3897238774 wait complete. 2024-06-06T06:50:28.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.612+0000 7f136d0e9700 1 Processor -- start 2024-06-06T06:50:28.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.612+0000 7f136d0e9700 1 -- start start 2024-06-06T06:50:28.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.612+0000 7f136d0e9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.612+0000 7f136d0e9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f136810c4d0 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f1366d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34518/0 (socket says 172.21.15.73:34518) 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f1366d9d700 1 -- 172.21.15.73:0/2646623955 learned_addr learned my addr 172.21.15.73:0/2646623955 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f1366d9d700 1 -- 172.21.15.73:0/2646623955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f136819db80 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f1366d9d700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13500056d0 tx=0x7f13500057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1350026070 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f136819de10 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f135000bc00 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13500039b0 con 0x7f1368108020 2024-06-06T06:50:28.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.613+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13681977c0 con 0x7f1368108020 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.614+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f1350029070 con 0x7f1368108020 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.614+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1368103740 con 0x7f1368108020 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.614+0000 7f135ffff700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 0x7f135403b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.614+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f135001e070 con 0x7f1368108020 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.614+0000 7f136659c700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 0x7f135403b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:28.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.615+0000 7f136659c700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 0x7f135403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1358006fd0 tx=0x7f1358006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:28.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.619+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1350013070 con 0x7f1368108020 2024-06-06T06:50:28.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:28.819+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1368063af0 con 0x7f1368108020 2024-06-06T06:50:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.120+0000 7f135ffff700 1 -- 172.21.15.73:0/2646623955 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1350030020 con 0x7f1368108020 2024-06-06T06:50:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 msgr2=0x7f135403b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 0x7f135403b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1358006fd0 tx=0x7f1358006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 msgr2=0x7f136819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f13500056d0 tx=0x7f13500057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 shutdown_connections 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13540390f0 0x7f135403b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 --2- 172.21.15.73:0/2646623955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1368108020 0x7f136819d640 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.122+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 >> 172.21.15.73:0/2646623955 conn(0x7f1368077510 msgr2=0x7f1368103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.123+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 shutdown_connections 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:29.123+0000 7f136d0e9700 1 -- 172.21.15.73:0/2646623955 wait complete. 2024-06-06T06:50:29.124 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-06-06T06:50:29.410 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:29 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2646623955' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T06:50:29.737 INFO:tasks.cephadm:Adding mon.a on smithi073 2024-06-06T06:50:29.737 INFO:tasks.cephadm:Adding mon.b on smithi136 2024-06-06T06:50:29.737 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-06-06T06:50:29.738 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c' 2024-06-06T06:50:29.972 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2646623955' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T06:50:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:30.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:30.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:31.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/1080408092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 msgr2=0x7f8c341024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:31.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/1080408092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c341024e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8c30009910 tx=0x7f8c30009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:31.268 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/1080408092 shutdown_connections 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/1080408092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c341024e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/1080408092 >> 172.21.15.170:0/1080408092 conn(0x7f8c340fd9d0 msgr2=0x7f8c340ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/1080408092 shutdown_connections 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.266+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/1080408092 wait complete. 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c3b3d6700 1 Processor -- start 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c3b3d6700 1 -- start start 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c3b3d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c3b3d6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8c3410c380 con 0x7f8c34102100 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c39172700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c39172700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.170:46782/0 (socket says 172.21.15.170:46782) 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.267+0000 7f8c39172700 1 -- 172.21.15.170:0/2032744591 learned_addr learned my addr 172.21.15.170:0/2032744591 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:50:31.269 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c39172700 1 -- 172.21.15.170:0/2032744591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c34076680 con 0x7f8c34102100 2024-06-06T06:50:31.270 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c39172700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c300056d0 tx=0x7f8c300057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c30026070 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c34074d30 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c3000bc00 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c300039b0 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.268+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c340751f0 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.269+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8c30029070 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.269+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c341035e0 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.269+0000 7f8c2a7fc700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 0x7f8c2003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.269+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8c3001e070 con 0x7f8c34102100 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.269+0000 7f8c38971700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 0x7f8c2003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:31.271 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.270+0000 7f8c38971700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 0x7f8c2003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006fd0 tx=0x7f8c24006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:31.275 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.274+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c30013070 con 0x7f8c34102100 2024-06-06T06:50:31.481 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.480+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}) -- 0x7f8c34075d30 con 0x7f8c200390f0 2024-06-06T06:50:31.488 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.488+0000 7f8c2a7fc700 1 -- 172.21.15.170:0/2032744591 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8c34075d30 con 0x7f8c200390f0 2024-06-06T06:50:31.488 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-06-06T06:50:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 msgr2=0x7f8c2003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 0x7f8c2003b5b0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8c24006fd0 tx=0x7f8c24006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 msgr2=0x7f8c34074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:31.490 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8c300056d0 tx=0x7f8c300057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 shutdown_connections 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c200390f0 0x7f8c2003b5b0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 --2- 172.21.15.170:0/2032744591 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c34102100 0x7f8c34074790 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 >> 172.21.15.170:0/2032744591 conn(0x7f8c340fd9d0 msgr2=0x7f8c34103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 shutdown_connections 2024-06-06T06:50:31.491 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:31.489+0000 7f8c3b3d6700 1 -- 172.21.15.170:0/2032744591 wait complete. 2024-06-06T06:50:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:31 smithi073 ceph-mon[32303]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:31 smithi073 ceph-mon[32303]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:31 smithi073 ceph-mon[32303]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T06:50:32.053 DEBUG:teuthology.orchestra.run.smithi136:mon.b> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.b.service 2024-06-06T06:50:32.056 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.c.service 2024-06-06T06:50:32.058 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T06:50:32.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T06:50:32.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:32.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:33 smithi073 ceph-mon[32303]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:33 smithi073 ceph-mon[32303]: Saving service mon spec with placement smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c;count:3 2024-06-06T06:50:33.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:33 smithi073 ceph-mon[32303]: Deploying daemon mon.c on smithi170 2024-06-06T06:50:33.597 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:34.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:34 smithi073 ceph-mon[32303]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- 172.21.15.170:0/1420938035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 msgr2=0x7fb3dc101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/1420938035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc101420 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4009910 tx=0x7fb3c4009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- 172.21.15.170:0/1420938035 shutdown_connections 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/1420938035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc101420 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- 172.21.15.170:0/1420938035 >> 172.21.15.170:0/1420938035 conn(0x7fb3dc0fc8f0 msgr2=0x7fb3dc0fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- 172.21.15.170:0/1420938035 shutdown_connections 2024-06-06T06:50:35.804 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- 172.21.15.170:0/1420938035 wait complete. 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 Processor -- start 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.802+0000 7fb3e29b0700 1 -- start start 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3e29b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3e29b0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3dc10b2c0 con 0x7fb3dc101040 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.170:52732/0 (socket says 172.21.15.170:52732) 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3dbfff700 1 -- 172.21.15.170:0/904675952 learned_addr learned my addr 172.21.15.170:0/904675952 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:50:35.805 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.803+0000 7fb3dbfff700 1 -- 172.21.15.170:0/904675952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3dc198c70 con 0x7fb3dc101040 2024-06-06T06:50:35.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3dbfff700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40056d0 tx=0x7fb3c40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:35.806 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3c4026070 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3dc198f60 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3dc199420 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3c400bba0 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.804+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb3c40037f0 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.805+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3dc102520 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.805+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fb3c4029070 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.805+0000 7fb3d97fa700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 0x7fb3c803b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.805+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3c401e070 con 0x7fb3dc101040 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.805+0000 7fb3db7fe700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 0x7fb3c803b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:35.807 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.806+0000 7fb3db7fe700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 0x7fb3c803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006fd0 tx=0x7fb3cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:35.811 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:35.810+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3c4013070 con 0x7fb3dc101040 2024-06-06T06:50:36.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.104+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fb3dc06a950 con 0x7fb3dc101040 2024-06-06T06:50:36.105 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.104+0000 7fb3d97fa700 1 -- 172.21.15.170:0/904675952 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fb3c4030020 con 0x7fb3dc101040 2024-06-06T06:50:36.105 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:50:36.106 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","modified":"2024-06-06T06:48:44.267203Z","created":"2024-06-06T06:48:44.267203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-06-06T06:50:36.107 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 msgr2=0x7fb3c803b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 0x7fb3c803b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006fd0 tx=0x7fb3cc006e40 comp rx=0 tx=0).stop 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 msgr2=0x7fb3dc198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb3c40056d0 tx=0x7fb3c40057b0 comp rx=0 tx=0).stop 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 shutdown_connections 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3c80390f0 0x7fb3c803b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 --2- 172.21.15.170:0/904675952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3dc101040 0x7fb3dc198730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 >> 172.21.15.170:0/904675952 conn(0x7fb3dc0fc8f0 msgr2=0x7fb3dc102a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 shutdown_connections 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:36.107+0000 7fb3e29b0700 1 -- 172.21.15.170:0/904675952 wait complete. 2024-06-06T06:50:36.108 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-06-06T06:50:36.203 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(???) e0 preinit fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:36.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 bash[36077]: 6dabbe5f80af62b3a3bb0bc1e6d10981a57e93cd1bb262dc4d66fd8464c13e86 2024-06-06T06:50:36.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 systemd[1]: Started Ceph mon.c for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:50:36.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).mds e1 new map 2024-06-06T06:50:36.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).mds e1 print_map 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-06-06T06:50:36.483 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi136", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:50:36.484 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Deploying cephadm binary to smithi170 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.485 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Added host smithi170 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2646623955' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2646623955' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.486 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.487 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Saving service mon spec with placement smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c;count:3 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: Deploying daemon mon.c on smithi170 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:36.488 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:36 smithi170 ceph-mon[36108]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T06:50:37.760 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T06:50:37.760 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T06:50:38.004 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:50:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:50:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: mon.a calling monitor election 2024-06-06T06:50:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:41.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: mon.c calling monitor election 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: monmap epoch 2 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: last_changed 2024-06-06T06:50:36.216343+0000 2024-06-06T06:50:41.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: min_mon_release 19 (squid) 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: election_strategy: 1 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: fsmap 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: mgrmap e13: x(active, since 50s) 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: overall HEALTH_OK 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:41.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: mon.a calling monitor election 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: mon.c calling monitor election 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: monmap epoch 2 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: last_changed 2024-06-06T06:50:36.216343+0000 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: min_mon_release 19 (squid) 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: election_strategy: 1 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:41.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: fsmap 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: mgrmap e13: x(active, since 50s) 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: overall HEALTH_OK 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:41.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:42.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 -- 172.21.15.170:0/888736658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 msgr2=0x7fa2b4005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:42.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 --2- 172.21.15.170:0/888736658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2b4005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc100dc0 tx=0x7fa2bc00feb0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.409 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 -- 172.21.15.170:0/888736658 shutdown_connections 2024-06-06T06:50:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 --2- 172.21.15.170:0/888736658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2b4005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 -- 172.21.15.170:0/888736658 >> 172.21.15.170:0/888736658 conn(0x7fa2cc0fc9c0 msgr2=0x7fa2cc0fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 -- 172.21.15.170:0/888736658 shutdown_connections 2024-06-06T06:50:42.410 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.408+0000 7fa2d148f700 1 -- 172.21.15.170:0/888736658 wait complete. 2024-06-06T06:50:42.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 Processor -- start 2024-06-06T06:50:42.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 -- start start 2024-06-06T06:50:42.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2cc19c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:42.413 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc108dd0 con 0x7fa2b40035c0 2024-06-06T06:50:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.409+0000 7fa2d148f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2cc108c50 con 0x7fa2cc19cbc0 2024-06-06T06:50:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:43726/0 (socket says 172.21.15.170:43726) 2024-06-06T06:50:42.414 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 -- 172.21.15.170:0/2799065428 learned_addr learned my addr 172.21.15.170:0/2799065428 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2caffd700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2cc19c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 -- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 msgr2=0x7fa2cc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 -- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 msgr2=0x7fa2cc196750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2ca7fc700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2caffd700 1 -- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 msgr2=0x7fa2cc196750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2caffd700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2caffd700 1 -- 172.21.15.170:0/2799065428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2cc19d140 con 0x7fa2b40035c0 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.410+0000 7fa2caffd700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2cc19c680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc0050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.411+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa2bc004aa0 con 0x7fa2b40035c0 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.411+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2cc196cf0 con 0x7fa2b40035c0 2024-06-06T06:50:42.415 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.411+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2bc004030 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.411+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fa2bc011bb0 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.411+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2cc197280 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.412+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa2bc023070 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.412+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2cc1025d0 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.412+0000 7fa2b3fff700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 0x7fa2b803b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.412+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fa2bc020050 con 0x7fa2b40035c0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.412+0000 7fa2ca7fc700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 0x7fa2b803b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:42.416 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.413+0000 7fa2ca7fc700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 0x7fa2b803b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006b20 tx=0x7fa2c0006ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:42.419 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.418+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa2bc028020 con 0x7fa2b40035c0 2024-06-06T06:50:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:42 smithi170 ceph-mon[36108]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:42 smithi170 ceph-mon[36108]: Deploying daemon mon.b on smithi136 2024-06-06T06:50:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:42.697 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.696+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fa2cc197590 con 0x7fa2b40035c0 2024-06-06T06:50:42.698 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.697+0000 7fa2b3fff700 1 -- 172.21.15.170:0/2799065428 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fa2bc01a390 con 0x7fa2b40035c0 2024-06-06T06:50:42.698 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:50:42.698 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","modified":"2024-06-06T06:50:36.216343Z","created":"2024-06-06T06:48:44.267203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T06:50:42.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 msgr2=0x7fa2b803b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:42.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 0x7fa2b803b3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa2c0006b20 tx=0x7fa2c0006ab0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 msgr2=0x7fa2cc19c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:42.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2cc19c680 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000c00 tx=0x7fa2bc0050d0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.700 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 shutdown_connections 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2b40035c0 0x7fa2cc19c680 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2b8038ee0 0x7fa2b803b3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 --2- 172.21.15.170:0/2799065428 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2cc19cbc0 0x7fa2cc196750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 >> 172.21.15.170:0/2799065428 conn(0x7fa2cc0fc9c0 msgr2=0x7fa2cc0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 shutdown_connections 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:42.699+0000 7fa2d148f700 1 -- 172.21.15.170:0/2799065428 wait complete. 2024-06-06T06:50:42.701 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-06-06T06:50:42.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:42 smithi073 ceph-mon[32303]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:42.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:42 smithi073 ceph-mon[32303]: Deploying daemon mon.b on smithi136 2024-06-06T06:50:42.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:43 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/2799065428' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T06:50:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:43 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/2799065428' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T06:50:44.230 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-06T06:50:44.230 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-06T06:50:44.469 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:50:44.578 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:44 smithi170 ceph-mon[36108]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:44.676 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:44 smithi073 ceph-mon[32303]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:45.102 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:45 smithi136 ceph-mon[35901]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-06-06T06:50:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:45 smithi136 bash[35882]: 93d43cff2aa56148bec10e031a66852a2b87ecbd32d48c8694c2f74429bd02a3 2024-06-06T06:50:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:45 smithi136 systemd[1]: Started Ceph mon.b for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:50:50.292 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 -- 172.21.15.170:0/207900137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03040035c0 msgr2=0x7f0304005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 --2- 172.21.15.170:0/207900137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03040035c0 0x7f0304005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f031c101ee0 tx=0x7f030c00fe60 comp rx=0 tx=0).stop 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 -- 172.21.15.170:0/207900137 shutdown_connections 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 --2- 172.21.15.170:0/207900137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03040035c0 0x7f0304005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 -- 172.21.15.170:0/207900137 >> 172.21.15.170:0/207900137 conn(0x7f031c0fdaa0 msgr2=0x7f031c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 -- 172.21.15.170:0/207900137 shutdown_connections 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.291+0000 7f0322759700 1 -- 172.21.15.170:0/207900137 wait complete. 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 Processor -- start 2024-06-06T06:50:50.293 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 -- start start 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 0x7f031c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f031c109e50 con 0x7f031c19dd80 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f031c109cd0 con 0x7f031c197f90 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.292+0000 7f0322759700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f031c109fd0 con 0x7f03040035c0 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:38412/0 (socket says 172.21.15.170:38412) 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 -- 172.21.15.170:0/3720950382 learned_addr learned my addr 172.21.15.170:0/3720950382 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:50:50.295 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 0x7f031c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 msgr2=0x7f031c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 msgr2=0x7f031c19d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031bfff700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f0320cf6700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f0320cf6700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 msgr2=0x7f031c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 msgr2=0x7f031c19d840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f0320cf6700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 msgr2=0x7f031c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f0320cf6700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 msgr2=0x7f031c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f0320cf6700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T06:50:50.296 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 -- 172.21.15.170:0/3720950382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f031c1a4710 con 0x7f031c19dd80 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.293+0000 7f031b7fe700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 0x7f031c197a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f031000b550 tx=0x7f031000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.294+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310004230 con 0x7f031c19dd80 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.294+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03100043d0 con 0x7f031c19dd80 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.294+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f031c1a4940 con 0x7f031c19dd80 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.294+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0310011b80 con 0x7f031c19dd80 2024-06-06T06:50:50.297 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.294+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f031c1a4e80 con 0x7f031c19dd80 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.295+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f031001a400 con 0x7f031c19dd80 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.295+0000 7f03197fa700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 0x7f030803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.295+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f031004cdc0 con 0x7f031c19dd80 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.295+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f031c063af0 con 0x7f031c19dd80 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.295+0000 7f031bfff700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 0x7f030803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:50.298 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.296+0000 7f031bfff700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 0x7f030803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f030c00fa10 tx=0x7f030c0038d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:50.301 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.301+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0310020500 con 0x7f031c19dd80 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: mon.a calling monitor election 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: mon.c calling monitor election 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: monmap epoch 3 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: min_mon_release 19 (squid) 2024-06-06T06:50:50.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: election_strategy: 1 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: fsmap 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: mgrmap e13: x(active, since 58s) 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: overall HEALTH_OK 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:50.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: mon.a calling monitor election 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: mon.c calling monitor election 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: monmap epoch 3 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: min_mon_release 19 (squid) 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: election_strategy: 1 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: fsmap 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: mgrmap e13: x(active, since 58s) 2024-06-06T06:50:50.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: overall HEALTH_OK 2024-06-06T06:50:50.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:50.600 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:50.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.599+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f031c1a51a0 con 0x7f031c19dd80 2024-06-06T06:50:50.600 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.599+0000 7f03197fa700 1 -- 172.21.15.170:0/3720950382 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f03100202a0 con 0x7f031c19dd80 2024-06-06T06:50:50.601 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-06-06T06:50:50.601 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","modified":"2024-06-06T06:50:45.075844Z","created":"2024-06-06T06:48:44.267203Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:3300","nonce":0},{"type":"v1","addr":"172.21.15.136:6789","nonce":0}]},"addr":"172.21.15.136:6789/0","public_addr":"172.21.15.136:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 msgr2=0x7f030803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 0x7f030803ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f030c00fa10 tx=0x7f030c0038d0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 msgr2=0x7f031c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 0x7f031c197a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f031000b550 tx=0x7f031000b910 comp rx=0 tx=0).stop 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 shutdown_connections 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0308039580 0x7f030803ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03040035c0 0x7f031c19d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f031c197f90 0x7f031c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.603 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 --2- 172.21.15.170:0/3720950382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f031c19dd80 0x7f031c197a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:50.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 >> 172.21.15.170:0/3720950382 conn(0x7f031c0fdaa0 msgr2=0x7f031c105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:50.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 shutdown_connections 2024-06-06T06:50:50.604 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:50.602+0000 7f0322759700 1 -- 172.21.15.170:0/3720950382 wait complete. 2024-06-06T06:50:50.604 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-06-06T06:50:51.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:51.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:51.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:51.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:51 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/3720950382' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T06:50:51.273 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:51.299 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-06T06:50:51.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-06T06:50:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:51 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/3720950382' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T06:50:51.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:51.546 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:50:52.523 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.a calling monitor election 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.c calling monitor election 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:52.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: monmap epoch 3 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: min_mon_release 19 (squid) 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: election_strategy: 1 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: fsmap 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mgrmap e13: x(active, since 58s) 2024-06-06T06:50:52.525 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: overall HEALTH_OK 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/3720950382' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.b calling monitor election 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.b calling monitor election 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.a calling monitor election 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.c calling monitor election 2024-06-06T06:50:52.526 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: monmap epoch 3 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: min_mon_release 19 (squid) 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: election_strategy: 1 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: fsmap 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: mgrmap e13: x(active, since 60s) 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: overall HEALTH_OK 2024-06-06T06:50:52.527 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T06:50:52.528 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:52 smithi136 ceph-mon[35901]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T06:50:52.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mon.b calling monitor election 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mon.b calling monitor election 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mon.a calling monitor election 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mon.c calling monitor election 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: monmap epoch 3 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: min_mon_release 19 (squid) 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: election_strategy: 1 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: fsmap 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: mgrmap e13: x(active, since 60s) 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: overall HEALTH_OK 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T06:50:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:52 smithi073 ceph-mon[32303]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T06:50:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mon.b calling monitor election 2024-06-06T06:50:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mon.b calling monitor election 2024-06-06T06:50:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mon.a calling monitor election 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mon.c calling monitor election 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: monmap epoch 3 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: last_changed 2024-06-06T06:50:45.075844+0000 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: created 2024-06-06T06:48:44.267203+0000 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: min_mon_release 19 (squid) 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: election_strategy: 1 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: 0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: 1: [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon.c 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: 2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.b 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: fsmap 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: osdmap e4: 0 total, 0 up, 0 in 2024-06-06T06:50:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: mgrmap e13: x(active, since 60s) 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: overall HEALTH_OK 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T06:50:52.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:52 smithi170 ceph-mon[36108]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.289+0000 7f5404364700 1 -- 172.21.15.73:0/725666290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc100fe0 msgr2=0x7f53fc1013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.289+0000 7f5404364700 1 --2- 172.21.15.73:0/725666290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc100fe0 0x7f53fc1013c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f53f8009910 tx=0x7f53f8009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- 172.21.15.73:0/725666290 shutdown_connections 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 --2- 172.21.15.73:0/725666290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc100fe0 0x7f53fc1013c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- 172.21.15.73:0/725666290 >> 172.21.15.73:0/725666290 conn(0x7f53fc0fc890 msgr2=0x7f53fc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- 172.21.15.73:0/725666290 shutdown_connections 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- 172.21.15.73:0/725666290 wait complete. 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 Processor -- start 2024-06-06T06:50:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- start start 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53fc100fe0 0x7f53fc102a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53fc1a3cf0 0x7f53fc1a60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10b260 con 0x7f53fc102fc0 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10b0e0 con 0x7f53fc1a3cf0 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f5404364700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f53fc10b3e0 con 0x7f53fc100fe0 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f54018ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f54018ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48860/0 (socket says 172.21.15.73:48860) 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.290+0000 7f54018ff700 1 -- 172.21.15.73:0/2374943177 learned_addr learned my addr 172.21.15.73:0/2374943177 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 -- 172.21.15.73:0/2374943177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53fc100fe0 msgr2=0x7f53fc102a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53fc100fe0 0x7f53fc102a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 -- 172.21.15.73:0/2374943177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53fc1a3cf0 msgr2=0x7f53fc1a60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53fc1a3cf0 0x7f53fc1a60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 -- 172.21.15.73:0/2374943177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53fc1a6810 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f54018ff700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00b780 tx=0x7f53ec00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec004230 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53ec0043d0 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec011b80 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53fc1a6b00 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.291+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53fc1a7090 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.292+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f53ec0036a0 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.292+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53fc063af0 con 0x7f53fc102fc0 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.292+0000 7f53f37fe700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 0x7f53e803ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.293+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f53ec04bd40 con 0x7f53fc102fc0 2024-06-06T06:50:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.293+0000 7f5402100700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 0x7f53e803ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.293+0000 7f5402100700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 0x7f53e803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b5c0 tx=0x7f53f8005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:53.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.297+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53ec021e00 con 0x7f53fc102fc0 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.498+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f53fc1a7560 con 0x7f53fc102fc0 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.498+0000 7f53f37fe700 1 -- 172.21.15.73:0/2374943177 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7f53ec01a920 con 0x7f53fc102fc0 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stdout:# minimal ceph.conf for b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T06:50:53.500 INFO:teuthology.orchestra.run.smithi073.stdout: mon_host = [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 msgr2=0x7f53e803ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 0x7f53e803ba40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f53f800b5c0 tx=0x7f53f8005da0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 msgr2=0x7f53fc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00b780 tx=0x7f53ec00bb40 comp rx=0 tx=0).stop 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 shutdown_connections 2024-06-06T06:50:53.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f53e8039580 0x7f53e803ba40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f53fc100fe0 0x7f53fc102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f53fc1a3cf0 0x7f53fc1a60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 --2- 172.21.15.73:0/2374943177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53fc102fc0 0x7f53fc103440 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 >> 172.21.15.73:0/2374943177 conn(0x7f53fc0fc890 msgr2=0x7f53fc108b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 shutdown_connections 2024-06-06T06:50:53.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:50:53.501+0000 7f5404364700 1 -- 172.21.15.73:0/2374943177 wait complete. 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:53.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: Reconfiguring daemon mon.a on smithi073 2024-06-06T06:50:53.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:53.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:53.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:53 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2374943177' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: Reconfiguring daemon mon.a on smithi073 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:54.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:53 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2374943177' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: Reconfiguring mon.a (unknown last config time)... 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: Reconfiguring daemon mon.a on smithi073 2024-06-06T06:50:54.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:50:54.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:54.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:53 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2374943177' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:54.836 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-06T06:50:54.836 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:54.837 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T06:50:54.868 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:54.868 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:54.935 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:50:54.935 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T06:50:54.968 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:50:54.968 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:55.038 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:50:55.038 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-06-06T06:50:55.073 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:50:55.073 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-06T06:50:55.143 INFO:tasks.cephadm:Adding mgr.x on smithi073 2024-06-06T06:50:55.143 INFO:tasks.cephadm:Adding mgr.y on smithi136 2024-06-06T06:50:55.143 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-06-06T06:50:55.144 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi073=x;smithi136=y;smithi170=z' 2024-06-06T06:50:55.419 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: Reconfiguring mon.b (monmap changed)... 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: Reconfiguring daemon mon.b on smithi136 2024-06-06T06:50:56.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:56 smithi136 ceph-mon[35901]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:56.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: Reconfiguring mon.b (monmap changed)... 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: Reconfiguring daemon mon.b on smithi136 2024-06-06T06:50:56.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:56 smithi170 ceph-mon[36108]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:56.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: Reconfiguring mon.b (monmap changed)... 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: Reconfiguring daemon mon.b on smithi136 2024-06-06T06:50:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:56 smithi073 ceph-mon[32303]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 -- 172.21.15.170:0/2200125117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104102010 msgr2=0x7f2104102430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 --2- 172.21.15.170:0/2200125117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104102010 0x7f2104102430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2100009910 tx=0x7f2100009c20 comp rx=0 tx=0).stop 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 -- 172.21.15.170:0/2200125117 shutdown_connections 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 --2- 172.21.15.170:0/2200125117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104102010 0x7f2104102430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 -- 172.21.15.170:0/2200125117 >> 172.21.15.170:0/2200125117 conn(0x7f21040fd5b0 msgr2=0x7f21040ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 -- 172.21.15.170:0/2200125117 shutdown_connections 2024-06-06T06:50:57.028 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.025+0000 7f210ba63700 1 -- 172.21.15.170:0/2200125117 wait complete. 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 Processor -- start 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 -- start start 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104197cb0 0x7f210419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2104198130 0x7f210419f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2104107f50 con 0x7f2104197cb0 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2104107dd0 con 0x7f2104198130 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f210ba63700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21041080d0 con 0x7f2104102010 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f21097ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f21097ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:36676/0 (socket says 172.21.15.170:36676) 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f21097ff700 1 -- 172.21.15.170:0/1360117814 learned_addr learned my addr 172.21.15.170:0/1360117814 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:50:57.029 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.026+0000 7f2108ffe700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104197cb0 0x7f210419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 -- 172.21.15.170:0/1360117814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2104198130 msgr2=0x7f210419f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2104198130 0x7f210419f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 -- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104197cb0 msgr2=0x7f210419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104197cb0 0x7f210419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 -- 172.21.15.170:0/1360117814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210419fb80 con 0x7f2104102010 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f21097ff700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2100005e00 tx=0x7f2100009e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100027070 con 0x7f2104102010 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f210000be10 con 0x7f2104102010 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210419fe10 con 0x7f2104102010 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f210000f460 con 0x7f2104102010 2024-06-06T06:50:57.030 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.027+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21041a02e0 con 0x7f2104102010 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.028+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f210002a070 con 0x7f2104102010 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.028+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210402dca0 con 0x7f2104102010 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.028+0000 7f20fa7fc700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 0x7f20f003ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.028+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f210004d3c0 con 0x7f2104102010 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.028+0000 7f2108ffe700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 0x7f20f003ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:50:57.032 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.029+0000 7f2108ffe700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 0x7f20f003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009850 tx=0x7f20f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:50:57.034 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.033+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2100013070 con 0x7f2104102010 2024-06-06T06:50:57.250 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.249+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi073=x;smithi136=y;smithi170=z", "target": ["mon-mgr", ""]}) -- 0x7f2104062960 con 0x7f20f0039580 2024-06-06T06:50:57.259 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.258+0000 7f20fa7fc700 1 -- 172.21.15.170:0/1360117814 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2104062960 con 0x7f20f0039580 2024-06-06T06:50:57.259 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-06-06T06:50:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 msgr2=0x7f20f003ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 0x7f20f003ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f20f4009850 tx=0x7f20f4008040 comp rx=0 tx=0).stop 2024-06-06T06:50:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 msgr2=0x7f2104197770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:50:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2100005e00 tx=0x7f2100009e60 comp rx=0 tx=0).stop 2024-06-06T06:50:57.261 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 shutdown_connections 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f20f0039580 0x7f20f003ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2104102010 0x7f2104197770 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2104198130 0x7f210419f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 --2- 172.21.15.170:0/1360117814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2104197cb0 0x7f210419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 >> 172.21.15.170:0/1360117814 conn(0x7f21040fd5b0 msgr2=0x7f21040fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 shutdown_connections 2024-06-06T06:50:57.262 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:50:57.260+0000 7f210ba63700 1 -- 172.21.15.170:0/1360117814 wait complete. 2024-06-06T06:50:57.866 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: Reconfiguring mon.c (monmap changed)... 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: Reconfiguring daemon mon.c on smithi170 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi073=x;smithi136=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: Saving service mgr spec with placement smithi073=x;smithi136=y;smithi170=z;count:3 2024-06-06T06:50:57.867 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.894 DEBUG:teuthology.orchestra.run.smithi136:mgr.y> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-06T06:50:57.896 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-06T06:50:57.899 INFO:tasks.cephadm:Deploying OSDs... 2024-06-06T06:50:57.899 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:50:57.899 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T06:50:57.918 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T06:50:57.918 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: Reconfiguring mon.c (monmap changed)... 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: Reconfiguring daemon mon.c on smithi170 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi073=x;smithi136=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:57.921 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: Saving service mgr spec with placement smithi073=x;smithi136=y;smithi170=z;count:3 2024-06-06T06:50:57.922 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-06-06 06:50:03.736603048 +0000 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-06-06 06:45:31.913845847 +0000 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-06-06 06:45:31.913845847 +0000 2024-06-06T06:50:57.976 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-06-06 06:45:31.913845847 +0000 2024-06-06T06:50:57.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T06:50:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: Reconfiguring mon.c (monmap changed)... 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: Reconfiguring daemon mon.c on smithi170 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi073=x;smithi136=y;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: Saving service mgr spec with placement smithi073=x;smithi136=y;smithi170=z;count:3 2024-06-06T06:50:58.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-06-06T06:50:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-06-06T06:50:58.042 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000300624 s, 1.7 MB/s 2024-06-06T06:50:58.044 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T06:50:58.101 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-06-06T06:50:58.161 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T06:50:58.161 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:58.161 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-06-06 06:50:03.769602313 +0000 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-06-06 06:45:32.243838471 +0000 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-06-06 06:45:32.243838471 +0000 2024-06-06T06:50:58.162 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-06-06 06:45:32.243838471 +0000 2024-06-06T06:50:58.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T06:50:58.226 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-06-06T06:50:58.226 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-06-06T06:50:58.226 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000179542 s, 2.9 MB/s 2024-06-06T06:50:58.228 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T06:50:58.284 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-06-06 06:50:03.800601622 +0000 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-06-06 06:45:32.581830916 +0000 2024-06-06T06:50:58.340 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-06-06 06:45:32.581830916 +0000 2024-06-06T06:50:58.341 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-06-06 06:45:32.581830916 +0000 2024-06-06T06:50:58.341 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T06:50:58.404 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-06-06T06:50:58.404 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-06-06T06:50:58.404 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000221097 s, 2.3 MB/s 2024-06-06T06:50:58.405 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T06:50:58.461 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-06-06T06:50:58.516 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T06:50:58.516 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-06-06 06:50:03.833600886 +0000 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-06-06 06:45:32.931823093 +0000 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-06-06 06:45:32.931823093 +0000 2024-06-06T06:50:58.517 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-06-06 06:45:32.931823093 +0000 2024-06-06T06:50:58.517 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T06:50:58.580 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-06-06T06:50:58.580 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-06-06T06:50:58.580 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000194115 s, 2.6 MB/s 2024-06-06T06:50:58.581 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T06:50:58.637 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-06-06T06:50:58.637 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T06:50:58.656 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T06:50:58.657 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-06-06T06:50:58.714 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 670 Links: 1 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-06 06:50:44.522475494 +0000 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-06 06:45:32.759946930 +0000 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-06 06:45:32.759946930 +0000 2024-06-06T06:50:58.715 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-06 06:45:32.759946930 +0000 2024-06-06T06:50:58.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T06:50:58.782 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-06T06:50:58.782 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-06T06:50:58.782 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000221433 s, 2.3 MB/s 2024-06-06T06:50:58.783 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T06:50:58.841 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 690 Links: 1 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-06 06:50:44.523475472 +0000 2024-06-06T06:50:58.898 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-06 06:45:33.078940163 +0000 2024-06-06T06:50:58.899 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-06 06:45:33.078940163 +0000 2024-06-06T06:50:58.899 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-06 06:45:33.078940163 +0000 2024-06-06T06:50:58.899 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T06:50:58.963 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-06T06:50:58.964 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-06T06:50:58.964 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000212107 s, 2.4 MB/s 2024-06-06T06:50:58.965 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T06:50:59.022 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-06-06T06:50:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T06:50:59.078 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.078 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 708 Links: 1 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-06 06:50:44.523475472 +0000 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-06 06:45:33.425932804 +0000 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-06 06:45:33.425932804 +0000 2024-06-06T06:50:59.079 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-06 06:45:33.425932804 +0000 2024-06-06T06:50:59.079 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T06:50:59.143 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-06T06:50:59.143 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-06T06:50:59.143 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000213785 s, 2.4 MB/s 2024-06-06T06:50:59.144 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T06:50:59.201 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-06-06T06:50:59.257 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-06-06 06:50:44.523475472 +0000 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-06-06 06:45:33.756925784 +0000 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-06-06 06:45:33.756925784 +0000 2024-06-06T06:50:59.258 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: 2024-06-06 06:45:33.756925784 +0000 2024-06-06T06:50:59.258 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T06:50:59.336 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-06-06T06:50:59.336 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-06-06T06:50:59.336 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000199366 s, 2.6 MB/s 2024-06-06T06:50:59.337 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T06:50:59.396 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-06-06T06:50:59.397 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-06-06T06:50:59.413 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-06T06:50:59.413 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-06-06T06:50:59.471 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-06T06:50:59.471 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-06-06 06:50:35.308182339 +0000 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-06-06 06:45:32.988536754 +0000 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-06-06 06:45:32.988536754 +0000 2024-06-06T06:50:59.472 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-06-06 06:45:32.988536754 +0000 2024-06-06T06:50:59.473 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-06T06:50:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-06-06T06:50:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-06-06T06:50:59.539 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000113088 s, 4.5 MB/s 2024-06-06T06:50:59.541 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-06T06:50:59.601 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-06-06T06:50:59.657 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-06-06 06:50:35.309182315 +0000 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-06-06 06:45:33.299529372 +0000 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-06-06 06:45:33.299529372 +0000 2024-06-06T06:50:59.658 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-06-06 06:45:33.299529372 +0000 2024-06-06T06:50:59.658 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-06T06:50:59.721 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-06-06T06:50:59.722 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-06-06T06:50:59.722 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.227e-05 s, 5.5 MB/s 2024-06-06T06:50:59.723 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-06T06:50:59.780 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-06-06 06:50:35.309182315 +0000 2024-06-06T06:50:59.837 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-06-06 06:45:33.616521846 +0000 2024-06-06T06:50:59.838 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-06-06 06:45:33.616521846 +0000 2024-06-06T06:50:59.838 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-06-06 06:45:33.616521846 +0000 2024-06-06T06:50:59.838 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:50:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:50:59.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:50:59.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:50:59.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:50:59.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:50:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:50:59.871 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-06-06T06:50:59.871 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-06-06T06:50:59.871 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.3702e-05 s, 5.5 MB/s 2024-06-06T06:50:59.873 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-06T06:50:59.930 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-06-06T06:50:59.985 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-06T06:50:59.985 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-06T06:50:59.985 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-06-06 06:50:35.310182291 +0000 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-06-06 06:45:33.933514321 +0000 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-06-06 06:45:33.933514321 +0000 2024-06-06T06:50:59.986 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-06-06 06:45:33.933514321 +0000 2024-06-06T06:50:59.986 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-06T06:51:00.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:00.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:51:00.027 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:00.027 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:50:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:00.050 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-06-06T06:51:00.051 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-06-06T06:51:00.051 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.8082e-05 s, 5.8 MB/s 2024-06-06T06:51:00.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:00.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:50:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:00.053 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-06T06:51:00.109 INFO:tasks.cephadm:Deploying osd.0 on smithi073 with /dev/vg_nvme/lv_4... 2024-06-06T06:51:00.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T06:51:00.342 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:51:01.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:00 smithi073 ceph-mon[32303]: Deploying daemon mgr.z on smithi170 2024-06-06T06:51:01.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:00 smithi136 ceph-mon[35901]: Deploying daemon mgr.z on smithi170 2024-06-06T06:51:01.090 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:00 smithi170 ceph-mon[36108]: Deploying daemon mgr.z on smithi170 2024-06-06T06:51:01.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 systemd[1]: Starting Ceph mgr.z for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:51:01.772 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 podman[37925]: 2024-06-06 06:51:01.423118269 +0000 UTC m=+0.016955930 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:51:01.772 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 podman[37925]: 2024-06-06 06:51:01.537655042 +0000 UTC m=+0.131492677 container create 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:51:01.773 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 podman[37925]: 2024-06-06 06:51:01.750619492 +0000 UTC m=+0.344457132 container init 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T06:51:01.773 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 podman[37925]: 2024-06-06 06:51:01.753317426 +0000 UTC m=+0.347155066 container start 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-06T06:51:02.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:01 smithi073 ceph-mon[32303]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:02.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:01 smithi136 ceph-mon[35901]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:02.092 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:01 smithi170 ceph-mon[36108]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:02.092 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 ceph-mgr[37944]: -- 172.21.15.170:0/549474945 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e573645a0 con 0x557e57339000 2024-06-06T06:51:02.092 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 bash[37925]: 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc 2024-06-06T06:51:02.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 systemd[1]: Started Ceph mgr.z for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:51:02.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:01.908+0000 7fefa6ff9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T06:51:02.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:02.091+0000 7fefa6ff9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:51:02.222 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T06:51:02.222 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T06:51:02.222 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-06-06T06:51:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-06-06T06:51:02.223 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133925 s, 783 MB/s 2024-06-06T06:51:02.223 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-06-06T06:51:02.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:02.194+0000 7fefa6ff9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T06:51:02.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:02.504+0000 7fefa6ff9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T06:51:02.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:02.589+0000 7fefa6ff9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T06:51:02.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_4 2024-06-06T06:51:03.091 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:03.187 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:02 smithi170 ceph-mon[36108]: Deploying daemon mgr.y on smithi136 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:03.200 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:02 smithi073 ceph-mon[32303]: Deploying daemon mgr.y on smithi136 2024-06-06T06:51:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:02 smithi136 ceph-mon[35901]: Deploying daemon mgr.y on smithi136 2024-06-06T06:51:03.894 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:03 smithi136 systemd[1]: Starting Ceph mgr.y for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:51:04.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:03 smithi170 ceph-mon[36108]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:04.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:03.821+0000 7fefa6ff9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T06:51:04.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:03.994+0000 7fefa6ff9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T06:51:04.130 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:03 smithi073 ceph-mon[32303]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:04.190 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:03 smithi136 ceph-mon[35901]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:04.190 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:03 smithi136 podman[37246]: 2024-06-06 06:51:03.884977548 +0000 UTC m=+0.017296263 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:51:04.190 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:03 smithi136 podman[37246]: 2024-06-06 06:51:03.996415061 +0000 UTC m=+0.128733763 container create b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-06-06T06:51:04.190 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 podman[37246]: 2024-06-06 06:51:04.19009272 +0000 UTC m=+0.322411429 container init b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-06T06:51:04.460 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:04.171+0000 7fefa6ff9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T06:51:04.504 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:04.458+0000 7fefa6ff9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T06:51:04.553 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 podman[37246]: 2024-06-06 06:51:04.194642062 +0000 UTC m=+0.326960758 container start b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:51:04.553 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 ceph-mgr[37266]: -- 172.21.15.136:0/1947230254 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557974cda5a0 con 0x557974caf000 2024-06-06T06:51:04.553 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 bash[37246]: b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 2024-06-06T06:51:04.553 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 systemd[1]: Started Ceph mgr.y for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:51:04.553 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:04.377+0000 7faaf0a7b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T06:51:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.622+0000 7fbe61234700 1 -- 172.21.15.73:0/1878619765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c104140 msgr2=0x7fbe5c10a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.622+0000 7fbe61234700 1 --2- 172.21.15.73:0/1878619765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c104140 0x7fbe5c10a9f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400b0a0 tx=0x7fbe5400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 -- 172.21.15.73:0/1878619765 shutdown_connections 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 --2- 172.21.15.73:0/1878619765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c104140 0x7fbe5c10a9f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 --2- 172.21.15.73:0/1878619765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c1007c0 0x7fbe5c100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 --2- 172.21.15.73:0/1878619765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 -- 172.21.15.73:0/1878619765 >> 172.21.15.73:0/1878619765 conn(0x7fbe5c0fc100 msgr2=0x7fbe5c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:51:04.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 -- 172.21.15.73:0/1878619765 shutdown_connections 2024-06-06T06:51:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 -- 172.21.15.73:0/1878619765 wait complete. 2024-06-06T06:51:04.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.623+0000 7fbe61234700 1 Processor -- start 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 -- start start 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1007c0 0x7fbe5c103100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c104140 0x7fbe5c101c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10df90 con 0x7fbe5c1037a0 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10de10 con 0x7fbe5c104140 2024-06-06T06:51:04.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe61234700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c10e110 con 0x7fbe5c1007c0 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60850/0 (socket says 172.21.15.73:60850) 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 -- 172.21.15.73:0/2274565909 learned_addr learned my addr 172.21.15.73:0/2274565909 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5ad9d700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1007c0 0x7fbe5c103100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5b59e700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c104140 0x7fbe5c101c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 -- 172.21.15.73:0/2274565909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1007c0 msgr2=0x7fbe5c103100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:04.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1007c0 0x7fbe5c103100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 -- 172.21.15.73:0/2274565909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c104140 msgr2=0x7fbe5c101c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c104140 0x7fbe5c101c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 -- 172.21.15.73:0/2274565909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c102540 con 0x7fbe5c1037a0 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.624+0000 7fbe5a59c700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d980 tx=0x7fbe5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.625+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50017070 con 0x7fbe5c1037a0 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.625+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c1a2700 con 0x7fbe5c1037a0 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.625+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe50005020 con 0x7fbe5c1037a0 2024-06-06T06:51:04.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.625+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000fa90 con 0x7fbe5c1037a0 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.625+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c1a2bc0 con 0x7fbe5c1037a0 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fbe5000fc30 con 0x7fbe5c1037a0 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c069180 con 0x7fbe5c1037a0 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe43fff700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 0x7fbe4403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fbe5004c160 con 0x7fbe5c1037a0 2024-06-06T06:51:04.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe5ad9d700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 0x7fbe4403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:04.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.626+0000 7fbe5ad9d700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 0x7fbe4403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c06aa40 tx=0x7fbe4c00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:04.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.631+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbe50020020 con 0x7fbe5c1037a0 2024-06-06T06:51:04.765 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:04.550+0000 7fefa6ff9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T06:51:04.766 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:04.764+0000 7fefa6ff9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T06:51:04.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:04.837+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbe5c0628d0 con 0x7fbe44039330 2024-06-06T06:51:05.018 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:04.574+0000 7faaf0a7b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:51:05.018 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:04.686+0000 7faaf0a7b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T06:51:05.018 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:05.017+0000 7faaf0a7b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T06:51:05.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:04.964+0000 7fefa6ff9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:51:05.302 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:05.125+0000 7faaf0a7b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-mon[36108]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:05.626 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.626 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.626 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.626 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.626 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:05.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:05.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:05.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:05.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:05.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:05 smithi136 ceph-mon[35901]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:05.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:05.654 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:05.654 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:05.654 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:05 smithi073 ceph-mon[32303]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:06.041 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:05.607+0000 7fefa6ff9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T06:51:06.341 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:06.039+0000 7fefa6ff9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T06:51:06.341 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:06.140+0000 7fefa6ff9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T06:51:06.341 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:06.340+0000 7fefa6ff9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T06:51:06.708 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:06.373+0000 7faaf0a7b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T06:51:06.708 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:06.539+0000 7faaf0a7b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T06:51:06.709 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:06.707+0000 7faaf0a7b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T06:51:06.993 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:06.994 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:06.994 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:06.994 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: Reconfiguring daemon mgr.x on smithi073 2024-06-06T06:51:06.994 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: Reconfiguring daemon mgr.x on smithi073 2024-06-06T06:51:07.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:06 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:06.869+0000 7fefa6ff9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:06.961+0000 7fefa6ff9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:07.205 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: Reconfiguring mgr.x (unknown last config time)... 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: Reconfiguring daemon mgr.x on smithi073 2024-06-06T06:51:07.206 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:06 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:07.284 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:06.991+0000 7faaf0a7b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T06:51:07.285 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:07.081+0000 7faaf0a7b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T06:51:07.285 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:07.282+0000 7faaf0a7b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T06:51:07.548 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:07.203+0000 7fefa6ff9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T06:51:07.549 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:07.338+0000 7fefa6ff9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T06:51:07.549 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:07.432+0000 7fefa6ff9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T06:51:07.549 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:07.547+0000 7fefa6ff9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T06:51:07.551 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:07.472+0000 7faaf0a7b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T06:51:08.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:07 smithi073 ceph-mon[32303]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:08.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:07 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]: dispatch 2024-06-06T06:51:08.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:07 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]': finished 2024-06-06T06:51:08.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:07 smithi073 ceph-mon[32303]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T06:51:08.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:08.301 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:08.066+0000 7faaf0a7b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T06:51:08.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-mon[35901]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:08.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]: dispatch 2024-06-06T06:51:08.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]': finished 2024-06-06T06:51:08.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-mon[35901]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T06:51:08.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:07.848+0000 7fefa6ff9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-mon[36108]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]: dispatch 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4057135349' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f1af7d41-9f4c-4b4a-8a05-c38530312412"}]': finished 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-mon[36108]: osdmap e5: 1 total, 0 up, 1 in 2024-06-06T06:51:08.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:08.774 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:08.482+0000 7faaf0a7b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T06:51:08.775 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:08.577+0000 7faaf0a7b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T06:51:08.775 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:08.773+0000 7faaf0a7b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T06:51:08.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:08.441+0000 7fefa6ff9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T06:51:08.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T06:51:08.656+0000 7fefa6ff9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T06:51:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:08.869+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7fbe5001b400 con 0x7fbe5c1037a0 2024-06-06T06:51:09.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1036330451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:09.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: Standby manager daemon z started 2024-06-06T06:51:09.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T06:51:09.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:09.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T06:51:09.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:08 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1036330451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: Standby manager daemon z started 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T06:51:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:08 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1036330451' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: Standby manager daemon z started 2024-06-06T06:51:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T06:51:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T06:51:09.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:08 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/2264253639' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:09.551 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.289+0000 7faaf0a7b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T06:51:09.551 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.379+0000 7faaf0a7b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T06:51:09.877 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.616+0000 7faaf0a7b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T06:51:09.877 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.746+0000 7faaf0a7b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T06:51:09.877 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.839+0000 7faaf0a7b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T06:51:10.254 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:09.953+0000 7faaf0a7b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T06:51:10.254 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:10.252+0000 7faaf0a7b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T06:51:10.254 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-mon[35901]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T06:51:10.254 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T06:51:10.255 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:09 smithi136 ceph-mon[35901]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:09 smithi073 ceph-mon[32303]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T06:51:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T06:51:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:09 smithi073 ceph-mon[32303]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:09 smithi170 ceph-mon[36108]: mgrmap e14: x(active, since 77s), standbys: z 2024-06-06T06:51:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T06:51:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:09 smithi170 ceph-mon[36108]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:11.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:11.115+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fbe50017ca0 con 0x7fbe5c1037a0 2024-06-06T06:51:11.122 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:10.841+0000 7faaf0a7b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T06:51:11.122 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T06:51:11.052+0000 7faaf0a7b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T06:51:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:11 smithi073 ceph-mon[32303]: Standby manager daemon y started 2024-06-06T06:51:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:11 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T06:51:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:11 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:11 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T06:51:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:11 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-mon[35901]: Standby manager daemon y started 2024-06-06T06:51:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T06:51:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T06:51:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:11 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:11.602 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:11 smithi170 ceph-mon[36108]: Standby manager daemon y started 2024-06-06T06:51:11.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:11 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T06:51:11.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:11 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T06:51:11.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:11 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T06:51:11.603 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:11 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/4092101330' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T06:51:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:12 smithi073 ceph-mon[32303]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T06:51:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:12 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T06:51:12.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:12 smithi073 ceph-mon[32303]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:12.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:12 smithi136 ceph-mon[35901]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T06:51:12.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:12 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T06:51:12.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:12 smithi136 ceph-mon[35901]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:12 smithi170 ceph-mon[36108]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-06T06:51:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:12 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T06:51:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:12 smithi170 ceph-mon[36108]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:14.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:14.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:14.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:14.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:14 smithi170 ceph-mon[36108]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.395 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:14 smithi073 ceph-mon[32303]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:14.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:14 smithi136 ceph-mon[35901]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:16.436 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:16 smithi073 ceph-mon[32303]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:16.436 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:16 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T06:51:16.436 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:16 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:16.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:16 smithi136 ceph-mon[35901]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:16.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:16 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T06:51:16.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:16 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:16.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:16 smithi170 ceph-mon[36108]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:16.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:16 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-06T06:51:16.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:16 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:17.442 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:17 smithi073 ceph-mon[32303]: Deploying daemon osd.0 on smithi073 2024-06-06T06:51:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:17 smithi170 ceph-mon[36108]: Deploying daemon osd.0 on smithi073 2024-06-06T06:51:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:17 smithi136 ceph-mon[35901]: Deploying daemon osd.0 on smithi073 2024-06-06T06:51:18.437 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:18 smithi073 ceph-mon[32303]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:18 smithi170 ceph-mon[36108]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:18 smithi136 ceph-mon[35901]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:20.412 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:20 smithi073 ceph-mon[32303]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:20.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:20 smithi170 ceph-mon[36108]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:20 smithi136 ceph-mon[35901]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:22.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:21 smithi073 ceph-mon[32303]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:21 smithi136 ceph-mon[35901]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:22.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:22.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:22.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:21 smithi170 ceph-mon[36108]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:24.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.200+0000 7fbe43fff700 1 -- 172.21.15.73:0/2274565909 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbe5c0628d0 con 0x7fbe44039330 2024-06-06T06:51:24.201 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 0 on host 'smithi073' 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 msgr2=0x7fbe4403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 0x7fbe4403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c06aa40 tx=0x7fbe4c00a3b0 comp rx=0 tx=0).stop 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 msgr2=0x7fbe5c101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d980 tx=0x7fbe5000dc90 comp rx=0 tx=0).stop 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 shutdown_connections 2024-06-06T06:51:24.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbe44039330 0x7fbe4403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbe5c1007c0 0x7fbe5c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbe5c104140 0x7fbe5c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 --2- 172.21.15.73:0/2274565909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe5c1037a0 0x7fbe5c101750 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 >> 172.21.15.73:0/2274565909 conn(0x7fbe5c0fc100 msgr2=0x7fbe5c106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 shutdown_connections 2024-06-06T06:51:24.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:51:24.202+0000 7fbe61234700 1 -- 172.21.15.73:0/2274565909 wait complete. 2024-06-06T06:51:24.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:24 smithi073 ceph-mon[32303]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:24.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:24.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:24.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:24 smithi136 ceph-mon[35901]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:24.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:24 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:24.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:24 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:24 smithi170 ceph-mon[36108]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:24 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:24 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:25.213 DEBUG:teuthology.orchestra.run.smithi073:osd.0> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.0.service 2024-06-06T06:51:25.215 INFO:tasks.cephadm:Deploying osd.1 on smithi136 with /dev/vg_nvme/lv_4... 2024-06-06T06:51:25.215 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T06:51:25.464 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:51:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:25 smithi073 ceph-mon[32303]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T06:51:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:25 smithi136 ceph-mon[35901]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T06:51:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:25 smithi170 ceph-mon[36108]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-06T06:51:26.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:26.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:26.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T06:51:26.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T06:51:26.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:51:26.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:26.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:26.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: osdmap e6: 1 total, 0 up, 1 in 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:27.025 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0[47405]: 2024-06-06T06:51:26.762+0000 7f70eb69c700 -1 osd.0 0 waiting for initial osdmap 2024-06-06T06:51:27.025 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:51:26 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0[47405]: 2024-06-06T06:51:26.767+0000 7f70e3487700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00842284 s, 1.2 GB/s 2024-06-06T06:51:27.318 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-06T06:51:27.456 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:27 smithi073 ceph-mon[32303]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:51:27.456 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:27 smithi073 ceph-mon[32303]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T06:51:27.456 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.456 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.456 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:27 smithi073 ceph-mon[32303]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' 2024-06-06T06:51:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:27 smithi170 ceph-mon[36108]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:51:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:27 smithi170 ceph-mon[36108]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T06:51:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:27 smithi170 ceph-mon[36108]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' 2024-06-06T06:51:27.617 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:27 smithi136 ceph-mon[35901]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:51:27.617 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:27 smithi136 ceph-mon[35901]: osdmap e7: 1 total, 0 up, 1 in 2024-06-06T06:51:27.618 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.618 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:27.618 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:27 smithi136 ceph-mon[35901]: from='osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649]' entity='osd.0' 2024-06-06T06:51:28.009 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_4 2024-06-06T06:51:28.257 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:51:28.588 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:51:28.588 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:51:28.588 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:28.588 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:28.589 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] boot 2024-06-06T06:51:28.589 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T06:51:28.589 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:28 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] boot 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T06:51:28.590 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:28 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: osd.0 [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] boot 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: osdmap e8: 1 total, 1 up, 1 in 2024-06-06T06:51:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:28 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- 172.21.15.136:0/824807761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87581005a0 msgr2=0x7f8758100a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 --2- 172.21.15.136:0/824807761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87581005a0 0x7f8758100a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f874c0099c0 tx=0x7f874c009cd0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- 172.21.15.136:0/824807761 shutdown_connections 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 --2- 172.21.15.136:0/824807761 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8758100f60 0x7f875810ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 --2- 172.21.15.136:0/824807761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758103b20 0x7f8758103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 --2- 172.21.15.136:0/824807761 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f87581005a0 0x7f8758100a20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- 172.21.15.136:0/824807761 >> 172.21.15.136:0/824807761 conn(0x7f87580fc140 msgr2=0x7f87580fe580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- 172.21.15.136:0/824807761 shutdown_connections 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- 172.21.15.136:0/824807761 wait complete. 2024-06-06T06:51:29.766 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 Processor -- start 2024-06-06T06:51:29.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.765+0000 7f875db67700 1 -- start start 2024-06-06T06:51:29.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87581005a0 0x7f8758198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:29.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758100f60 0x7f8758198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:29.768 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f875810e030 con 0x7f8758100f60 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f875810deb0 con 0x7f8758103b20 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f875810e1b0 con 0x7f87581005a0 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8756ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758100f60 0x7f8758198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:39160/0 (socket says 172.21.15.136:39160) 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 -- 172.21.15.136:0/2622812731 learned_addr learned my addr 172.21.15.136:0/2622812731 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f87577fe700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87581005a0 0x7f8758198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 -- 172.21.15.136:0/2622812731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87581005a0 msgr2=0x7f8758198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87581005a0 0x7f8758198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 -- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758100f60 msgr2=0x7f8758198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:29.769 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758100f60 0x7f8758198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 -- 172.21.15.136:0/2622812731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87581a02d0 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8757fff700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f875000c970 tx=0x7f875000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8750004a30 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f87500054d0 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.766+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87581a05c0 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.767+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875000ebb0 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.767+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87581a0b00 con 0x7f8758103b20 2024-06-06T06:51:29.770 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.767+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8758069180 con 0x7f8758103b20 2024-06-06T06:51:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.769+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f875000ed50 con 0x7f8758103b20 2024-06-06T06:51:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.770+0000 7f8754ff9700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 0x7f87400a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.770+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7f87500cd300 con 0x7f8758103b20 2024-06-06T06:51:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.770+0000 7f87577fe700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 0x7f87400a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:29.771 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.770+0000 7f87577fe700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 0x7f87400a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f875806a900 tx=0x7f87480091c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:29.776 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.776+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8750014030 con 0x7f8758103b20 2024-06-06T06:51:29.987 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:29.986+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f87580628d0 con 0x7f87400a2b90 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi073 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 17088M 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:29.988 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:29.989 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:29.989 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T06:51:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi073 2024-06-06T06:51:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 17088M 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:30.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: osdmap e9: 1 total, 1 up, 1 in 2024-06-06T06:51:30.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi073 2024-06-06T06:51:30.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 17088M 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:30.278 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:30.872 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:30 smithi136 ceph-mon[35901]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:30.873 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:30.873 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:30.873 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:31.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:30 smithi170 ceph-mon[36108]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:31.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:31.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:31.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:30 smithi073 ceph-mon[32303]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:31.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:31.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:31.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:31.911 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:31 smithi136 ceph-mon[35901]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:32.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:31 smithi170 ceph-mon[36108]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:32.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:31 smithi073 ceph-mon[32303]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:33.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:32 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/3549583302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:32 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:32 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]': finished 2024-06-06T06:51:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:32 smithi136 ceph-mon[35901]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T06:51:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:32 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:32 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/3549583302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:32 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:32 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]': finished 2024-06-06T06:51:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:32 smithi170 ceph-mon[36108]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T06:51:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:32 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:32 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/3549583302' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:32 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]: dispatch 2024-06-06T06:51:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:32 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ec8a9433-7b32-4f79-8a39-4137d445acfc"}]': finished 2024-06-06T06:51:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:32 smithi073 ceph-mon[32303]: osdmap e10: 2 total, 1 up, 2 in 2024-06-06T06:51:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:34.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:33 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/3353416913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:34.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:33 smithi136 ceph-mon[35901]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:34.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:33 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/3353416913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:34.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:33 smithi170 ceph-mon[36108]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:34.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:33 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/3353416913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:34.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:33 smithi073 ceph-mon[32303]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:36.524 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:36 smithi136 ceph-mon[35901]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:36 smithi170 ceph-mon[36108]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:36 smithi073 ceph-mon[32303]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:38.516 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:38 smithi136 ceph-mon[35901]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:38 smithi170 ceph-mon[36108]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:38 smithi073 ceph-mon[32303]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:40.455 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:40 smithi136 ceph-mon[35901]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:40.455 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T06:51:40.455 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:40 smithi170 ceph-mon[36108]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T06:51:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:40 smithi073 ceph-mon[32303]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-06T06:51:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:41.494 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:41 smithi136 ceph-mon[35901]: Deploying daemon osd.1 on smithi136 2024-06-06T06:51:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:41 smithi170 ceph-mon[36108]: Deploying daemon osd.1 on smithi136 2024-06-06T06:51:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:41 smithi073 ceph-mon[32303]: Deploying daemon osd.1 on smithi136 2024-06-06T06:51:42.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:42 smithi136 ceph-mon[35901]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:42 smithi170 ceph-mon[36108]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:42.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:42 smithi073 ceph-mon[32303]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:44.495 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:44 smithi136 ceph-mon[35901]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:44.576 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:44 smithi073 ceph-mon[32303]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:44 smithi170 ceph-mon[36108]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:45.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:45.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:45.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:45 smithi136 ceph-mon[35901]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:46.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:46.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:46.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:46.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:45 smithi073 ceph-mon[32303]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:45 smithi170 ceph-mon[36108]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:48.148 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.147+0000 7f8754ff9700 1 -- 172.21.15.136:0/2622812731 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f87580628d0 con 0x7f87400a2b90 2024-06-06T06:51:48.148 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 1 on host 'smithi136' 2024-06-06T06:51:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 msgr2=0x7f87400a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 0x7f87400a5050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f875806a900 tx=0x7f87480091c0 comp rx=0 tx=0).stop 2024-06-06T06:51:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 msgr2=0x7f875819fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f875000c970 tx=0x7f875000cc80 comp rx=0 tx=0).stop 2024-06-06T06:51:48.150 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 shutdown_connections 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f87400a2b90 0x7f87400a5050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f87581005a0 0x7f8758198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8758103b20 0x7f875819fc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 --2- 172.21.15.136:0/2622812731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8758100f60 0x7f8758198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 >> 172.21.15.136:0/2622812731 conn(0x7f87580fc140 msgr2=0x7f8758106bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 shutdown_connections 2024-06-06T06:51:48.151 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:51:48.149+0000 7f875db67700 1 -- 172.21.15.136:0/2622812731 wait complete. 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.384 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:48.669 DEBUG:teuthology.orchestra.run.smithi136:osd.1> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.1.service 2024-06-06T06:51:48.671 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/vg_nvme/lv_4... 2024-06-06T06:51:48.672 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-06T06:51:48.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:51:48.917 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:51:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:49 smithi136 ceph-mon[35901]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:49.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:49 smithi136 ceph-mon[35901]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:49 smithi170 ceph-mon[36108]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:49 smithi170 ceph-mon[36108]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:49 smithi073 ceph-mon[32303]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:49.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:49 smithi073 ceph-mon[32303]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-06T06:51:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T06:51:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T06:51:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:50.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:50 smithi170 ceph-mon[36108]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00902813 s, 1.2 GB/s 2024-06-06T06:51:50.689 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:50 smithi073 ceph-mon[32303]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: from='osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: osdmap e11: 2 total, 1 up, 2 in 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:50.777 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-mon[35901]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:51:51.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1[40429]: 2024-06-06T06:51:50.774+0000 7fb083b04700 -1 osd.1 0 waiting for initial osdmap 2024-06-06T06:51:51.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:51:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1[40429]: 2024-06-06T06:51:50.778+0000 7fb07b0ee700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:51:51.315 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_4 2024-06-06T06:51:51.547 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:51:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:51 smithi136 ceph-mon[35901]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:51:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:51 smithi136 ceph-mon[35901]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T06:51:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:51 smithi136 ceph-mon[35901]: from='osd.1 ' entity='osd.1' 2024-06-06T06:51:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:51 smithi170 ceph-mon[36108]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:51:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:51 smithi170 ceph-mon[36108]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T06:51:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:51 smithi170 ceph-mon[36108]: from='osd.1 ' entity='osd.1' 2024-06-06T06:51:51.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:51 smithi073 ceph-mon[32303]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:51:51.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:51 smithi073 ceph-mon[32303]: osdmap e12: 2 total, 1 up, 2 in 2024-06-06T06:51:51.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:51.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:51 smithi073 ceph-mon[32303]: from='osd.1 ' entity='osd.1' 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.585 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] boot 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T06:51:52.586 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:52.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] boot 2024-06-06T06:51:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T06:51:52.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:52.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:51:52.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:51:52.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: osd.1 [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] boot 2024-06-06T06:51:52.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: osdmap e13: 2 total, 2 up, 2 in 2024-06-06T06:51:52.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 -- 172.21.15.170:0/341708809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 msgr2=0x7fe19c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/341708809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c101810 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe190009990 tx=0x7fe190009ca0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 -- 172.21.15.170:0/341708809 shutdown_connections 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/341708809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c101810 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/341708809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe19c102f90 0x7fe19c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/341708809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 -- 172.21.15.170:0/341708809 >> 172.21.15.170:0/341708809 conn(0x7fe19c0fc990 msgr2=0x7fe19c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:51:53.112 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 -- 172.21.15.170:0/341708809 shutdown_connections 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.108+0000 7fe1a3c47700 1 -- 172.21.15.170:0/341708809 wait complete. 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 Processor -- start 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 -- start start 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe19c102f90 0x7fe19c19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10cf60 con 0x7fe19c1025f0 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10cde0 con 0x7fe19c102f90 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a3c47700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe19c10d0e0 con 0x7fe19c1013f0 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a19e3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:53.113 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a19e3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:40240/0 (socket says 172.21.15.170:40240) 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.109+0000 7fe1a19e3700 1 -- 172.21.15.170:0/3075387325 learned_addr learned my addr 172.21.15.170:0/3075387325 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 -- 172.21.15.170:0/3075387325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe19c102f90 msgr2=0x7fe19c19fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe19c102f90 0x7fe19c19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 -- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 msgr2=0x7fe19c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a11e2700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 -- 172.21.15.170:0/3075387325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe19c1a0400 con 0x7fe19c1013f0 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a11e2700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:51:53.114 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a19e3700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1900047b0 tx=0x7fe190009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190026070 con 0x7fe19c1013f0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe190004810 con 0x7fe19c1013f0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe19c1a0690 con 0x7fe19c1013f0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190005390 con 0x7fe19c1013f0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.110+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe19c1a0bc0 con 0x7fe19c1013f0 2024-06-06T06:51:53.115 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.111+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe19c069180 con 0x7fe19c1013f0 2024-06-06T06:51:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.112+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fe190029070 con 0x7fe19c1013f0 2024-06-06T06:51:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.113+0000 7fe18effd700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 0x7fe1880a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:51:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.113+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7fe1900ce850 con 0x7fe19c1013f0 2024-06-06T06:51:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.113+0000 7fe1a11e2700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 0x7fe1880a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:51:53.116 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.114+0000 7fe1a11e2700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 0x7fe1880a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe198007900 tx=0x7fe198008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:51:53.117 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.116+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe190030020 con 0x7fe19c1013f0 2024-06-06T06:51:53.322 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:51:53.322+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fe19c06b2d0 con 0x7fe1880a2c10 2024-06-06T06:51:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:53 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi136 2024-06-06T06:51:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:53 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-06T06:51:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:53 smithi170 ceph-mon[36108]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T06:51:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:53 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi136 2024-06-06T06:51:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:53 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-06T06:51:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:53 smithi073 ceph-mon[32303]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T06:51:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:53 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi136 2024-06-06T06:51:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:53 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 17088M 2024-06-06T06:51:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:53 smithi136 ceph-mon[35901]: osdmap e14: 2 total, 2 up, 2 in 2024-06-06T06:51:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:54 smithi170 ceph-mon[36108]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:54 smithi170 ceph-mon[36108]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:54 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:54 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:54 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:54 smithi073 ceph-mon[32303]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:54 smithi073 ceph-mon[32303]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:54 smithi136 ceph-mon[35901]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:54.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:54 smithi136 ceph-mon[35901]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:51:54.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:54 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:51:54.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:54 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:51:54.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:54 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/164882446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]': finished 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T06:51:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:51:56.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/164882446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]': finished 2024-06-06T06:51:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T06:51:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:51:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/164882446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]: dispatch 2024-06-06T06:51:56.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3443c257-8d23-4b70-ab00-5d3add9a8b3d"}]': finished 2024-06-06T06:51:56.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: osdmap e15: 3 total, 2 up, 3 in 2024-06-06T06:51:56.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:51:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:57 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/3791915047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:57 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/3791915047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:57 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/3791915047' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:51:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:51:58 smithi170 ceph-mon[36108]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:51:58 smithi073 ceph-mon[32303]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:51:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:51:58 smithi136 ceph-mon[35901]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:00 smithi170 ceph-mon[36108]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:00 smithi073 ceph-mon[32303]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:00 smithi136 ceph-mon[35901]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:02.433 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:02 smithi170 ceph-mon[36108]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:02.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:02 smithi073 ceph-mon[32303]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:02 smithi136 ceph-mon[35901]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:03 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T06:52:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:03 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:03.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:03 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T06:52:03.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:03 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:03 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-06T06:52:03.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:03 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:04.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:04 smithi170 ceph-mon[36108]: Deploying daemon osd.2 on smithi170 2024-06-06T06:52:04.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:04 smithi170 ceph-mon[36108]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:04 smithi073 ceph-mon[32303]: Deploying daemon osd.2 on smithi170 2024-06-06T06:52:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:04 smithi073 ceph-mon[32303]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:04 smithi136 ceph-mon[35901]: Deploying daemon osd.2 on smithi170 2024-06-06T06:52:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:04 smithi136 ceph-mon[35901]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:06.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:06 smithi170 ceph-mon[36108]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:06.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:06 smithi073 ceph-mon[32303]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:06 smithi136 ceph-mon[35901]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:07 smithi170 ceph-mon[36108]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:07.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:07 smithi073 ceph-mon[32303]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:07 smithi136 ceph-mon[35901]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:09.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:10.282 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:10 smithi170 ceph-mon[36108]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:10 smithi073 ceph-mon[32303]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:10 smithi136 ceph-mon[35901]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:11.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.687+0000 7fe18effd700 1 -- 172.21.15.170:0/3075387325 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fe19c06b2d0 con 0x7fe1880a2c10 2024-06-06T06:52:11.689 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 msgr2=0x7fe1880a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 0x7fe1880a50d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe198007900 tx=0x7fe198008040 comp rx=0 tx=0).stop 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 msgr2=0x7fe19c1987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe1900047b0 tx=0x7fe190009ee0 comp rx=0 tx=0).stop 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 shutdown_connections 2024-06-06T06:52:11.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe1880a2c10 0x7fe1880a50d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe19c1013f0 0x7fe19c1987a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe19c102f90 0x7fe19c19fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 --2- 172.21.15.170:0/3075387325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe19c1025f0 0x7fe19c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 >> 172.21.15.170:0/3075387325 conn(0x7fe19c0fc990 msgr2=0x7fe19c06abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 shutdown_connections 2024-06-06T06:52:11.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:52:11.690+0000 7fe1a3c47700 1 -- 172.21.15.170:0/3075387325 wait complete. 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.275 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:12.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:12.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:12.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:12.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:12.301 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.2.service 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:12.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:12.305 INFO:tasks.cephadm:Deploying osd.3 on smithi073 with /dev/vg_nvme/lv_3... 2024-06-06T06:52:12.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T06:52:12.550 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:52:13.010 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:12 smithi073 ceph-mon[32303]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.010 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:12 smithi073 ceph-mon[32303]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:12 smithi170 ceph-mon[36108]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.130 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:12 smithi170 ceph-mon[36108]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:12 smithi136 ceph-mon[35901]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:12 smithi136 ceph-mon[35901]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-06T06:52:13.958 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T06:52:13.959 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:13.959 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T06:52:13.959 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:13.959 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:13.959 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:13 smithi073 ceph-mon[32303]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:13 smithi170 ceph-mon[36108]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:14.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-06T06:52:14.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: from='osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:14.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: osdmap e16: 3 total, 2 up, 3 in 2024-06-06T06:52:14.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:14.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:52:14.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:13 smithi136 ceph-mon[35901]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113922 s, 920 MB/s 2024-06-06T06:52:14.582 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-06-06T06:52:14.847 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2[41209]: 2024-06-06T06:52:14.446+0000 7f4b791ee700 -1 osd.2 0 waiting for initial osdmap 2024-06-06T06:52:14.847 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2[41209]: 2024-06-06T06:52:14.450+0000 7f4b70fd9700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:52:15.270 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:14 smithi073 ceph-mon[32303]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:14 smithi073 ceph-mon[32303]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:14 smithi073 ceph-mon[32303]: from='osd.2 ' entity='osd.2' 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-mon[36108]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-mon[36108]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.271 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:14 smithi170 ceph-mon[36108]: from='osd.2 ' entity='osd.2' 2024-06-06T06:52:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:14 smithi136 ceph-mon[35901]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:52:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:14 smithi136 ceph-mon[35901]: osdmap e17: 3 total, 2 up, 3 in 2024-06-06T06:52:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:15.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:14 smithi136 ceph-mon[35901]: from='osd.2 ' entity='osd.2' 2024-06-06T06:52:15.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_3 2024-06-06T06:52:15.557 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] boot 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T06:52:16.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:16.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] boot 2024-06-06T06:52:16.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T06:52:16.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:16.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:52:16.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:52:16.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: osd.2 [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] boot 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: osdmap e18: 3 total, 3 up, 3 in 2024-06-06T06:52:16.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 -- 172.21.15.73:0/1263707792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102600 msgr2=0x7fdaa4102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 --2- 172.21.15.73:0/1263707792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102600 0x7fdaa4102a60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00099c0 tx=0x7fdaa0009cd0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 -- 172.21.15.73:0/1263707792 shutdown_connections 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 --2- 172.21.15.73:0/1263707792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 0x7fdaa4109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 --2- 172.21.15.73:0/1263707792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4101400 0x7fdaa4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 --2- 172.21.15.73:0/1263707792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102600 0x7fdaa4102a60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 -- 172.21.15.73:0/1263707792 >> 172.21.15.73:0/1263707792 conn(0x7fdaa40fc9c0 msgr2=0x7fdaa40fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:52:17.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 -- 172.21.15.73:0/1263707792 shutdown_connections 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.169+0000 7fdaab582700 1 -- 172.21.15.73:0/1263707792 wait complete. 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 Processor -- start 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 -- start start 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 0x7fdaa41039e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 0x7fdaa4103f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410cea0 con 0x7fdaa4101400 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410cd20 con 0x7fdaa4102600 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaab582700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa410d020 con 0x7fdaa4102fa0 2024-06-06T06:52:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaa931e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaa931e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42384/0 (socket says 172.21.15.73:42384) 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaa931e700 1 -- 172.21.15.73:0/3246275543 learned_addr learned my addr 172.21.15.73:0/3246275543 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.170+0000 7fdaa8b1d700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 0x7fdaa41039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa9b1f700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 0x7fdaa4103f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 -- 172.21.15.73:0/3246275543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 msgr2=0x7fdaa4103f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 0x7fdaa4103f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 -- 172.21.15.73:0/3246275543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 msgr2=0x7fdaa41039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 0x7fdaa41039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 -- 172.21.15.73:0/3246275543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa41047d0 con 0x7fdaa4101400 2024-06-06T06:52:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa8b1d700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 0x7fdaa41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaa931e700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c970 tx=0x7fda9800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98004220 con 0x7fdaa4101400 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa41a2840 con 0x7fdaa4101400 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda98007d50 con 0x7fdaa4101400 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda98003d70 con 0x7fdaa4101400 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.171+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41a2d80 con 0x7fdaa4101400 2024-06-06T06:52:17.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.172+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa4069180 con 0x7fdaa4101400 2024-06-06T06:52:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.173+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fda980078a0 con 0x7fdaa4101400 2024-06-06T06:52:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.174+0000 7fda967fc700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 0x7fda900a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.174+0000 7fdaa8b1d700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 0x7fda900a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.174+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 2851+0+0 (secure 0 0 0) 0x7fda980cd9d0 con 0x7fdaa4101400 2024-06-06T06:52:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.174+0000 7fdaa8b1d700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 0x7fda900a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00038f0 tx=0x7fdaa0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:52:17.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.178+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fda98014030 con 0x7fdaa4101400 2024-06-06T06:52:17.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:16 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi170 2024-06-06T06:52:17.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:16 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 17088M 2024-06-06T06:52:17.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:16 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi170 2024-06-06T06:52:17.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:16 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 17088M 2024-06-06T06:52:17.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:16 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi170 2024-06-06T06:52:17.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:16 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 17088M 2024-06-06T06:52:17.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:17.385+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fdaa406b2f0 con 0x7fda900a2b90 2024-06-06T06:52:18.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T06:52:18.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:18.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:18.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:18.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:18.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:17 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:18.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T06:52:18.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:18.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:18.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:17 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: osdmap e19: 3 total, 3 up, 3 in 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: pgmap v82: 0 pgs: ; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:18.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:17 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:19.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:18 smithi073 ceph-mon[32303]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:19.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:19.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:18 smithi073 ceph-mon[32303]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T06:52:19.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:18 smithi136 ceph-mon[35901]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:18 smithi136 ceph-mon[35901]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T06:52:19.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:18 smithi170 ceph-mon[36108]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:18 smithi170 ceph-mon[36108]: osdmap e20: 3 total, 3 up, 3 in 2024-06-06T06:52:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-06T06:52:19.563 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52098]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:19.563 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52098]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:19.563 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52098]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:19.563 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52098]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:19.961 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52107]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:19.961 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52107]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:19.961 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52107]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:19.961 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 06:52:19 smithi073 sudo[52107]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:19.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:19.966+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fda98097ed0 con 0x7fdaa4101400 2024-06-06T06:52:20.093 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:19 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:20.093 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:19 smithi136 ceph-mon[35901]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T06:52:20.093 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:19 smithi136 ceph-mon[35901]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:20.093 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:19 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:20.093 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:19 smithi136 sudo[44524]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:20.093 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:19 smithi136 sudo[44524]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:20.093 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:19 smithi136 sudo[44524]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:20.093 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:19 smithi136 sudo[44524]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:20.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:19 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:19 smithi073 ceph-mon[32303]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T06:52:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:19 smithi073 ceph-mon[32303]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:19 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:19 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-06T06:52:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:19 smithi170 ceph-mon[36108]: osdmap e21: 3 total, 3 up, 3 in 2024-06-06T06:52:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:19 smithi170 ceph-mon[36108]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:19 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:20.551 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:20 smithi136 sudo[44527]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:20.552 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:20 smithi136 sudo[44527]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:20.552 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:20 smithi136 sudo[44527]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:20.552 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 06:52:20 smithi136 sudo[44527]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:20.615 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45065]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:20.615 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45065]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:20.615 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45065]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:20.615 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45065]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:21.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:20 smithi073 sudo[52190]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:20 smithi073 sudo[52190]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:20 smithi073 sudo[52190]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:20 smithi073 sudo[52190]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:21.097 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45068]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:21.097 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45068]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:21.097 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45068]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:21.097 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 06:52:20 smithi170 sudo[45068]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]: dispatch 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]': finished 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.416 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45071]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:21.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45071]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:21.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45071]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:21.417 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45071]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 sudo[52238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 sudo[52238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 sudo[52238]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 sudo[52238]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]: dispatch 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]': finished 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-06T06:52:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: osdmap e22: 3 total, 3 up, 3 in 2024-06-06T06:52:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]: dispatch 2024-06-06T06:52:21.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/90995659' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e503419a-b7b8-4a0e-ab38-9459dbf2fdf9"}]': finished 2024-06-06T06:52:21.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: osdmap e23: 4 total, 3 up, 4 in 2024-06-06T06:52:21.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:21.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45074]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45074]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45074]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:21 smithi170 sudo[45074]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:22.148 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 sudo[44530]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-06T06:52:22.148 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 sudo[44530]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:22.148 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 sudo[44530]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:22.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:21 smithi136 sudo[44530]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:22.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 sudo[44533]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-06T06:52:22.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 sudo[44533]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-06T06:52:22.149 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 sudo[44533]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/808100111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:22 smithi073 ceph-mon[32303]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 sudo[44533]: pam_unix(sudo:session): session closed for user root 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/808100111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:22 smithi136 ceph-mon[35901]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/808100111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:22.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:22.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-06T06:52:22.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T06:52:22.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T06:52:22.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T06:52:22.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:22 smithi170 ceph-mon[36108]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-06T06:52:24.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:24 smithi136 ceph-mon[35901]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:24 smithi170 ceph-mon[36108]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:24.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:24 smithi073 ceph-mon[32303]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:26.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:26 smithi136 ceph-mon[35901]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:26 smithi170 ceph-mon[36108]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:26.680 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:26 smithi073 ceph-mon[32303]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:28.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:28 smithi073 ceph-mon[32303]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:28.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:28 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T06:52:28.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:28 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:28 smithi170 ceph-mon[36108]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:28 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T06:52:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:28 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:28.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:28 smithi136 ceph-mon[35901]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:28.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:28 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-06T06:52:28.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:28 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:29 smithi073 ceph-mon[32303]: Deploying daemon osd.3 on smithi073 2024-06-06T06:52:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:29 smithi170 ceph-mon[36108]: Deploying daemon osd.3 on smithi073 2024-06-06T06:52:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:29 smithi136 ceph-mon[35901]: Deploying daemon osd.3 on smithi073 2024-06-06T06:52:30.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:30 smithi170 ceph-mon[36108]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:30.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:30 smithi073 ceph-mon[32303]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:30.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:30 smithi136 ceph-mon[35901]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:32.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:32 smithi073 ceph-mon[32303]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:32.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:32 smithi170 ceph-mon[36108]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:32 smithi136 ceph-mon[35901]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:34.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:34 smithi073 ceph-mon[32303]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:34.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:34 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:34 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:34 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:34 smithi170 ceph-mon[36108]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:34 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:34 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:34 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:34.603 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:34 smithi136 ceph-mon[35901]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:34.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:34 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:34 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:34.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:34 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:36 smithi170 ceph-mon[36108]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:36.644 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:36 smithi073 ceph-mon[32303]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:36 smithi136 ceph-mon[35901]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:37.342 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 3 on host 'smithi073' 2024-06-06T06:52:37.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.341+0000 7fda967fc700 1 -- 172.21.15.73:0/3246275543 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdaa406b2f0 con 0x7fda900a2b90 2024-06-06T06:52:37.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 msgr2=0x7fda900a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:37.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 0x7fda900a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdaa00038f0 tx=0x7fdaa0005fd0 comp rx=0 tx=0).stop 2024-06-06T06:52:37.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 msgr2=0x7fdaa4105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:37.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda9800c970 tx=0x7fda9800cd30 comp rx=0 tx=0).stop 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 shutdown_connections 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fda900a2b90 0x7fda900a5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdaa4102fa0 0x7fdaa4103f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4101400 0x7fdaa4105390 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 --2- 172.21.15.73:0/3246275543 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdaa4102600 0x7fdaa41039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 >> 172.21.15.73:0/3246275543 conn(0x7fdaa40fc9c0 msgr2=0x7fdaa406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 shutdown_connections 2024-06-06T06:52:37.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:52:37.343+0000 7fdaab582700 1 -- 172.21.15.73:0/3246275543 wait complete. 2024-06-06T06:52:37.990 DEBUG:teuthology.orchestra.run.smithi073:osd.3> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.3.service 2024-06-06T06:52:37.992 INFO:tasks.cephadm:Deploying osd.4 on smithi136 with /dev/vg_nvme/lv_3... 2024-06-06T06:52:37.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T06:52:38.238 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:38 smithi073 ceph-mon[32303]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:38 smithi170 ceph-mon[36108]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T06:52:38.627 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:38.628 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.628 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.628 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.628 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:38.628 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:38 smithi136 ceph-mon[35901]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-06T06:52:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:39 smithi136 ceph-mon[35901]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T06:52:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:39 smithi136 ceph-mon[35901]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T06:52:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:39 smithi136 ceph-mon[35901]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:52:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:39 smithi136 ceph-mon[35901]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:39 smithi073 ceph-mon[32303]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T06:52:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:39 smithi073 ceph-mon[32303]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T06:52:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:39 smithi073 ceph-mon[32303]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:52:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:39 smithi073 ceph-mon[32303]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00959307 s, 1.1 GB/s 2024-06-06T06:52:40.064 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-06T06:52:40.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:39 smithi170 ceph-mon[36108]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-06T06:52:40.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:39 smithi170 ceph-mon[36108]: osdmap e24: 4 total, 3 up, 4 in 2024-06-06T06:52:40.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:39 smithi170 ceph-mon[36108]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:52:40.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:39 smithi170 ceph-mon[36108]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 96 MiB used, 268 GiB / 268 GiB avail 2024-06-06T06:52:40.475 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3[54187]: 2024-06-06T06:52:40.049+0000 7f3060926700 -1 osd.3 0 waiting for initial osdmap 2024-06-06T06:52:40.475 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3[54187]: 2024-06-06T06:52:40.056+0000 7f3058711700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:52:40.704 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] boot 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T06:52:40.705 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.731 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_3 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] boot 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T06:52:40.759 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:40.965 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: osdmap e25: 4 total, 3 up, 4 in 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: from='osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250]' entity='osd.3' 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: osd.3 [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] boot 2024-06-06T06:52:41.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: osdmap e26: 4 total, 4 up, 4 in 2024-06-06T06:52:41.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T06:52:41.752 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi073 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 8544M 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:41.753 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:41.754 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:41.754 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:41.754 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:41.754 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:41.754 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:41 smithi136 ceph-mon[35901]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi073 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 8544M 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:42.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:41 smithi073 ceph-mon[32303]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi073 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 8544M 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:42.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:41 smithi170 ceph-mon[36108]: osdmap e27: 4 total, 4 up, 4 in 2024-06-06T06:52:42.508 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.506+0000 7fd854a6d700 1 -- 172.21.15.136:0/3223141498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850102600 msgr2=0x7fd850102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.506+0000 7fd854a6d700 1 --2- 172.21.15.136:0/3223141498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850102600 0x7fd850102a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd8440099c0 tx=0x7fd844009cd0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 -- 172.21.15.136:0/3223141498 shutdown_connections 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 --2- 172.21.15.136:0/3223141498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 0x7fd850109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 --2- 172.21.15.136:0/3223141498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850101400 0x7fd850101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 --2- 172.21.15.136:0/3223141498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850102600 0x7fd850102a60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 -- 172.21.15.136:0/3223141498 >> 172.21.15.136:0/3223141498 conn(0x7fd8500fc9c0 msgr2=0x7fd8500fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 -- 172.21.15.136:0/3223141498 shutdown_connections 2024-06-06T06:52:42.509 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 -- 172.21.15.136:0/3223141498 wait complete. 2024-06-06T06:52:42.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.507+0000 7fd854a6d700 1 Processor -- start 2024-06-06T06:52:42.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 -- start start 2024-06-06T06:52:42.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:42.511 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850102600 0x7fd85019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 0x7fd85019fab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd85010cef0 con 0x7fd850102600 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd85010cd70 con 0x7fd850101400 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd854a6d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd85010d070 con 0x7fd850102fa0 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:50534/0 (socket says 172.21.15.136:50534) 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 -- 172.21.15.136:0/1642424498 learned_addr learned my addr 172.21.15.136:0/1642424498 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84ed9d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 0x7fd85019fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84dd9b700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850102600 0x7fd85019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 -- 172.21.15.136:0/1642424498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 msgr2=0x7fd85019fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 0x7fd85019fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 -- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850102600 msgr2=0x7fd85019d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:52:42.512 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850102600 0x7fd85019d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 -- 172.21.15.136:0/1642424498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8501a0380 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.508+0000 7fd84e59c700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c970 tx=0x7fd84000cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.509+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840004220 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.509+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd840007d50 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.509+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd840003d00 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.509+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8501a0670 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.509+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8501a0c30 con 0x7fd850101400 2024-06-06T06:52:42.513 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.510+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd850069180 con 0x7fd850101400 2024-06-06T06:52:42.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.511+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd8400078a0 con 0x7fd850101400 2024-06-06T06:52:42.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.511+0000 7fd83f7fe700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:52:42.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.512+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7fd8400cddc0 con 0x7fd850101400 2024-06-06T06:52:42.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.512+0000 7fd84dd9b700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:52:42.514 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.512+0000 7fd84dd9b700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 0x7fd8380a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd85006bcc0 tx=0x7fd844005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:52:42.517 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.515+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd840014030 con 0x7fd850101400 2024-06-06T06:52:42.724 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:52:42.722+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd8500628d0 con 0x7fd8380a2c20 2024-06-06T06:52:43.986 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T06:52:43.987 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:43.987 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:43.987 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:43.987 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:43.987 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:43 smithi136 ceph-mon[35901]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T06:52:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:44.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:44.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:43 smithi073 ceph-mon[32303]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: osdmap e28: 4 total, 4 up, 4 in 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: from='client.24200 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:44.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:43 smithi170 ceph-mon[36108]: pgmap v104: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:44 smithi136 ceph-mon[35901]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T06:52:44.925 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:44 smithi073 ceph-mon[32303]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T06:52:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:44 smithi170 ceph-mon[36108]: osdmap e29: 4 total, 4 up, 4 in 2024-06-06T06:52:46.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/3279832994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]': finished 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T06:52:46.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:52:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T06:52:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/3279832994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]': finished 2024-06-06T06:52:46.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T06:52:46.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: osdmap e30: 4 total, 4 up, 4 in 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: pgmap v107: 1 pgs: 1 remapped+peering; 577 KiB data, 153 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/3279832994' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]: dispatch 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f622b141-b741-4f89-9c8c-39615ff7919f"}]': finished 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: osdmap e31: 5 total, 4 up, 5 in 2024-06-06T06:52:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:52:47.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/1584239784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:47.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:46 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/1584239784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:47.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:46 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/1584239784' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:52:48.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:47 smithi073 ceph-mon[32303]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:48.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:47 smithi136 ceph-mon[35901]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:48.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:47 smithi170 ceph-mon[36108]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:52:50.461 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:50 smithi136 ceph-mon[35901]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:52:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:50 smithi170 ceph-mon[36108]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:52:50.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:50 smithi073 ceph-mon[32303]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:52:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:52 smithi170 ceph-mon[36108]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T06:52:52.624 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:52 smithi136 ceph-mon[35901]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T06:52:52.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:52 smithi073 ceph-mon[32303]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-06-06T06:52:53.543 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T06:52:53.543 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T06:52:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-06T06:52:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:52:54.506 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:54 smithi136 ceph-mon[35901]: Deploying daemon osd.4 on smithi136 2024-06-06T06:52:54.506 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:54 smithi136 ceph-mon[35901]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T06:52:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:54 smithi170 ceph-mon[36108]: Deploying daemon osd.4 on smithi136 2024-06-06T06:52:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:54 smithi170 ceph-mon[36108]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T06:52:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:54 smithi073 ceph-mon[32303]: Deploying daemon osd.4 on smithi136 2024-06-06T06:52:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:54 smithi073 ceph-mon[32303]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-06-06T06:52:56.466 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:56 smithi136 ceph-mon[35901]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:52:56.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:56 smithi073 ceph-mon[32303]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:52:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:56 smithi170 ceph-mon[36108]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:52:58.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:58 smithi136 ceph-mon[35901]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-06T06:52:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:52:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:58 smithi170 ceph-mon[36108]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-06T06:52:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:52:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:58 smithi073 ceph-mon[32303]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-06-06T06:52:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:52:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:52:58.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:52:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:00.572 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:00 smithi136 ceph-mon[35901]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T06:53:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:00 smithi170 ceph-mon[36108]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T06:53:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:00 smithi073 ceph-mon[32303]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-06T06:53:01.837 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.836+0000 7fd83f7fe700 1 -- 172.21.15.136:0/1642424498 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd8500628d0 con 0x7fd8380a2c20 2024-06-06T06:53:01.837 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 4 on host 'smithi136' 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 msgr2=0x7fd8380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 0x7fd8380a50e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd85006bcc0 tx=0x7fd844005fd0 comp rx=0 tx=0).stop 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 msgr2=0x7fd85006b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd84000c970 tx=0x7fd84000cc80 comp rx=0 tx=0).stop 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 shutdown_connections 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd8380a2c20 0x7fd8380a50e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd850102fa0 0x7fd85019fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd850101400 0x7fd85006b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:01.840 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 --2- 172.21.15.136:0/1642424498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850102600 0x7fd85019d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:01.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 >> 172.21.15.136:0/1642424498 conn(0x7fd8500fc9c0 msgr2=0x7fd850105a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:01.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 shutdown_connections 2024-06-06T06:53:01.841 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:01.839+0000 7fd854a6d700 1 -- 172.21.15.136:0/1642424498 wait complete. 2024-06-06T06:53:02.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:02.203 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.203 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.203 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:02.204 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:02.427 DEBUG:teuthology.orchestra.run.smithi136:osd.4> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.4.service 2024-06-06T06:53:02.429 INFO:tasks.cephadm:Deploying osd.5 on smithi170 with /dev/vg_nvme/lv_3... 2024-06-06T06:53:02.429 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-06T06:53:02.675 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:53:03.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:02 smithi136 ceph-mon[35901]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:03.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:02 smithi136 ceph-mon[35901]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:03.052 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:02 smithi170 ceph-mon[36108]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:03.052 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:02 smithi170 ceph-mon[36108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:02 smithi073 ceph-mon[32303]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:03.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:02 smithi073 ceph-mon[32303]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-mon[35901]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:04.052 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T06:53:03.872+0000 7f6651689700 -1 osd.4 0 waiting for initial osdmap 2024-06-06T06:53:04.053 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 06:53:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T06:53:03.879+0000 7f664d47c700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.075 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:03 smithi170 ceph-mon[36108]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:04.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-06T06:53:04.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: from='osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: osdmap e32: 5 total, 4 up, 5 in 2024-06-06T06:53:04.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:04.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:53:04.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:03 smithi073 ceph-mon[32303]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:04.652 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-06T06:53:04.652 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-06T06:53:04.652 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-06-06T06:53:04.653 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-06-06T06:53:04.653 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00909629 s, 1.2 GB/s 2024-06-06T06:53:04.653 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-06-06T06:53:04.972 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:04 smithi170 ceph-mon[36108]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:53:04.972 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:04 smithi170 ceph-mon[36108]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T06:53:04.973 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:04.973 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.173 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:04 smithi136 ceph-mon[35901]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:53:05.173 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:04 smithi136 ceph-mon[35901]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T06:53:05.173 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.173 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:04 smithi073 ceph-mon[32303]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:53:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:04 smithi073 ceph-mon[32303]: osdmap e33: 5 total, 4 up, 5 in 2024-06-06T06:53:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.293 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_3 2024-06-06T06:53:05.532 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:53:05.934 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] boot 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi136 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:05.935 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:05.936 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] boot 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi136 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:06.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-06T06:53:06.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:06.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:06.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: osd.4 [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] boot 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: osdmap e34: 5 total, 5 up, 5 in 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi136 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:06.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 8544M 2024-06-06T06:53:06.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:06.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:06.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:06.927 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:06 smithi170 ceph-mon[36108]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T06:53:07.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 -- 172.21.15.170:0/4256635150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 msgr2=0x7fb01c10a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:07.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 --2- 172.21.15.170:0/4256635150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c10a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb01400b0a0 tx=0x7fb01400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 -- 172.21.15.170:0/4256635150 shutdown_connections 2024-06-06T06:53:07.139 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 --2- 172.21.15.170:0/4256635150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c10a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 --2- 172.21.15.170:0/4256635150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1024f0 0x7fb01c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 --2- 172.21.15.170:0/4256635150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1036f0 0x7fb01c103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 -- 172.21.15.170:0/4256635150 >> 172.21.15.170:0/4256635150 conn(0x7fb01c0fda90 msgr2=0x7fb01c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 -- 172.21.15.170:0/4256635150 shutdown_connections 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.138+0000 7fb0243bb700 1 -- 172.21.15.170:0/4256635150 wait complete. 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 Processor -- start 2024-06-06T06:53:07.140 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 -- start start 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1024f0 0x7fb01c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1036f0 0x7fb01c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10e010 con 0x7fb01c1024f0 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10de90 con 0x7fb01c1036f0 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb0243bb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb01c10e190 con 0x7fb01c104090 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:51662/0 (socket says 172.21.15.170:51662) 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 -- 172.21.15.170:0/3019288128 learned_addr learned my addr 172.21.15.170:0/3019288128 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:53:07.142 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb021956700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1036f0 0x7fb01c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022157700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1024f0 0x7fb01c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 -- 172.21.15.170:0/3019288128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1036f0 msgr2=0x7fb01c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1036f0 0x7fb01c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 -- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1024f0 msgr2=0x7fb01c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1024f0 0x7fb01c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 -- 172.21.15.170:0/3019288128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb01c1a02e0 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.139+0000 7fb022958700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb014006070 tx=0x7fb014004e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.140+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb014021070 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.140+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb01c1a0570 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.140+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb014007d00 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.140+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb01401d450 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.140+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb01c1a0aa0 con 0x7fb01c104090 2024-06-06T06:53:07.143 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.141+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb01c069180 con 0x7fb01c104090 2024-06-06T06:53:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.142+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fb0140040b0 con 0x7fb01c104090 2024-06-06T06:53:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.142+0000 7fb0137fe700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 0x7fb0080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.142+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7fb01400f020 con 0x7fb01c104090 2024-06-06T06:53:07.144 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.143+0000 7fb022157700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 0x7fb0080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:07.145 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.143+0000 7fb022157700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 0x7fb0080a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:07.148 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.147+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb01402a020 con 0x7fb01c104090 2024-06-06T06:53:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:06 smithi073 ceph-mon[32303]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T06:53:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:06 smithi136 ceph-mon[35901]: osdmap e35: 5 total, 5 up, 5 in 2024-06-06T06:53:07.353 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:07.352+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb01c0628d0 con 0x7fb0080a2c20 2024-06-06T06:53:08.210 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:07 smithi170 ceph-mon[36108]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:08.210 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:08.210 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:08.211 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:08.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:07 smithi073 ceph-mon[32303]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:08.320 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:08.320 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:08.320 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:08.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:07 smithi136 ceph-mon[35901]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:08.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:08.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:08.321 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:09.248 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:08 smithi170 ceph-mon[36108]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:08 smithi073 ceph-mon[32303]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:08 smithi136 ceph-mon[35901]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:09 smithi073 ceph-mon[32303]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:10.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:09 smithi136 ceph-mon[35901]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:09 smithi170 ceph-mon[36108]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/2065894725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]': finished 2024-06-06T06:53:11.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T06:53:11.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:11.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:10 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/986203773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/2065894725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]': finished 2024-06-06T06:53:11.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T06:53:11.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:11.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:10 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/986203773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/2065894725' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]: dispatch 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27b58b5d-313a-46d9-a7f5-e6c72db937c7"}]': finished 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: osdmap e36: 6 total, 5 up, 6 in 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:10 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/986203773' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:12.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:11 smithi073 ceph-mon[32303]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:12.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:11 smithi136 ceph-mon[35901]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:12.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:11 smithi170 ceph-mon[36108]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:14.332 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:14 smithi170 ceph-mon[36108]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:14.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:14 smithi073 ceph-mon[32303]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:14.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:14 smithi136 ceph-mon[35901]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:16.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:16 smithi170 ceph-mon[36108]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:16.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:16 smithi073 ceph-mon[32303]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:16.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:16 smithi136 ceph-mon[35901]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:18 smithi170 ceph-mon[36108]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T06:53:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:18 smithi073 ceph-mon[32303]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T06:53:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:18 smithi136 ceph-mon[35901]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-06T06:53:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:19.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:19 smithi170 ceph-mon[36108]: Deploying daemon osd.5 on smithi170 2024-06-06T06:53:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:19 smithi073 ceph-mon[32303]: Deploying daemon osd.5 on smithi170 2024-06-06T06:53:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:19 smithi136 ceph-mon[35901]: Deploying daemon osd.5 on smithi170 2024-06-06T06:53:20.549 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:20 smithi170 ceph-mon[36108]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:20.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:20 smithi073 ceph-mon[32303]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:20 smithi136 ceph-mon[35901]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:22.468 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:22 smithi170 ceph-mon[36108]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:22 smithi073 ceph-mon[32303]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:22 smithi136 ceph-mon[35901]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:23 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:23 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:23 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:23 smithi170 ceph-mon[36108]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:24.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:23 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:23 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:23 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:24.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:23 smithi073 ceph-mon[32303]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:24.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:23 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:23 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:24.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:23 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:24.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:23 smithi136 ceph-mon[35901]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.671 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 5 on host 'smithi170' 2024-06-06T06:53:26.671 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.669+0000 7fb0137fe700 1 -- 172.21.15.170:0/3019288128 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb01c0628d0 con 0x7fb0080a2c20 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 msgr2=0x7fb0080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 0x7fb0080a50e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fb00c006fd0 tx=0x7fb00c008040 comp rx=0 tx=0).stop 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 msgr2=0x7fb01c19fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb014006070 tx=0x7fb014004e40 comp rx=0 tx=0).stop 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 shutdown_connections 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb0080a2c20 0x7fb0080a50e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb01c104090 0x7fb01c19fc40 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:26.673 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb01c1024f0 0x7fb01c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:26.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 --2- 172.21.15.170:0/3019288128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb01c1036f0 0x7fb01c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:26.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 >> 172.21.15.170:0/3019288128 conn(0x7fb01c0fda90 msgr2=0x7fb01c106a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:26.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 shutdown_connections 2024-06-06T06:53:26.674 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:53:26.672+0000 7fb0243bb700 1 -- 172.21.15.170:0/3019288128 wait complete. 2024-06-06T06:53:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:27.343 DEBUG:teuthology.orchestra.run.smithi170:osd.5> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.5.service 2024-06-06T06:53:27.346 INFO:tasks.cephadm:Deploying osd.6 on smithi073 with /dev/vg_nvme/lv_2... 2024-06-06T06:53:27.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T06:53:27.591 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:27.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:27 smithi073 ceph-mon[32303]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:28.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:28.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:28.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:28.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:28.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:28.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:27 smithi136 ceph-mon[35901]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:28.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-06T06:53:28.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:27 smithi170 ceph-mon[36108]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:28 smithi170 ceph-mon[36108]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T06:53:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:28 smithi170 ceph-mon[36108]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:28 smithi170 ceph-mon[36108]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T06:53:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:28 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:28 smithi170 ceph-mon[36108]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:29.009 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:28 smithi073 ceph-mon[32303]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T06:53:29.009 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:28 smithi073 ceph-mon[32303]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:29.009 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:28 smithi073 ceph-mon[32303]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T06:53:29.009 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:28 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:29.009 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:28 smithi073 ceph-mon[32303]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:29.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:28 smithi136 ceph-mon[35901]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-06T06:53:29.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:28 smithi136 ceph-mon[35901]: from='osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:29.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:28 smithi136 ceph-mon[35901]: osdmap e37: 6 total, 5 up, 6 in 2024-06-06T06:53:29.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:28 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:29.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:28 smithi136 ceph-mon[35901]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:53:29.593 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T06:53:29.593 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T06:53:29.594 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-06-06T06:53:29.594 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-06-06T06:53:29.594 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0176766 s, 593 MB/s 2024-06-06T06:53:29.594 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-06-06T06:53:29.597 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T06:53:29.109+0000 7f6112476700 -1 osd.5 0 waiting for initial osdmap 2024-06-06T06:53:29.597 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T06:53:29.115+0000 7f610a261700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: from='osd.5 ' entity='osd.5' 2024-06-06T06:53:29.922 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:29 smithi073 ceph-mon[32303]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:53:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T06:53:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: from='osd.5 ' entity='osd.5' 2024-06-06T06:53:30.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:29 smithi136 ceph-mon[35901]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: osdmap e38: 6 total, 5 up, 6 in 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: from='osd.5 ' entity='osd.5' 2024-06-06T06:53:30.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:29 smithi170 ceph-mon[36108]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-06T06:53:30.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_2 2024-06-06T06:53:30.553 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:53:30.822 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:53:30.822 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:53:30.822 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] boot 2024-06-06T06:53:30.822 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:30.823 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] boot 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:31.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: osd.5 [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] boot 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: osdmap e39: 6 total, 6 up, 6 in 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:31.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:31.971 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:31 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi170 2024-06-06T06:53:31.971 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:31 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 8544M 2024-06-06T06:53:31.971 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:31 smithi073 ceph-mon[32303]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T06:53:31.971 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:31 smithi073 ceph-mon[32303]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:32.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:31 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi170 2024-06-06T06:53:32.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:31 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 8544M 2024-06-06T06:53:32.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:31 smithi136 ceph-mon[35901]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T06:53:32.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:31 smithi136 ceph-mon[35901]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:32.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:31 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi170 2024-06-06T06:53:32.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:31 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 8544M 2024-06-06T06:53:32.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:31 smithi170 ceph-mon[36108]: osdmap e40: 6 total, 6 up, 6 in 2024-06-06T06:53:32.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:31 smithi170 ceph-mon[36108]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 -- 172.21.15.73:0/1282783222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03681024f0 msgr2=0x7f0368102910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 --2- 172.21.15.73:0/1282783222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03681024f0 0x7f0368102910 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03640099c0 tx=0x7f0364009cd0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 -- 172.21.15.73:0/1282783222 shutdown_connections 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 --2- 172.21.15.73:0/1282783222 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0368104090 0x7f036810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 --2- 172.21.15.73:0/1282783222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03681024f0 0x7f0368102910 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 --2- 172.21.15.73:0/1282783222 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 -- 172.21.15.73:0/1282783222 >> 172.21.15.73:0/1282783222 conn(0x7f03680fda90 msgr2=0x7f03680ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 -- 172.21.15.73:0/1282783222 shutdown_connections 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.204+0000 7f036f912700 1 -- 172.21.15.73:0/1282783222 wait complete. 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 Processor -- start 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 -- start start 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 0x7f03681986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f036810dff0 con 0x7f0368104090 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f036810de70 con 0x7f03681024f0 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036f912700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f036810e170 con 0x7f03681036f0 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:32.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60122/0 (socket says 172.21.15.73:60122) 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 -- 172.21.15.73:0/2120997081 learned_addr learned my addr 172.21.15.73:0/2120997081 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036cead700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036d6ae700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 0x7f03681986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 -- 172.21.15.73:0/2120997081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 msgr2=0x7f0368198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 -- 172.21.15.73:0/2120997081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 msgr2=0x7f03681986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 0x7f03681986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036deaf700 1 -- 172.21.15.73:0/2120997081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03681a0350 con 0x7f0368104090 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036d6ae700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 0x7f03681986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.205+0000 7f036cead700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f036deaf700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f036000add0 tx=0x7f036000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:32.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360017020 con 0x7f0368104090 2024-06-06T06:53:32.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f036000cdf0 con 0x7f0368104090 2024-06-06T06:53:32.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03681a0640 con 0x7f0368104090 2024-06-06T06:53:32.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0360005460 con 0x7f0368104090 2024-06-06T06:53:32.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.206+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03681a0bb0 con 0x7f0368104090 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.207+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0368069180 con 0x7f0368104090 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.208+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0360004ad0 con 0x7f0368104090 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.209+0000 7f035e7fc700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 0x7f03540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.209+0000 7f036d6ae700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 0x7f03540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.209+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f03600cdf50 con 0x7f0368104090 2024-06-06T06:53:32.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.209+0000 7f036d6ae700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 0x7f03540a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0364006010 tx=0x7f0364005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:32.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.212+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0360021020 con 0x7f0368104090 2024-06-06T06:53:32.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:32.420+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f03680628d0 con 0x7f03540a2c20 2024-06-06T06:53:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:32 smithi073 ceph-mon[32303]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T06:53:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:32 smithi073 ceph-mon[32303]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T06:53:32.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:32.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:32.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:32 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:32.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:32 smithi170 ceph-mon[36108]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T06:53:32.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:32 smithi170 ceph-mon[36108]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T06:53:32.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:32 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:32.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:32 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:32.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:32 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:33.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:32 smithi136 ceph-mon[35901]: osdmap e41: 6 total, 6 up, 6 in 2024-06-06T06:53:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:32 smithi136 ceph-mon[35901]: osdmap e42: 6 total, 6 up, 6 in 2024-06-06T06:53:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:32 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:32 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:33.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:32 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:34.390 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:34 smithi073 ceph-mon[32303]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:34.390 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:34 smithi073 ceph-mon[32303]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:34.390 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:34 smithi073 ceph-mon[32303]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T06:53:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:34 smithi170 ceph-mon[36108]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:34 smithi170 ceph-mon[36108]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:34 smithi170 ceph-mon[36108]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T06:53:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:34 smithi136 ceph-mon[35901]: from='client.14355 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:53:34.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:34 smithi136 ceph-mon[35901]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 193 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:34.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:34 smithi136 ceph-mon[35901]: osdmap e43: 6 total, 6 up, 6 in 2024-06-06T06:53:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:35 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]: dispatch 2024-06-06T06:53:35.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:35 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]': finished 2024-06-06T06:53:35.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:35 smithi136 ceph-mon[35901]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T06:53:35.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:35 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]: dispatch 2024-06-06T06:53:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:35 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]': finished 2024-06-06T06:53:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:35 smithi170 ceph-mon[36108]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T06:53:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:35.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:35 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]: dispatch 2024-06-06T06:53:35.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:35 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3293545934' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "de018a90-38be-4bea-b847-951585c33b94"}]': finished 2024-06-06T06:53:35.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:35 smithi073 ceph-mon[32303]: osdmap e44: 7 total, 6 up, 7 in 2024-06-06T06:53:35.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:36 smithi170 ceph-mon[36108]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:36 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/706770134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:36 smithi073 ceph-mon[32303]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:36 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/706770134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:36 smithi136 ceph-mon[35901]: pgmap v146: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:53:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:36 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/706770134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:53:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:38 smithi170 ceph-mon[36108]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-06T06:53:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:38 smithi073 ceph-mon[32303]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-06T06:53:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:38 smithi136 ceph-mon[35901]: pgmap v147: 1 pgs: 1 peering; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-06-06T06:53:40.581 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:40 smithi073 ceph-mon[32303]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-06T06:53:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:40 smithi170 ceph-mon[36108]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-06T06:53:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:40 smithi136 ceph-mon[35901]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-06-06T06:53:42.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:42 smithi073 ceph-mon[32303]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:42 smithi170 ceph-mon[36108]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:42 smithi136 ceph-mon[35901]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:43 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T06:53:43.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:43 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:43 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T06:53:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:43 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:43 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-06T06:53:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:43 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:44 smithi170 ceph-mon[36108]: Deploying daemon osd.6 on smithi073 2024-06-06T06:53:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:44 smithi170 ceph-mon[36108]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:44 smithi073 ceph-mon[32303]: Deploying daemon osd.6 on smithi073 2024-06-06T06:53:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:44 smithi073 ceph-mon[32303]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:44 smithi136 ceph-mon[35901]: Deploying daemon osd.6 on smithi073 2024-06-06T06:53:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:44 smithi136 ceph-mon[35901]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:46.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:46 smithi136 ceph-mon[35901]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:46.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:46 smithi170 ceph-mon[36108]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:46.726 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:46 smithi073 ceph-mon[32303]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:48 smithi073 ceph-mon[32303]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:48 smithi170 ceph-mon[36108]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:48 smithi136 ceph-mon[35901]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:50 smithi073 ceph-mon[32303]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:50.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:50 smithi136 ceph-mon[35901]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:50 smithi170 ceph-mon[36108]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:52.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:52 smithi073 ceph-mon[32303]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:52.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:52 smithi136 ceph-mon[35901]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:52 smithi170 ceph-mon[36108]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:53.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.235+0000 7f035e7fc700 1 -- 172.21.15.73:0/2120997081 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f03680628d0 con 0x7f03540a2c20 2024-06-06T06:53:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 6 on host 'smithi073' 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.237+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 msgr2=0x7f03540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.237+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 0x7f03540a50e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0364006010 tx=0x7f0364005c40 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.237+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 msgr2=0x7f036819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.237+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f036000add0 tx=0x7f036000c330 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 shutdown_connections 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0368104090 0x7f036819fcb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03681024f0 0x7f03681986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03540a2c20 0x7f03540a50e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 --2- 172.21.15.73:0/2120997081 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03681036f0 0x7f0368198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 >> 172.21.15.73:0/2120997081 conn(0x7f03680fda90 msgr2=0x7f0368106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 shutdown_connections 2024-06-06T06:53:53.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:53:53.238+0000 7f036f912700 1 -- 172.21.15.73:0/2120997081 wait complete. 2024-06-06T06:53:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:53.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:53.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:53:53.937 DEBUG:teuthology.orchestra.run.smithi073:osd.6> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.6.service 2024-06-06T06:53:53.940 INFO:tasks.cephadm:Deploying osd.7 on smithi136 with /dev/vg_nvme/lv_2... 2024-06-06T06:53:53.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T06:53:54.185 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:53:54.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-mon[32303]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T06:53:54.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-mon[32303]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T06:53:54.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-mon[32303]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:53:54.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:54.594 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:54 smithi136 ceph-mon[35901]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T06:53:54.594 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:54 smithi136 ceph-mon[35901]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T06:53:54.595 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:54 smithi136 ceph-mon[35901]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:53:54.595 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:54 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:54 smithi170 ceph-mon[36108]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-06T06:53:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:54 smithi170 ceph-mon[36108]: osdmap e45: 7 total, 6 up, 7 in 2024-06-06T06:53:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:54 smithi170 ceph-mon[36108]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:53:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:54 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.139 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T06:53:54.824+0000 7f95dd901700 -1 osd.6 0 waiting for initial osdmap 2024-06-06T06:53:55.139 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 06:53:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T06:53:54.833+0000 7f95d4eeb700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' 2024-06-06T06:53:55.481 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:55 smithi136 ceph-mon[35901]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' 2024-06-06T06:53:55.765 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:55 smithi073 ceph-mon[32303]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: osdmap e46: 7 total, 6 up, 7 in 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: from='osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786]' entity='osd.6' 2024-06-06T06:53:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:55 smithi170 ceph-mon[36108]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 189 MiB used, 536 GiB / 536 GiB avail 2024-06-06T06:53:56.067 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T06:53:56.068 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T06:53:56.068 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-06T06:53:56.068 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-06T06:53:56.068 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00860226 s, 1.2 GB/s 2024-06-06T06:53:56.068 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-06T06:53:56.709 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:53:56.709 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:53:56.710 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.710 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] boot 2024-06-06T06:53:56.710 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T06:53:56.710 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.714 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:53:56.714 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:53:56.714 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.714 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] boot 2024-06-06T06:53:56.715 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T06:53:56.715 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.743 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_2 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: osd.6 [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] boot 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: osdmap e47: 7 total, 7 up, 7 in 2024-06-06T06:53:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T06:53:56.986 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi073 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 5696M 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:57.971 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:57.972 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:57 smithi136 ceph-mon[35901]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:53:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T06:53:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi073 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 5696M 2024-06-06T06:53:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:58.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:58.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:57 smithi170 ceph-mon[36108]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: osdmap e48: 7 total, 7 up, 7 in 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi073 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 5696M 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:58.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:53:58.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:53:58.276 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:57 smithi073 ceph-mon[32303]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:53:58.464 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.461+0000 7f05c8dd3700 1 -- 172.21.15.136:0/4013381921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 msgr2=0x7f05c4107d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.461+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/4013381921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4107d00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f05b8009990 tx=0x7f05b8009ca0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 -- 172.21.15.136:0/4013381921 shutdown_connections 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/4013381921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 0x7f05c410a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/4013381921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 0x7f05c41053d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/4013381921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4107d00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 -- 172.21.15.136:0/4013381921 >> 172.21.15.136:0/4013381921 conn(0x7f05c40fc9b0 msgr2=0x7f05c40fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 -- 172.21.15.136:0/4013381921 shutdown_connections 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 -- 172.21.15.136:0/4013381921 wait complete. 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 Processor -- start 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 -- start start 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.462+0000 7f05c8dd3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 0x7f05c41987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:58.465 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 0x7f05c419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410e090 con 0x7f05c4102fe0 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410df10 con 0x7f05c4105910 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05c410e210 con 0x7f05c4108240 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:60888/0 (socket says 172.21.15.136:60888) 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 -- 172.21.15.136:0/285213890 learned_addr learned my addr 172.21.15.136:0/285213890 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c2d9d700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 0x7f05c419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c259c700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 0x7f05c41987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:58.466 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 -- 172.21.15.136:0/285213890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 msgr2=0x7f05c419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 0x7f05c419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 -- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 msgr2=0x7f05c41987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 0x7f05c41987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 -- 172.21.15.136:0/285213890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c41a0400 con 0x7f05c4105910 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c1d9b700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b5c0 tx=0x7f05b8005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8003c90 con 0x7f05c4105910 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05b8004830 con 0x7f05c4105910 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8005540 con 0x7f05c4105910 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c41a0690 con 0x7f05c4105910 2024-06-06T06:53:58.467 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.463+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c41a0b00 con 0x7f05c4105910 2024-06-06T06:53:58.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.464+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c4069180 con 0x7f05c4105910 2024-06-06T06:53:58.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.465+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f05b80049e0 con 0x7f05c4105910 2024-06-06T06:53:58.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.466+0000 7f05b37fe700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 0x7f05ac0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:53:58.468 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.466+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f05b80cf950 con 0x7f05c4105910 2024-06-06T06:53:58.469 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.466+0000 7f05c259c700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 0x7f05ac0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:53:58.469 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.467+0000 7f05c259c700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 0x7f05ac0a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05b400a850 tx=0x7f05b4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:53:58.470 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.469+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f05b8013070 con 0x7f05c4105910 2024-06-06T06:53:58.673 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:53:58.672+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f05c40628d0 con 0x7f05ac0a2c20 2024-06-06T06:53:59.382 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:59.382 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:59.382 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:53:59 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:59.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:59.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:53:59 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:53:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:53:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:53:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:53:59 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:00.208 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:00 smithi136 ceph-mon[35901]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:00.208 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:00 smithi136 ceph-mon[35901]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:00.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:00 smithi073 ceph-mon[32303]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:00.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:00 smithi073 ceph-mon[32303]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:00 smithi170 ceph-mon[36108]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:00 smithi170 ceph-mon[36108]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:02.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:02.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/1194165878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]': finished 2024-06-06T06:54:02.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T06:54:02.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:02.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:02 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/2375472519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/1194165878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]': finished 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:02.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:02 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/2375472519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/1194165878' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]: dispatch 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "51b08617-a3c2-4de4-b0f7-3f505b13e7e6"}]': finished 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: osdmap e49: 8 total, 7 up, 8 in 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:02 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/2375472519' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:04.595 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:04 smithi170 ceph-mon[36108]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:04.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:04 smithi136 ceph-mon[35901]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:04 smithi073 ceph-mon[32303]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:06.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:06 smithi136 ceph-mon[35901]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:06.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:06 smithi170 ceph-mon[36108]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:06.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:06 smithi073 ceph-mon[32303]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:08.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:08 smithi170 ceph-mon[36108]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:08.642 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:08 smithi136 ceph-mon[35901]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:08.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:08 smithi073 ceph-mon[32303]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:09.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T06:54:09.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:09 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T06:54:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:09 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-06T06:54:09.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:09 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:10 smithi136 ceph-mon[35901]: Deploying daemon osd.7 on smithi136 2024-06-06T06:54:10.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:10 smithi136 ceph-mon[35901]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:10 smithi170 ceph-mon[36108]: Deploying daemon osd.7 on smithi136 2024-06-06T06:54:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:10 smithi170 ceph-mon[36108]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:10.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:10 smithi073 ceph-mon[32303]: Deploying daemon osd.7 on smithi136 2024-06-06T06:54:10.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:10 smithi073 ceph-mon[32303]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:12 smithi170 ceph-mon[36108]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:12.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:12 smithi073 ceph-mon[32303]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:12 smithi136 ceph-mon[35901]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:14 smithi170 ceph-mon[36108]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:14.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:14 smithi073 ceph-mon[32303]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:14 smithi136 ceph-mon[35901]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:14.775 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:16.596 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:16 smithi136 ceph-mon[35901]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:16.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:16 smithi170 ceph-mon[36108]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:16.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:16 smithi073 ceph-mon[32303]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:18.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.035+0000 7f05b37fe700 1 -- 172.21.15.136:0/285213890 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05c40628d0 con 0x7f05ac0a2c20 2024-06-06T06:54:18.038 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 7 on host 'smithi136' 2024-06-06T06:54:18.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 msgr2=0x7f05ac0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:18.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 0x7f05ac0a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f05b400a850 tx=0x7f05b4008040 comp rx=0 tx=0).stop 2024-06-06T06:54:18.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 msgr2=0x7f05c4198ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:18.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05b800b5c0 tx=0x7f05b8005e90 comp rx=0 tx=0).stop 2024-06-06T06:54:18.040 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 shutdown_connections 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f05ac0a2c20 0x7f05ac0a50e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05c4108240 0x7f05c419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05c4102fe0 0x7f05c41987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 --2- 172.21.15.136:0/285213890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05c4105910 0x7f05c4198ce0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 >> 172.21.15.136:0/285213890 conn(0x7f05c40fc9b0 msgr2=0x7f05c4103920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 shutdown_connections 2024-06-06T06:54:18.041 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:54:18.038+0000 7f05c8dd3700 1 -- 172.21.15.136:0/285213890 wait complete. 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:18 smithi136 ceph-mon[35901]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:18 smithi073 ceph-mon[32303]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:18.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:18.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:18.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:18 smithi170 ceph-mon[36108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-06T06:54:18.716 DEBUG:teuthology.orchestra.run.smithi136:osd.7> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.7.service 2024-06-06T06:54:18.718 INFO:tasks.cephadm:Deploying osd.8 on smithi170 with /dev/vg_nvme/lv_2... 2024-06-06T06:54:18.719 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-06T06:54:18.966 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.242 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-mon[35901]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:20.243 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T06:54:20.020+0000 7fa4a5f89700 -1 osd.7 0 waiting for initial osdmap 2024-06-06T06:54:20.243 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 06:54:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T06:54:20.031+0000 7fa4a1d7c700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:54:20.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T06:54:20.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T06:54:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:20 smithi073 ceph-mon[32303]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-06T06:54:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: osdmap e50: 8 total, 7 up, 8 in 2024-06-06T06:54:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: from='osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:20.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:54:20.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:20 smithi170 ceph-mon[36108]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 220 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:21.134 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-06T06:54:21.134 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-06T06:54:21.134 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-06-06T06:54:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-06-06T06:54:21.135 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00882829 s, 1.2 GB/s 2024-06-06T06:54:21.135 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:54:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:54:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:54:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: osdmap e51: 8 total, 7 up, 8 in 2024-06-06T06:54:21.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:21.790 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_2 2024-06-06T06:54:22.035 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:54:22.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] boot 2024-06-06T06:54:22.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T06:54:22.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.288 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:22.289 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:22 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] boot 2024-06-06T06:54:22.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:22.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:22 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: osd.7 [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] boot 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: osdmap e52: 8 total, 8 up, 8 in 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 221 MiB used, 626 GiB / 626 GiB avail 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:22.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:22 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:23.291 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:23 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi136 2024-06-06T06:54:23.291 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:23 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-06T06:54:23.292 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:23 smithi170 ceph-mon[36108]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T06:54:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:23 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi136 2024-06-06T06:54:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:23 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-06T06:54:23.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:23 smithi136 ceph-mon[35901]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T06:54:23.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:23 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi136 2024-06-06T06:54:23.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:23 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 5696M 2024-06-06T06:54:23.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:23 smithi073 ceph-mon[32303]: osdmap e53: 8 total, 8 up, 8 in 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- 172.21.15.170:0/3546996288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 msgr2=0x7f859c102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 --2- 172.21.15.170:0/3546996288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c102db0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f85900099c0 tx=0x7f8590009cd0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- 172.21.15.170:0/3546996288 shutdown_connections 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 --2- 172.21.15.170:0/3546996288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c102db0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 --2- 172.21.15.170:0/3546996288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c109930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 --2- 172.21.15.170:0/3546996288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.688 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- 172.21.15.170:0/3546996288 >> 172.21.15.170:0/3546996288 conn(0x7f859c0fc100 msgr2=0x7f859c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- 172.21.15.170:0/3546996288 shutdown_connections 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- 172.21.15.170:0/3546996288 wait complete. 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 Processor -- start 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.685+0000 7f85a21b8700 1 -- start start 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c19a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10ce70 con 0x7f859c102070 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10ccf0 con 0x7f859c1016d0 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f859c10cff0 con 0x7f859c102990 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:23.689 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:41392/0 (socket says 172.21.15.170:41392) 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 -- 172.21.15.170:0/2091615345 learned_addr learned my addr 172.21.15.170:0/2091615345 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859b7fe700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c19a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859affd700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 -- 172.21.15.170:0/2091615345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 msgr2=0x7f859c19a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c19a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 -- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 msgr2=0x7f859c19adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c19adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 -- 172.21.15.170:0/2091615345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859c1a24c0 con 0x7f859c102990 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859affd700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859b7fe700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:54:23.690 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f859bfff700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f859400add0 tx=0x7f859400c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:54:23.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594017020 con 0x7f859c102990 2024-06-06T06:54:23.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859c1a27b0 con 0x7f859c102990 2024-06-06T06:54:23.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f859400cdf0 con 0x7f859c102990 2024-06-06T06:54:23.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8594005460 con 0x7f859c102990 2024-06-06T06:54:23.691 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.686+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f859c1a2ca0 con 0x7f859c102990 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.688+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859c069180 con 0x7f859c102990 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.689+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8594004ad0 con 0x7f859c102990 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.689+0000 7f8598ff9700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 0x7f85840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.689+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f85940cdd40 con 0x7f859c102990 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.689+0000 7f859b7fe700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 0x7f85840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:23.692 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.690+0000 7f859b7fe700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 0x7f85840a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8590000c00 tx=0x7f8590005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:54:23.694 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.694+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8594021020 con 0x7f859c102990 2024-06-06T06:54:23.898 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:23.897+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f859c1a2fc0 con 0x7f85840a2c20 2024-06-06T06:54:24.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:24 smithi136 ceph-mon[35901]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T06:54:24.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:24 smithi136 ceph-mon[35901]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:24.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:24 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:24.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:24 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:24.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:24 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:24.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:24 smithi170 ceph-mon[36108]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T06:54:24.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:24 smithi170 ceph-mon[36108]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:24.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:24 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:24.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:24 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:24.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:24 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:24.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:24 smithi073 ceph-mon[32303]: osdmap e54: 8 total, 8 up, 8 in 2024-06-06T06:54:24.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:24 smithi073 ceph-mon[32303]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:24.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:24.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:24.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:24 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:25.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:25 smithi170 ceph-mon[36108]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:25.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:25 smithi170 ceph-mon[36108]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T06:54:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:25 smithi073 ceph-mon[32303]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:25.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:25 smithi073 ceph-mon[32303]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T06:54:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:25 smithi136 ceph-mon[35901]: from='client.24295 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:25 smithi136 ceph-mon[35901]: osdmap e55: 8 total, 8 up, 8 in 2024-06-06T06:54:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:26 smithi170 ceph-mon[36108]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T06:54:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:26 smithi170 ceph-mon[36108]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:26.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:26 smithi073 ceph-mon[32303]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T06:54:26.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:26 smithi073 ceph-mon[32303]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:26.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:26 smithi136 ceph-mon[35901]: osdmap e56: 8 total, 8 up, 8 in 2024-06-06T06:54:26.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:26 smithi136 ceph-mon[35901]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:27 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/2470774165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:27 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:27 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]': finished 2024-06-06T06:54:27.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:27 smithi073 ceph-mon[32303]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T06:54:27.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:27 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/2470774165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:27 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:27 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]': finished 2024-06-06T06:54:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:27 smithi170 ceph-mon[36108]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T06:54:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:27 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/2470774165' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:27 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]: dispatch 2024-06-06T06:54:27.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:27 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "edab0c0c-df86-49f8-81b9-90f481b58410"}]': finished 2024-06-06T06:54:27.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:27 smithi136 ceph-mon[35901]: osdmap e57: 9 total, 8 up, 9 in 2024-06-06T06:54:27.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:28 smithi170 ceph-mon[36108]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:28 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/826775399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:28.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:28 smithi073 ceph-mon[32303]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:28.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:28 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/826775399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:28.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:28 smithi136 ceph-mon[35901]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:28.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:28 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/826775399' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:30.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:30 smithi170 ceph-mon[36108]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:30.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:30 smithi073 ceph-mon[32303]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:30.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:30 smithi136 ceph-mon[35901]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:32.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:32 smithi170 ceph-mon[36108]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:32 smithi073 ceph-mon[32303]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:32 smithi136 ceph-mon[35901]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:34 smithi170 ceph-mon[36108]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:34 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T06:54:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:34 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:34.603 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:34 smithi136 ceph-mon[35901]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:34.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:34 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T06:54:34.604 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:34 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:34.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:34 smithi073 ceph-mon[32303]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:34.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:34 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-06T06:54:34.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:34 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:35 smithi170 ceph-mon[36108]: Deploying daemon osd.8 on smithi170 2024-06-06T06:54:35.619 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:35 smithi136 ceph-mon[35901]: Deploying daemon osd.8 on smithi170 2024-06-06T06:54:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:35 smithi073 ceph-mon[32303]: Deploying daemon osd.8 on smithi170 2024-06-06T06:54:36.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:36 smithi170 ceph-mon[36108]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:36 smithi073 ceph-mon[32303]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:36.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:36 smithi136 ceph-mon[35901]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:38.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:38 smithi170 ceph-mon[36108]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:38 smithi073 ceph-mon[32303]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:38 smithi136 ceph-mon[35901]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:40 smithi170 ceph-mon[36108]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:40.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:40 smithi073 ceph-mon[32303]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:40 smithi136 ceph-mon[35901]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:42 smithi170 ceph-mon[36108]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:42.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:42 smithi073 ceph-mon[32303]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:42 smithi136 ceph-mon[35901]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:43.263 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 8 on host 'smithi170' 2024-06-06T06:54:43.263 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.262+0000 7f8598ff9700 1 -- 172.21.15.170:0/2091615345 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f859c1a2fc0 con 0x7f85840a2c20 2024-06-06T06:54:43.265 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 msgr2=0x7f85840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 0x7f85840a50e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8590000c00 tx=0x7f8590005fd0 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 msgr2=0x7f859c1a1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f859400add0 tx=0x7f859400c330 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 shutdown_connections 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f85840a2c20 0x7f85840a50e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f859c102990 0x7f859c1a1e20 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f859c1016d0 0x7f859c19a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 --2- 172.21.15.170:0/2091615345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f859c102070 0x7f859c19adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:43.266 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.264+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 >> 172.21.15.170:0/2091615345 conn(0x7f859c0fc100 msgr2=0x7f859c0fdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:54:43.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.265+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 shutdown_connections 2024-06-06T06:54:43.267 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:54:43.265+0000 7f85a21b8700 1 -- 172.21.15.170:0/2091615345 wait complete. 2024-06-06T06:54:44.266 DEBUG:teuthology.orchestra.run.smithi170:osd.8> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.8.service 2024-06-06T06:54:44.268 INFO:tasks.cephadm:Deploying osd.9 on smithi073 with /dev/vg_nvme/lv_1... 2024-06-06T06:54:44.268 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T06:54:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:54:44.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:44 smithi073 ceph-mon[32303]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:44.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:44 smithi073 ceph-mon[32303]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:44.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:44 smithi073 ceph-mon[32303]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:44.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:44 smithi136 ceph-mon[35901]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:44.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:44 smithi136 ceph-mon[35901]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:44.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:44 smithi136 ceph-mon[35901]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:44 smithi170 ceph-mon[36108]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:44 smithi170 ceph-mon[36108]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:44 smithi170 ceph-mon[36108]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.423 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-06T06:54:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: osdmap e58: 9 total, 8 up, 9 in 2024-06-06T06:54:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:54:45.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:45.598 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T06:54:45.266+0000 7fc1a6b89700 -1 osd.8 0 waiting for initial osdmap 2024-06-06T06:54:45.598 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 06:54:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T06:54:45.278+0000 7fc1a297c700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:54:46.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:46 smithi170 ceph-mon[36108]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:54:46.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:46 smithi170 ceph-mon[36108]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T06:54:46.272 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:46 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.273 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:46 smithi170 ceph-mon[36108]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:46.273 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:46 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:46 smithi073 ceph-mon[32303]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:54:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:46 smithi073 ceph-mon[32303]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T06:54:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:46 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:46 smithi073 ceph-mon[32303]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:46.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:46 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:46 smithi136 ceph-mon[35901]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:54:46.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:46 smithi136 ceph-mon[35901]: osdmap e59: 9 total, 8 up, 9 in 2024-06-06T06:54:46.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:46 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:46 smithi136 ceph-mon[35901]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:46.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:46 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr: stderr: 10+0 records in 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr:10+0 records out 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113387 s, 925 MB/s 2024-06-06T06:54:46.727 INFO:teuthology.orchestra.run.smithi073.stderr:--> Zapping successful for: 2024-06-06T06:54:47.493 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:47 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:54:47.493 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:47 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:54:47.493 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:47 smithi073 ceph-mon[32303]: osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] boot 2024-06-06T06:54:47.493 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:47 smithi073 ceph-mon[32303]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T06:54:47.493 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:47 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:47.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_1 2024-06-06T06:54:47.537 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:47 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:54:47.537 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:47 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:54:47.537 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:47 smithi170 ceph-mon[36108]: osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] boot 2024-06-06T06:54:47.537 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:47 smithi170 ceph-mon[36108]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T06:54:47.537 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:47 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:47 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:54:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:47 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:54:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:47 smithi136 ceph-mon[35901]: osd.8 [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] boot 2024-06-06T06:54:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:47 smithi136 ceph-mon[35901]: osdmap e60: 9 total, 9 up, 9 in 2024-06-06T06:54:47.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:47 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T06:54:47.761 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:54:48.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:48.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:48.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: pgmap v198: 1 pgs: 1 remapped+peering; 577 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-06-06T06:54:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: osdmap e61: 9 total, 9 up, 9 in 2024-06-06T06:54:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:54:48.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:54:49.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 -- 172.21.15.73:0/660651850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 msgr2=0x7f5a54109940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 --2- 172.21.15.73:0/660651850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54109940 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00b0d0 tx=0x7f5a4c00b3e0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 -- 172.21.15.73:0/660651850 shutdown_connections 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 --2- 172.21.15.73:0/660651850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 0x7f5a54102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 --2- 172.21.15.73:0/660651850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54109940 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.502+0000 7f5a5a945700 1 --2- 172.21.15.73:0/660651850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 0x7f5a54101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 -- 172.21.15.73:0/660651850 >> 172.21.15.73:0/660651850 conn(0x7f5a540fc130 msgr2=0x7f5a540fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 -- 172.21.15.73:0/660651850 shutdown_connections 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 -- 172.21.15.73:0/660651850 wait complete. 2024-06-06T06:54:49.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 Processor -- start 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 -- start start 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.503+0000 7f5a5a945700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 0x7f5a54198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 0x7f5a5419fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410ce80 con 0x7f5a54102080 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410cd00 con 0x7f5a541016e0 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a5410d000 con 0x7f5a541029a0 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41064/0 (socket says 172.21.15.73:41064) 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 -- 172.21.15.73:0/3674728496 learned_addr learned my addr 172.21.15.73:0/3674728496 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a53fff700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 0x7f5a54198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:49.507 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a58ee2700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 0x7f5a5419fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 -- 172.21.15.73:0/3674728496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 msgr2=0x7f5a5419fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 0x7f5a5419fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 -- 172.21.15.73:0/3674728496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 msgr2=0x7f5a54198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 0x7f5a54198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 -- 172.21.15.73:0/3674728496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a541a00a0 con 0x7f5a54102080 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a537fe700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c9e0 tx=0x7f5a4800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48007ac0 con 0x7f5a54102080 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a48007c60 con 0x7f5a54102080 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48018660 con 0x7f5a54102080 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a541a0390 con 0x7f5a54102080 2024-06-06T06:54:49.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.504+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a541a0880 con 0x7f5a54102080 2024-06-06T06:54:49.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.505+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a54069180 con 0x7f5a54102080 2024-06-06T06:54:49.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.507+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5a4800f450 con 0x7f5a54102080 2024-06-06T06:54:49.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.507+0000 7f5a517fa700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 0x7f5a3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:54:49.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.507+0000 7f5a53fff700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 0x7f5a3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:54:49.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.507+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5829+0+0 (secure 0 0 0) 0x7f5a480ce8d0 con 0x7f5a54102080 2024-06-06T06:54:49.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.508+0000 7f5a53fff700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 0x7f5a3c0a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a5406a960 tx=0x7f5a4400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:54:49.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.511+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a48014030 con 0x7f5a54102080 2024-06-06T06:54:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:49 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi170 2024-06-06T06:54:49.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:49 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 5696M 2024-06-06T06:54:49.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:49 smithi136 ceph-mon[35901]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T06:54:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:49 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi170 2024-06-06T06:54:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:49 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 5696M 2024-06-06T06:54:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:49 smithi170 ceph-mon[36108]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T06:54:49.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:54:49.722+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5a540fe190 con 0x7f5a3c0a2c20 2024-06-06T06:54:49.724 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:49 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi170 2024-06-06T06:54:49.724 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:49 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 5696M 2024-06-06T06:54:49.724 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:49 smithi073 ceph-mon[32303]: osdmap e62: 9 total, 9 up, 9 in 2024-06-06T06:54:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:50 smithi136 ceph-mon[35901]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:50 smithi136 ceph-mon[35901]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T06:54:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:50.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:50 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:50.561 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:50 smithi073 ceph-mon[32303]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:50.561 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:50 smithi073 ceph-mon[32303]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T06:54:50.562 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:50.562 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:50.562 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:50 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:50 smithi170 ceph-mon[36108]: pgmap v201: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:50 smithi170 ceph-mon[36108]: osdmap e63: 9 total, 9 up, 9 in 2024-06-06T06:54:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:54:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:54:50.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:50 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:54:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:51 smithi170 ceph-mon[36108]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:51 smithi170 ceph-mon[36108]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T06:54:51.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:51 smithi073 ceph-mon[32303]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:51.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:51 smithi073 ceph-mon[32303]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T06:54:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:51 smithi136 ceph-mon[35901]: from='client.14409 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:54:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:51 smithi136 ceph-mon[35901]: osdmap e64: 9 total, 9 up, 9 in 2024-06-06T06:54:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:52 smithi170 ceph-mon[36108]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:52 smithi073 ceph-mon[32303]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:52 smithi136 ceph-mon[35901]: pgmap v204: 1 pgs: 1 remapped+peering; 577 KiB data, 275 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:54:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:53 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]: dispatch 2024-06-06T06:54:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:53 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]': finished 2024-06-06T06:54:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:53 smithi170 ceph-mon[36108]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T06:54:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:54:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:53 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1079730555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:53 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]: dispatch 2024-06-06T06:54:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:53 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]': finished 2024-06-06T06:54:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:53 smithi073 ceph-mon[32303]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T06:54:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:54:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:53 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1079730555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:53 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]: dispatch 2024-06-06T06:54:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:53 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/16816815' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cd7bb4f4-bc75-4e2d-9a15-b414a84302ce"}]': finished 2024-06-06T06:54:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:53 smithi136 ceph-mon[35901]: osdmap e65: 10 total, 9 up, 10 in 2024-06-06T06:54:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:54:53.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:53 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1079730555' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:54:54.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:54 smithi170 ceph-mon[36108]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-06-06T06:54:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:54 smithi073 ceph-mon[32303]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-06-06T06:54:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:54 smithi136 ceph-mon[35901]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-06-06T06:54:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:56 smithi073 ceph-mon[32303]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:54:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:56 smithi170 ceph-mon[36108]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:54:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:56 smithi136 ceph-mon[35901]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:54:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:54:58 smithi170 ceph-mon[36108]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T06:54:58.696 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:54:58 smithi073 ceph-mon[32303]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T06:54:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:54:58 smithi136 ceph-mon[35901]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-06-06T06:55:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:00 smithi170 ceph-mon[36108]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T06:55:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:00 smithi073 ceph-mon[32303]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T06:55:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:00 smithi136 ceph-mon[35901]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-06-06T06:55:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T06:55:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:01 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:01.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T06:55:01.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:01 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-06T06:55:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:01 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:02.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:02 smithi170 ceph-mon[36108]: Deploying daemon osd.9 on smithi073 2024-06-06T06:55:02.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:02 smithi170 ceph-mon[36108]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:55:02.619 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:02 smithi073 ceph-mon[32303]: Deploying daemon osd.9 on smithi073 2024-06-06T06:55:02.619 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:02 smithi073 ceph-mon[32303]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:55:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:02 smithi136 ceph-mon[35901]: Deploying daemon osd.9 on smithi073 2024-06-06T06:55:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:02 smithi136 ceph-mon[35901]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-06T06:55:04.582 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:04 smithi073 ceph-mon[32303]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:04.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:04 smithi170 ceph-mon[36108]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:04.603 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:04 smithi136 ceph-mon[35901]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:06.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:06 smithi073 ceph-mon[32303]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:06.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:06 smithi170 ceph-mon[36108]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:06 smithi136 ceph-mon[35901]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:07.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:07.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:07.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:07.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:07 smithi073 ceph-mon[32303]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:07 smithi170 ceph-mon[36108]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:08.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:08.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:07 smithi136 ceph-mon[35901]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:10.555 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:10.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:10.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.601 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.600+0000 7f5a517fa700 1 -- 172.21.15.73:0/3674728496 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5a540fe190 con 0x7f5a3c0a2c20 2024-06-06T06:55:10.601 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 9 on host 'smithi073' 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 msgr2=0x7f5a3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 0x7f5a3c0a50e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f5a5406a960 tx=0x7f5a4400a3b0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 msgr2=0x7f5a54198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800c9e0 tx=0x7f5a4800ccf0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 shutdown_connections 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a3c0a2c20 0x7f5a3c0a50e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a541029a0 0x7f5a5419fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a541016e0 0x7f5a54198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 --2- 172.21.15.73:0/3674728496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a54102080 0x7f5a54198980 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 >> 172.21.15.73:0/3674728496 conn(0x7f5a540fc130 msgr2=0x7f5a540fda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:55:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 shutdown_connections 2024-06-06T06:55:10.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:55:10.603+0000 7f5a5a945700 1 -- 172.21.15.73:0/3674728496 wait complete. 2024-06-06T06:55:10.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.329 DEBUG:teuthology.orchestra.run.smithi073:osd.9> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.9.service 2024-06-06T06:55:11.331 INFO:tasks.cephadm:Deploying osd.10 on smithi136 with /dev/vg_nvme/lv_1... 2024-06-06T06:55:11.332 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T06:55:11.563 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:11 smithi073 ceph-mon[32303]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T06:55:11.563 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.563 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.563 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:11 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:11.580 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:55:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:11 smithi170 ceph-mon[36108]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T06:55:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:11 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:11.701 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:11 smithi136 ceph-mon[35901]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-06T06:55:11.701 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:11.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:11 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:12 smithi170 ceph-mon[36108]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:12 smithi170 ceph-mon[36108]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T06:55:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:12 smithi170 ceph-mon[36108]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T06:55:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:12 smithi170 ceph-mon[36108]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:55:12.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:12 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-mon[32303]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-mon[32303]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T06:55:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-mon[32303]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T06:55:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-mon[32303]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:55:12.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:12.637 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:12 smithi136 ceph-mon[35901]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:12.638 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:12 smithi136 ceph-mon[35901]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-06T06:55:12.638 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:12 smithi136 ceph-mon[35901]: osdmap e66: 10 total, 9 up, 10 in 2024-06-06T06:55:12.638 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:12 smithi136 ceph-mon[35901]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-06-06T06:55:12.638 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:12 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:12.928 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T06:55:12.741+0000 7f46c1dd7700 -1 osd.9 0 waiting for initial osdmap 2024-06-06T06:55:12.928 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 06:55:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T06:55:12.754+0000 7f46b9bc2700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:55:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:13 smithi073 ceph-mon[32303]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:55:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:13 smithi073 ceph-mon[32303]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T06:55:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:13 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:13 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:13 smithi073 ceph-mon[32303]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr: stderr: 10+0 records in 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:10+0 records out 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00991723 s, 1.1 GB/s 2024-06-06T06:55:13.526 INFO:teuthology.orchestra.run.smithi136.stderr:--> Zapping successful for: 2024-06-06T06:55:13.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:13 smithi136 ceph-mon[35901]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:55:13.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:13 smithi136 ceph-mon[35901]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T06:55:13.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:13 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:13 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.578 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:13 smithi136 ceph-mon[35901]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' 2024-06-06T06:55:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:13 smithi170 ceph-mon[36108]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-06-06T06:55:13.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:13 smithi170 ceph-mon[36108]: osdmap e67: 10 total, 9 up, 10 in 2024-06-06T06:55:13.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:13 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:13 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:13.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:13 smithi170 ceph-mon[36108]: from='osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105]' entity='osd.9' 2024-06-06T06:55:14.234 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi136:vg_nvme/lv_1 2024-06-06T06:55:14.477 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T06:55:14.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:55:14.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:55:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] boot 2024-06-06T06:55:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T06:55:14.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] boot 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T06:55:14.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:14.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:55:14.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:55:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 276 MiB used, 804 GiB / 805 GiB avail 2024-06-06T06:55:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: osd.9 [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] boot 2024-06-06T06:55:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: osdmap e68: 10 total, 10 up, 10 in 2024-06-06T06:55:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T06:55:15.846 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi073 2024-06-06T06:55:15.846 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.846 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.846 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 4272M 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:15.847 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:15 smithi136 ceph-mon[35901]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi073 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 4272M 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:15.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:15.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:15.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:15 smithi170 ceph-mon[36108]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T06:55:15.926 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi073 2024-06-06T06:55:15.926 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 4272M 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.927 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:15.928 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:15.928 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:15.928 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:15.928 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:15 smithi073 ceph-mon[32303]: osdmap e69: 10 total, 10 up, 10 in 2024-06-06T06:55:16.097 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 -- 172.21.15.136:0/4189687489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 msgr2=0x7fd5fc102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 --2- 172.21.15.136:0/4189687489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc102930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd5f00099c0 tx=0x7fd5f0009cd0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 -- 172.21.15.136:0/4189687489 shutdown_connections 2024-06-06T06:55:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 --2- 172.21.15.136:0/4189687489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 0x7fd5fc10a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 --2- 172.21.15.136:0/4189687489 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc102930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.098 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 --2- 172.21.15.136:0/4189687489 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 -- 172.21.15.136:0/4189687489 >> 172.21.15.136:0/4189687489 conn(0x7fd5fc0fdab0 msgr2=0x7fd5fc0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:55:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 -- 172.21.15.136:0/4189687489 shutdown_connections 2024-06-06T06:55:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 -- 172.21.15.136:0/4189687489 wait complete. 2024-06-06T06:55:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.095+0000 7fd602ab8700 1 Processor -- start 2024-06-06T06:55:16.099 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 -- start start 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 0x7fd5fc1a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10dfc0 con 0x7fd5fc1040b0 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10de40 con 0x7fd5fc102510 2024-06-06T06:55:16.100 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd602ab8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5fc10e140 con 0x7fd5fc103710 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.136:41192/0 (socket says 172.21.15.136:41192) 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 -- 172.21.15.136:0/1526186975 learned_addr learned my addr 172.21.15.136:0/1526186975 (peer_addr_for_me v2:172.21.15.136:0/0) 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd601055700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 0x7fd5fc1a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd5fbfff700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:16.101 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 -- 172.21.15.136:0/1526186975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 msgr2=0x7fd5fc19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 -- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 msgr2=0x7fd5fc1a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 0x7fd5fc1a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 -- 172.21.15.136:0/1526186975 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5fc1a6870 con 0x7fd5fc102510 2024-06-06T06:55:16.102 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd5fbfff700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:55:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd600854700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c010 tx=0x7fd5f0003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:55:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.096+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f0027070 con 0x7fd5fc102510 2024-06-06T06:55:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.097+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5f00041d0 con 0x7fd5fc102510 2024-06-06T06:55:16.103 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.097+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5fc1a6b00 con 0x7fd5fc102510 2024-06-06T06:55:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.097+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f000f460 con 0x7fd5fc102510 2024-06-06T06:55:16.104 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.097+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5fc1a7030 con 0x7fd5fc102510 2024-06-06T06:55:16.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.098+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5fc069180 con 0x7fd5fc102510 2024-06-06T06:55:16.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.099+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd5f002a070 con 0x7fd5fc102510 2024-06-06T06:55:16.105 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.099+0000 7fd5f9ffb700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 0x7fd5ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:16.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.100+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd5f00d0150 con 0x7fd5fc102510 2024-06-06T06:55:16.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.100+0000 7fd5fbfff700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 0x7fd5ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:16.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.100+0000 7fd5fbfff700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 0x7fd5ec0a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005950 tx=0x7fd5e8005f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:55:16.106 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.103+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5f0031070 con 0x7fd5fc102510 2024-06-06T06:55:16.309 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:16.308+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd5fc0628d0 con 0x7fd5ec0a2ca0 2024-06-06T06:55:16.876 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:16 smithi073 ceph-mon[32303]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:16.877 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:16 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:16.877 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:16 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:16.877 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:16 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:16.923 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:16 smithi136 ceph-mon[35901]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:16.923 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:16 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:16.923 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:16 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:16.923 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:16 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:16 smithi170 ceph-mon[36108]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi136:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:16 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:16 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:16 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:17 smithi073 ceph-mon[32303]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:17 smithi136 ceph-mon[35901]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:18.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:17 smithi170 ceph-mon[36108]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:19.695 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:19.696 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/718224566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.696 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.696 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]': finished 2024-06-06T06:55:19.696 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T06:55:19.696 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:19 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/718224566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]': finished 2024-06-06T06:55:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T06:55:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:19 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/718224566' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]: dispatch 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cac56766-4505-4e9f-9b82-2dbf14ba3162"}]': finished 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: osdmap e70: 11 total, 10 up, 11 in 2024-06-06T06:55:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:19 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:20.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:20 smithi073 ceph-mon[32303]: from='client.? 172.21.15.136:0/2887640906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:20 smithi136 ceph-mon[35901]: from='client.? 172.21.15.136:0/2887640906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:20 smithi170 ceph-mon[36108]: from='client.? 172.21.15.136:0/2887640906' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:21 smithi073 ceph-mon[32303]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:21 smithi136 ceph-mon[35901]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:21 smithi170 ceph-mon[36108]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:24.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:24 smithi136 ceph-mon[35901]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:24 smithi170 ceph-mon[36108]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:24.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:24 smithi073 ceph-mon[32303]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:26.548 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:26 smithi136 ceph-mon[35901]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:26 smithi170 ceph-mon[36108]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:26.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:26 smithi073 ceph-mon[32303]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T06:55:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:27.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T06:55:27.631 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:27.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-06T06:55:27.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:28.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:28 smithi136 ceph-mon[35901]: Deploying daemon osd.10 on smithi136 2024-06-06T06:55:28.386 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:28 smithi136 ceph-mon[35901]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:28 smithi170 ceph-mon[36108]: Deploying daemon osd.10 on smithi136 2024-06-06T06:55:28.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:28 smithi170 ceph-mon[36108]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:28 smithi073 ceph-mon[32303]: Deploying daemon osd.10 on smithi136 2024-06-06T06:55:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:28 smithi073 ceph-mon[32303]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:30.595 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:30 smithi136 ceph-mon[35901]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:30.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:30 smithi170 ceph-mon[36108]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:30.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:30 smithi073 ceph-mon[32303]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:32.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:32 smithi136 ceph-mon[35901]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:32 smithi073 ceph-mon[32303]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:32.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:32 smithi170 ceph-mon[36108]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:33.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:33 smithi073 ceph-mon[32303]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:33.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:33.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:33 smithi136 ceph-mon[35901]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:33 smithi170 ceph-mon[36108]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:35.774 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.773+0000 7fd5f9ffb700 1 -- 172.21.15.136:0/1526186975 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd5fc0628d0 con 0x7fd5ec0a2ca0 2024-06-06T06:55:35.775 INFO:teuthology.orchestra.run.smithi136.stdout:Created osd(s) 10 on host 'smithi136' 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 msgr2=0x7fd5ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 0x7fd5ec0a5160 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8005950 tx=0x7fd5e8005f30 comp rx=0 tx=0).stop 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 msgr2=0x7fd5fc19ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd5f000c010 tx=0x7fd5f0003680 comp rx=0 tx=0).stop 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 shutdown_connections 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5fc1040b0 0x7fd5fc1a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:35.777 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd5fc102510 0x7fd5fc19ec20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:35.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5ec0a2ca0 0x7fd5ec0a5160 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:35.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 --2- 172.21.15.136:0/1526186975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5fc103710 0x7fd5fc19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:35.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 >> 172.21.15.136:0/1526186975 conn(0x7fd5fc0fdab0 msgr2=0x7fd5fc104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:55:35.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 shutdown_connections 2024-06-06T06:55:35.778 INFO:teuthology.orchestra.run.smithi136.stderr:2024-06-06T06:55:35.776+0000 7fd602ab8700 1 -- 172.21.15.136:0/1526186975 wait complete. 2024-06-06T06:55:36.423 DEBUG:teuthology.orchestra.run.smithi136:osd.10> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.10.service 2024-06-06T06:55:36.425 INFO:tasks.cephadm:Deploying osd.11 on smithi170 with /dev/vg_nvme/lv_1... 2024-06-06T06:55:36.425 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-06T06:55:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:36 smithi170 ceph-mon[36108]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:36 smithi170 ceph-mon[36108]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:36.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:36 smithi170 ceph-mon[36108]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:36.675 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:55:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:36 smithi073 ceph-mon[32303]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:36 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:36 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:36 smithi073 ceph-mon[32303]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:36 smithi073 ceph-mon[32303]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:36 smithi136 ceph-mon[35901]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:36 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:36 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:36.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:36 smithi136 ceph-mon[35901]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:36.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:36 smithi136 ceph-mon[35901]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:37.763 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-mon[35901]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:37.823 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:37 smithi170 ceph-mon[36108]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:38.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: osdmap e71: 11 total, 10 up, 11 in 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]: dispatch 2024-06-06T06:55:38.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:37 smithi073 ceph-mon[32303]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:38.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T06:55:37.761+0000 7f979a2e0700 -1 osd.10 0 waiting for initial osdmap 2024-06-06T06:55:38.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 06:55:37 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T06:55:37.773+0000 7f97960d3700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00935653 s, 1.1 GB/s 2024-06-06T06:55:38.768 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-06-06T06:55:38.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:38 smithi136 ceph-mon[35901]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:55:38.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:38 smithi136 ceph-mon[35901]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T06:55:38.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:38.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:38 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:38 smithi073 ceph-mon[32303]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:55:39.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:38 smithi073 ceph-mon[32303]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T06:55:39.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:38 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.068 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:38 smithi170 ceph-mon[36108]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi136", "root=default"]}]': finished 2024-06-06T06:55:39.068 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:38 smithi170 ceph-mon[36108]: osdmap e72: 11 total, 10 up, 11 in 2024-06-06T06:55:39.068 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.069 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:38 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.514 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi170:vg_nvme/lv_1 2024-06-06T06:55:39.768 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:55:39.935 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:55:39.935 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:55:39.935 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.936 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] boot 2024-06-06T06:55:39.936 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T06:55:39.936 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:39.936 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:39 smithi136 ceph-mon[35901]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] boot 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:40.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:39 smithi073 ceph-mon[32303]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: osd.10 [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] boot 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: osdmap e73: 11 total, 11 up, 11 in 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T06:55:40.042 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:39 smithi170 ceph-mon[36108]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 307 MiB used, 894 GiB / 894 GiB avail 2024-06-06T06:55:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T06:55:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi136 2024-06-06T06:55:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:40.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:40.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:40 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi136 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:41.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:41.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:41.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:41.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:41.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:40 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: osdmap e74: 11 total, 11 up, 11 in 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi136 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 4272M 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:41.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:41.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:41.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:41.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:55:41.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:40 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 -- 172.21.15.170:0/1473381931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901025e0 msgr2=0x7f3590102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 --2- 172.21.15.170:0/1473381931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901025e0 0x7f3590102a40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3584009990 tx=0x7f3584009ca0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 -- 172.21.15.170:0/1473381931 shutdown_connections 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 --2- 172.21.15.170:0/1473381931 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3590102f80 0x7f3590109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 --2- 172.21.15.170:0/1473381931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901013e0 0x7f3590101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 --2- 172.21.15.170:0/1473381931 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901025e0 0x7f3590102a40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 -- 172.21.15.170:0/1473381931 >> 172.21.15.170:0/1473381931 conn(0x7f35900fc980 msgr2=0x7f35900fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 -- 172.21.15.170:0/1473381931 shutdown_connections 2024-06-06T06:55:41.429 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.428+0000 7f3594a01700 1 -- 172.21.15.170:0/1473381931 wait complete. 2024-06-06T06:55:41.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 Processor -- start 2024-06-06T06:55:41.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 -- start start 2024-06-06T06:55:41.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:41.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901025e0 0x7f3590198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:41.435 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3590102f80 0x7f359019fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f359010ce60 con 0x7f35901025e0 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f359010cce0 con 0x7f3590102f80 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f3594a01700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f359010cfe0 con 0x7f35901013e0 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:33854/0 (socket says 172.21.15.170:33854) 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 -- 172.21.15.170:0/286779252 learned_addr learned my addr 172.21.15.170:0/286779252 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358dd9b700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901025e0 0x7f3590198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 -- 172.21.15.170:0/286779252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3590102f80 msgr2=0x7f359019fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3590102f80 0x7f359019fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 -- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901025e0 msgr2=0x7f3590198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901025e0 0x7f3590198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:55:41.436 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 -- 172.21.15.170:0/286779252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35901a0230 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.429+0000 7f358e59c700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f358000d9b0 tx=0x7f358000dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.430+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3580017030 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.430+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35800044d0 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.430+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35901a0520 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.430+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35800039e0 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.430+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35901a0a60 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.431+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3590069180 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.432+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f358001a070 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.432+0000 7f357f7fe700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 0x7f35780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.432+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f35800ce700 con 0x7f35901013e0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.433+0000 7f358dd9b700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 0x7f35780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:55:41.437 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.433+0000 7f358dd9b700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 0x7f35780a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f358400b5c0 tx=0x7f3584005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:55:41.438 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.436+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f358009bc90 con 0x7f35901013e0 2024-06-06T06:55:41.642 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:55:41.641+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f359006b110 con 0x7f35780a2c20 2024-06-06T06:55:42.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:41 smithi136 ceph-mon[35901]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T06:55:42.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:41 smithi136 ceph-mon[35901]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:42.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:42.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:42.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:41 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:41 smithi170 ceph-mon[36108]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T06:55:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:41 smithi170 ceph-mon[36108]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:42.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:41 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:41 smithi073 ceph-mon[32303]: osdmap e75: 11 total, 11 up, 11 in 2024-06-06T06:55:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:41 smithi073 ceph-mon[32303]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-06T06:55:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-06T06:55:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:41 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:43.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:42 smithi136 ceph-mon[35901]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:43.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:42 smithi136 ceph-mon[35901]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T06:55:43.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:42 smithi170 ceph-mon[36108]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:43.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:42 smithi170 ceph-mon[36108]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T06:55:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:42 smithi073 ceph-mon[32303]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:55:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:42 smithi073 ceph-mon[32303]: osdmap e76: 11 total, 11 up, 11 in 2024-06-06T06:55:43.836 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:43 smithi170 ceph-mon[36108]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T06:55:43.837 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:43 smithi170 ceph-mon[36108]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:55:44.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:43 smithi136 ceph-mon[35901]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T06:55:44.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:43 smithi136 ceph-mon[35901]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:55:44.176 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:43 smithi073 ceph-mon[32303]: osdmap e77: 11 total, 11 up, 11 in 2024-06-06T06:55:44.176 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:43 smithi073 ceph-mon[32303]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-06T06:55:45.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:44 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/3411428152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:44 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:44 smithi136 ceph-mon[35901]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]': finished 2024-06-06T06:55:45.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:44 smithi136 ceph-mon[35901]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T06:55:45.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:55:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:44 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/3411428152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:44 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:44 smithi170 ceph-mon[36108]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]': finished 2024-06-06T06:55:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:44 smithi170 ceph-mon[36108]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T06:55:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:55:45.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:44 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/3411428152' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:44 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]: dispatch 2024-06-06T06:55:45.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:44 smithi073 ceph-mon[32303]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c8610f25-d950-4f87-b337-29b59b3f71a7"}]': finished 2024-06-06T06:55:45.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:44 smithi073 ceph-mon[32303]: osdmap e78: 12 total, 11 up, 12 in 2024-06-06T06:55:45.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:55:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:45 smithi136 ceph-mon[35901]: from='client.? 172.21.15.170:0/1282410585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:45 smithi136 ceph-mon[35901]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-06-06T06:55:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:45 smithi170 ceph-mon[36108]: from='client.? 172.21.15.170:0/1282410585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:46.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:45 smithi170 ceph-mon[36108]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-06-06T06:55:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:45 smithi073 ceph-mon[32303]: from='client.? 172.21.15.170:0/1282410585' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-06T06:55:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:45 smithi073 ceph-mon[32303]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-06-06T06:55:48.508 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:48 smithi170 ceph-mon[36108]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:55:48.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:48 smithi073 ceph-mon[32303]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:55:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:48 smithi136 ceph-mon[35901]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 335 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-06T06:55:50.571 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:50 smithi170 ceph-mon[36108]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T06:55:50.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:50 smithi073 ceph-mon[32303]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T06:55:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:50 smithi136 ceph-mon[35901]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-06-06T06:55:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:52 smithi170 ceph-mon[36108]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T06:55:52.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:52 smithi073 ceph-mon[32303]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T06:55:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:52 smithi136 ceph-mon[35901]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-06T06:55:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:55:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:53 smithi170 ceph-mon[36108]: Deploying daemon osd.11 on smithi170 2024-06-06T06:55:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:53 smithi073 ceph-mon[32303]: Deploying daemon osd.11 on smithi170 2024-06-06T06:55:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:53 smithi136 ceph-mon[35901]: Deploying daemon osd.11 on smithi170 2024-06-06T06:55:54.557 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:54 smithi170 ceph-mon[36108]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:54 smithi073 ceph-mon[32303]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:54 smithi136 ceph-mon[35901]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:56 smithi170 ceph-mon[36108]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:56.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:56 smithi073 ceph-mon[32303]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:56 smithi136 ceph-mon[35901]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:58.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:58 smithi170 ceph-mon[36108]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:58.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.719 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:55:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:58 smithi073 ceph-mon[32303]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:58.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:55:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:55:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:58 smithi136 ceph-mon[35901]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:55:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:55:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:55:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:00 smithi170 ceph-mon[36108]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:00 smithi073 ceph-mon[32303]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:00 smithi136 ceph-mon[35901]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:02.211 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.210+0000 7f357f7fe700 1 -- 172.21.15.170:0/286779252 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f359006b110 con 0x7f35780a2c20 2024-06-06T06:56:02.211 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 11 on host 'smithi170' 2024-06-06T06:56:02.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 msgr2=0x7f35780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:02.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 0x7f35780a50e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f358400b5c0 tx=0x7f3584005fd0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 msgr2=0x7f35901985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:02.213 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f358000d9b0 tx=0x7f358000dcc0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 shutdown_connections 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f35780a2c20 0x7f35780a50e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f35901013e0 0x7f35901985d0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3590102f80 0x7f359019fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 --2- 172.21.15.170:0/286779252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f35901025e0 0x7f3590198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 >> 172.21.15.170:0/286779252 conn(0x7f35900fc980 msgr2=0x7f359006aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.212+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 shutdown_connections 2024-06-06T06:56:02.214 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:02.213+0000 7f3594a01700 1 -- 172.21.15.170:0/286779252 wait complete. 2024-06-06T06:56:02.214 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.215 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:02.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:02.864 DEBUG:teuthology.orchestra.run.smithi170:osd.11> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.11.service 2024-06-06T06:56:02.866 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-06T06:56:02.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-06T06:56:03.120 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:03.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-mon[36108]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T06:56:03.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-mon[36108]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T06:56:03.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:03.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-mon[36108]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:03.482 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-mon[36108]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:03.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:03 smithi073 ceph-mon[32303]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T06:56:03.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:03 smithi073 ceph-mon[32303]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T06:56:03.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:03 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:03.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:03 smithi073 ceph-mon[32303]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:03.491 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:03 smithi073 ceph-mon[32303]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:03 smithi136 ceph-mon[35901]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-06T06:56:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:03 smithi136 ceph-mon[35901]: osdmap e79: 12 total, 11 up, 12 in 2024-06-06T06:56:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:03 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:03 smithi136 ceph-mon[35901]: from='osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:03 smithi136 ceph-mon[35901]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-06-06T06:56:04.063 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T06:56:03.738+0000 7fc3c1917700 -1 osd.11 0 waiting for initial osdmap 2024-06-06T06:56:04.063 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 06:56:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T06:56:03.755+0000 7fc3b9702700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:04.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:04 smithi170 ceph-mon[36108]: from='osd.11 ' entity='osd.11' 2024-06-06T06:56:04.508 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:56:04.508 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T06:56:04.509 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.509 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.509 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:04.509 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:04 smithi136 ceph-mon[35901]: from='osd.11 ' entity='osd.11' 2024-06-06T06:56:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-06-06T06:56:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: osdmap e80: 12 total, 11 up, 12 in 2024-06-06T06:56:04.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:04.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 336 MiB used, 983 GiB / 983 GiB avail 2024-06-06T06:56:04.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:04 smithi073 ceph-mon[32303]: from='osd.11 ' entity='osd.11' 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1695958272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8102fd0 msgr2=0x7fa2a81053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1695958272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8102fd0 0x7fa2a81053c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa298009990 tx=0x7fa298009ca0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1695958272 shutdown_connections 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1695958272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8108230 0x7fa2a810a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1695958272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8102fd0 0x7fa2a81053c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1695958272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8105900 0x7fa2a8107cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1695958272 >> 172.21.15.73:0/1695958272 conn(0x7fa2a80fc980 msgr2=0x7fa2a80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1695958272 shutdown_connections 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1695958272 wait complete. 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 Processor -- start 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.911+0000 7fa2ad8bf700 1 -- start start 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8102fd0 0x7fa2a8102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8105900 0x7fa2a8100f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810df60 con 0x7fa2a8108230 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810dde0 con 0x7fa2a8102fd0 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa2a810e0e0 con 0x7fa2a8105900 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:04.919 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41790/0 (socket says 172.21.15.73:41790) 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 -- 172.21.15.73:0/1170058898 learned_addr learned my addr 172.21.15.73:0/1170058898 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a6ffd700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8102fd0 0x7fa2a8102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a67fc700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8105900 0x7fa2a8100f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 -- 172.21.15.73:0/1170058898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8105900 msgr2=0x7fa2a8100f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8105900 0x7fa2a8100f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 -- 172.21.15.73:0/1170058898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8102fd0 msgr2=0x7fa2a8102930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8102fd0 0x7fa2a8102930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 -- 172.21.15.73:0/1170058898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2a8101d70 con 0x7fa2a8108230 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2a77fe700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0007ec0 tx=0x7fa2a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a0004bb0 con 0x7fa2a8108230 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2a0004d50 con 0x7fa2a8108230 2024-06-06T06:56:04.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2a81a27c0 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2a00043e0 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.912+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2a81a2c80 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.913+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2a8069180 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.915+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa2a001a030 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.916+0000 7fa2ac8bd700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 0x7fa2940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.916+0000 7fa2a6ffd700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 0x7fa2940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.916+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa2a00ce6a0 con 0x7fa2a8108230 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.916+0000 7fa2a6ffd700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 0x7fa2940a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa298003780 tx=0x7fa298003850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:04.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:04.920+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa2a0021020 con 0x7fa2a8108230 2024-06-06T06:56:05.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.116+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa2a802dca0 con 0x7fa2a8108230 2024-06-06T06:56:05.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.116+0000 7fa2ac8bd700 1 -- 172.21.15.73:0/1170058898 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fa2a0010070 con 0x7fa2a8108230 2024-06-06T06:56:05.118 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 msgr2=0x7fa2940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 0x7fa2940a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa298003780 tx=0x7fa298003850 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 msgr2=0x7fa2a81014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa2a0007ec0 tx=0x7fa2a000d3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 shutdown_connections 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2a8108230 0x7fa2a81014c0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa2a8102fd0 0x7fa2a8102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa2940a2c20 0x7fa2940a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.118+0000 7fa2ad8bf700 1 --2- 172.21.15.73:0/1170058898 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa2a8105900 0x7fa2a8100f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.119+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 >> 172.21.15.73:0/1170058898 conn(0x7fa2a80fc980 msgr2=0x7fa2a81039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:05.120 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.119+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 shutdown_connections 2024-06-06T06:56:05.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:05.119+0000 7fa2ad8bf700 1 -- 172.21.15.73:0/1170058898 wait complete. 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: purged_snaps scrub starts 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: purged_snaps scrub ok 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] boot 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:05 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1170058898' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: purged_snaps scrub starts 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: purged_snaps scrub ok 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] boot 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:05 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1170058898' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T06:56:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: purged_snaps scrub starts 2024-06-06T06:56:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: purged_snaps scrub ok 2024-06-06T06:56:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: osd.11 [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] boot 2024-06-06T06:56:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: osdmap e81: 12 total, 12 up, 12 in 2024-06-06T06:56:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T06:56:05.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:05 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1170058898' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-06T06:56:05.917 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717656964,"num_in_osds":12,"osd_in_since":1717656944,"num_remapped_pgs":0} 2024-06-06T06:56:05.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:56:06.162 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:06.509 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:06 smithi073 ceph-mon[32303]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:06.509 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:06 smithi073 ceph-mon[32303]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T06:56:06.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:06 smithi136 ceph-mon[35901]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:06.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:06 smithi136 ceph-mon[35901]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T06:56:06.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:06 smithi170 ceph-mon[36108]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:06.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:06 smithi170 ceph-mon[36108]: osdmap e82: 12 total, 12 up, 12 in 2024-06-06T06:56:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: Detected new or changed devices on smithi170 2024-06-06T06:56:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 4272M 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:07.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: Detected new or changed devices on smithi170 2024-06-06T06:56:07.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 4272M 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:07.604 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: Detected new or changed devices on smithi170 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 4272M 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:07.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 -- 172.21.15.73:0/3785433225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39806b580 msgr2=0x7fe39810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 --2- 172.21.15.73:0/3785433225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39806b580 0x7fe39810f320 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe39000b0a0 tx=0x7fe39000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 -- 172.21.15.73:0/3785433225 shutdown_connections 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 --2- 172.21.15.73:0/3785433225 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe398108500 0x7fe3981088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 --2- 172.21.15.73:0/3785433225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39806b580 0x7fe39810f320 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 --2- 172.21.15.73:0/3785433225 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 0x7fe39806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 -- 172.21.15.73:0/3785433225 >> 172.21.15.73:0/3785433225 conn(0x7fe398078340 msgr2=0x7fe398078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 -- 172.21.15.73:0/3785433225 shutdown_connections 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.051+0000 7fe3a00db700 1 -- 172.21.15.73:0/3785433225 wait complete. 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 Processor -- start 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 -- start start 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 0x7fe39819e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39806b580 0x7fe39819ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe398112710 con 0x7fe398108500 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe398112590 con 0x7fe39806abc0 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe3a00db700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe398112890 con 0x7fe39806b580 2024-06-06T06:56:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60626/0 (socket says 172.21.15.73:60626) 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 -- 172.21.15.73:0/2914334480 learned_addr learned my addr 172.21.15.73:0/2914334480 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 -- 172.21.15.73:0/2914334480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39806b580 msgr2=0x7fe39819ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39de77700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 0x7fe39819e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39d676700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39806b580 0x7fe39819ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39806b580 0x7fe39819ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 -- 172.21.15.73:0/2914334480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 msgr2=0x7fe39819e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 0x7fe39819e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.052+0000 7fe39e678700 1 -- 172.21.15.73:0/2914334480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe398199210 con 0x7fe398108500 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe39e678700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3900062a0 tx=0x7fe390004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390021070 con 0x7fe398108500 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe390007d00 con 0x7fe398108500 2024-06-06T06:56:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3981994a0 con 0x7fe398108500 2024-06-06T06:56:08.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39001d450 con 0x7fe398108500 2024-06-06T06:56:08.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.053+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3981a57c0 con 0x7fe398108500 2024-06-06T06:56:08.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.054+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe398069180 con 0x7fe398108500 2024-06-06T06:56:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.055+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe390003a00 con 0x7fe398108500 2024-06-06T06:56:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.056+0000 7fe38effd700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.056+0000 7fe39de77700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.056+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fe39000f020 con 0x7fe398108500 2024-06-06T06:56:08.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.056+0000 7fe39de77700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 0x7fe3840a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe398199bf0 tx=0x7fe394008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.059+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe39002a020 con 0x7fe398108500 2024-06-06T06:56:08.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.256+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe39810a9e0 con 0x7fe398108500 2024-06-06T06:56:08.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.257+0000 7fe38effd700 1 -- 172.21.15.73:0/2914334480 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18429 (secure 0 0 0) 0x7fe39009db70 con 0x7fe398108500 2024-06-06T06:56:08.258 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:08.259 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":82,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:56:05.742822+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:56:08.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 msgr2=0x7fe3840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:08.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 0x7fe3840a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe398199bf0 tx=0x7fe394008040 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 msgr2=0x7fe398198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe3900062a0 tx=0x7fe390004ed0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 shutdown_connections 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe398108500 0x7fe398198ab0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39806abc0 0x7fe39819e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe3840a2c20 0x7fe3840a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 --2- 172.21.15.73:0/2914334480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39806b580 0x7fe39819ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 >> 172.21.15.73:0/2914334480 conn(0x7fe398078340 msgr2=0x7fe398100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.259+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 shutdown_connections 2024-06-06T06:56:08.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:08.260+0000 7fe3a00db700 1 -- 172.21.15.73:0/2914334480 wait complete. 2024-06-06T06:56:08.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:08 smithi073 ceph-mon[32303]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:08.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:08 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2914334480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:56:08.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:08 smithi170 ceph-mon[36108]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:08.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:08 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2914334480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:56:08.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:08 smithi136 ceph-mon[35901]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:08.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:08 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2914334480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:56:08.968 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T06:52:17.534158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-06T06:56:08.968 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-06T06:56:09.210 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:10 smithi170 ceph-mon[36108]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:10.605 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:10 smithi073 ceph-mon[32303]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:10.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:10 smithi136 ceph-mon[35901]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- 172.21.15.73:0/1464611493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8107840 msgr2=0x7f75b810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 --2- 172.21.15.73:0/1464611493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8107840 0x7f75b810f3c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f75b000b0d0 tx=0x7f75b000b3e0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- 172.21.15.73:0/1464611493 shutdown_connections 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 --2- 172.21.15.73:0/1464611493 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8102740 0x7f75b8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 --2- 172.21.15.73:0/1464611493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8107840 0x7f75b810f3c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 --2- 172.21.15.73:0/1464611493 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- 172.21.15.73:0/1464611493 >> 172.21.15.73:0/1464611493 conn(0x7f75b80795f0 msgr2=0x7f75b8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:10.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- 172.21.15.73:0/1464611493 shutdown_connections 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- 172.21.15.73:0/1464611493 wait complete. 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 Processor -- start 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.980+0000 7f75bebfe700 1 -- start start 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b819ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8107840 0x7f75b8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112700 con 0x7f75b8102740 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112580 con 0x7f75b81030f0 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f75b8112880 con 0x7f75b8107840 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60648/0 (socket says 172.21.15.73:60648) 2024-06-06T06:56:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 -- 172.21.15.73:0/3275302814 learned_addr learned my addr 172.21.15.73:0/3275302814 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bd19b700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8107840 0x7f75b8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75b7fff700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b819ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 -- 172.21.15.73:0/3275302814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8107840 msgr2=0x7f75b8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8107840 0x7f75b8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 -- 172.21.15.73:0/3275302814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 msgr2=0x7f75b819ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b819ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 -- 172.21.15.73:0/3275302814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b81991a0 con 0x7f75b8102740 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75b7fff700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bc99a700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b530 tx=0x7f75ac00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac015020 con 0x7f75b8102740 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b8199490 con 0x7f75b8102740 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f75ac011ce0 con 0x7f75b8102740 2024-06-06T06:56:10.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.981+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75ac010d40 con 0x7f75b8102740 2024-06-06T06:56:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.982+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b81a5780 con 0x7f75b8102740 2024-06-06T06:56:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.983+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b810aa80 con 0x7f75b8102740 2024-06-06T06:56:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.983+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f75ac018020 con 0x7f75b8102740 2024-06-06T06:56:10.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.984+0000 7f75b5ffb700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 0x7f75a80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:10.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.984+0000 7f75b7fff700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 0x7f75a80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:10.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.984+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f75ac0ce360 con 0x7f75b8102740 2024-06-06T06:56:10.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.984+0000 7f75b7fff700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 0x7f75a80a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75b8102080 tx=0x7f75a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:10.988+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f75ac01f020 con 0x7f75b8102740 2024-06-06T06:56:11.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.186+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f75b8199f20 con 0x7f75b8102740 2024-06-06T06:56:11.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.186+0000 7f75b5ffb700 1 -- 172.21.15.73:0/3275302814 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7f75ac09b760 con 0x7f75b8102740 2024-06-06T06:56:11.188 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 msgr2=0x7f75a80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 0x7f75a80a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f75b8102080 tx=0x7f75a4008040 comp rx=0 tx=0).stop 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 msgr2=0x7f75b819e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b530 tx=0x7f75ac00b8f0 comp rx=0 tx=0).stop 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 shutdown_connections 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f75a80a2c20 0x7f75a80a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:11.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f75b8107840 0x7f75b8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f75b8102740 0x7f75b819e830 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.188+0000 7f75bebfe700 1 --2- 172.21.15.73:0/3275302814 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f75b81030f0 0x7f75b819ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.189+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 >> 172.21.15.73:0/3275302814 conn(0x7f75b80795f0 msgr2=0x7f75b8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.189+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 shutdown_connections 2024-06-06T06:56:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:11.189+0000 7f75bebfe700 1 -- 172.21.15.73:0/3275302814 wait complete. 2024-06-06T06:56:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:11 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3275302814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T06:56:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:11 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3275302814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T06:56:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:11 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3275302814' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.a on smithi073 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.d on smithi073 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.g on smithi073 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.j on smithi073 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.b on smithi136 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.e on smithi136 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.h on smithi136 2024-06-06T06:56:11.940 INFO:tasks.cephadm:Adding mds.k on smithi136 2024-06-06T06:56:11.941 INFO:tasks.cephadm:Adding mds.c on smithi170 2024-06-06T06:56:11.941 INFO:tasks.cephadm:Adding mds.f on smithi170 2024-06-06T06:56:11.941 INFO:tasks.cephadm:Adding mds.i on smithi170 2024-06-06T06:56:11.941 INFO:tasks.cephadm:Adding mds.l on smithi170 2024-06-06T06:56:11.941 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l' 2024-06-06T06:56:12.194 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T06:56:12.566 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:12 smithi170 ceph-mon[36108]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:12.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:12 smithi073 ceph-mon[32303]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:12 smithi136 ceph-mon[35901]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 -- 172.21.15.170:0/4082821059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef8101ef0 msgr2=0x7f4ef8106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 --2- 172.21.15.170:0/4082821059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef8101ef0 0x7f4ef8106790 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000b0a0 tx=0x7f4ef000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 -- 172.21.15.170:0/4082821059 shutdown_connections 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 --2- 172.21.15.170:0/4082821059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef8101ef0 0x7f4ef8106790 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 --2- 172.21.15.170:0/4082821059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef81007c0 0x7f4ef8100be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.872 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 --2- 172.21.15.170:0/4082821059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 0x7f4ef81019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 -- 172.21.15.170:0/4082821059 >> 172.21.15.170:0/4082821059 conn(0x7f4ef80fc100 msgr2=0x7f4ef80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:13.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 -- 172.21.15.170:0/4082821059 shutdown_connections 2024-06-06T06:56:13.873 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.871+0000 7f4efff17700 1 -- 172.21.15.170:0/4082821059 wait complete. 2024-06-06T06:56:13.874 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 Processor -- start 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 -- start start 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 0x7f4ef819f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 0x7f4ef81a60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8114610 con 0x7f4ef8101530 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8114490 con 0x7f4ef8101ef0 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efff17700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8114790 con 0x7f4ef81007c0 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.170:57422/0 (socket says 172.21.15.170:57422) 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 -- 172.21.15.170:0/893958652 learned_addr learned my addr 172.21.15.170:0/893958652 (peer_addr_for_me v2:172.21.15.170:0/0) 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efd4b2700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 0x7f4ef819f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:13.875 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efe4b4700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 0x7f4ef81a60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 -- 172.21.15.170:0/893958652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 msgr2=0x7f4ef81a60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 0x7f4ef81a60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 -- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 msgr2=0x7f4ef819f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 0x7f4ef819f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efdcb3700 1 -- 172.21.15.170:0/893958652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef81a6740 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.872+0000 7f4efe4b4700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 0x7f4ef81a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4efdcb3700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400b560 tx=0x7f4ef400b920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4015020 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ef4011ce0 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef81a6a30 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4010da0 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.873+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81a6fa0 con 0x7f4ef81007c0 2024-06-06T06:56:13.876 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.874+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef8069180 con 0x7f4ef81007c0 2024-06-06T06:56:13.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.876+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4ef4018020 con 0x7f4ef81007c0 2024-06-06T06:56:13.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.876+0000 7f4eeeffd700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 0x7f4ee40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:13.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.877+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f4ef40ce990 con 0x7f4ef81007c0 2024-06-06T06:56:13.877 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.877+0000 7f4efd4b2700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 0x7f4ee40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:13.878 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.877+0000 7f4efd4b2700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 0x7f4ee40a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ea0 tx=0x7f4ee8005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:13.880 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:13.879+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4ef401f020 con 0x7f4ef81007c0 2024-06-06T06:56:14.089 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.089+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}) -- 0x7f4ef8062ad0 con 0x7f4ee40a2c20 2024-06-06T06:56:14.097 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.097+0000 7f4eeeffd700 1 -- 172.21.15.170:0/893958652 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f4ef8062ad0 con 0x7f4ee40a2c20 2024-06-06T06:56:14.098 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mds.all update... 2024-06-06T06:56:14.100 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 msgr2=0x7f4ee40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 0x7f4ee40a50e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8005ea0 tx=0x7f4ee8005e30 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 msgr2=0x7f4ef819eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400b560 tx=0x7f4ef400b920 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 shutdown_connections 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4ee40a2c20 0x7f4ee40a50e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ef81007c0 0x7f4ef819eb30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ef8101ef0 0x7f4ef81a60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 --2- 172.21.15.170:0/893958652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ef8101530 0x7f4ef819f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 >> 172.21.15.170:0/893958652 conn(0x7f4ef80fc100 msgr2=0x7f4ef8102770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 shutdown_connections 2024-06-06T06:56:14.101 INFO:teuthology.orchestra.run.smithi170.stderr:2024-06-06T06:56:14.099+0000 7f4efff17700 1 -- 172.21.15.170:0/893958652 wait complete. 2024-06-06T06:56:14.470 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:14 smithi170 ceph-mon[36108]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:14.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:14.471 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:14 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:14.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:14 smithi073 ceph-mon[32303]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:14.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:14 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:14.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:14 smithi136 ceph-mon[35901]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:14.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:14.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:14 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:14.825 DEBUG:teuthology.orchestra.run.smithi073:mds.a> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.a.service 2024-06-06T06:56:14.829 DEBUG:teuthology.orchestra.run.smithi073:mds.d> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.d.service 2024-06-06T06:56:14.873 DEBUG:teuthology.orchestra.run.smithi073:mds.g> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.g.service 2024-06-06T06:56:14.915 DEBUG:teuthology.orchestra.run.smithi073:mds.j> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.j.service 2024-06-06T06:56:14.959 DEBUG:teuthology.orchestra.run.smithi136:mds.b> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.b.service 2024-06-06T06:56:14.962 DEBUG:teuthology.orchestra.run.smithi136:mds.e> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service 2024-06-06T06:56:15.006 DEBUG:teuthology.orchestra.run.smithi136:mds.h> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.h.service 2024-06-06T06:56:15.048 DEBUG:teuthology.orchestra.run.smithi136:mds.k> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.k.service 2024-06-06T06:56:15.091 DEBUG:teuthology.orchestra.run.smithi170:mds.c> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.c.service 2024-06-06T06:56:15.094 DEBUG:teuthology.orchestra.run.smithi170:mds.f> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.f.service 2024-06-06T06:56:15.137 DEBUG:teuthology.orchestra.run.smithi170:mds.i> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.i.service 2024-06-06T06:56:15.180 DEBUG:teuthology.orchestra.run.smithi170:mds.l> sudo journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.l.service 2024-06-06T06:56:15.223 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-06T06:56:15.224 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:15.460 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:56:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: Saving service mds.all spec with placement smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-06-06T06:56:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:15.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:15 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: Saving service mds.all spec with placement smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:15.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:15 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:15.736 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: Saving service mds.all spec with placement smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:15.737 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:15 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:16.580 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:16 smithi170 ceph-mon[36108]: Deploying daemon mds.l on smithi170 2024-06-06T06:56:16.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:16 smithi170 ceph-mon[36108]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:16.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:16 smithi073 ceph-mon[32303]: Deploying daemon mds.l on smithi170 2024-06-06T06:56:16.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:16 smithi073 ceph-mon[32303]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:16.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:16 smithi136 ceph-mon[35901]: Deploying daemon mds.l on smithi170 2024-06-06T06:56:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:16 smithi136 ceph-mon[35901]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:17.258 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 systemd[1]: Starting Ceph mds.l for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:17.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 -- 172.21.15.73:0/2414646092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102f10 msgr2=0x7fa84c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:17.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 --2- 172.21.15.73:0/2414646092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102f10 0x7fa84c10f330 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa84400b0a0 tx=0x7fa84400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 -- 172.21.15.73:0/2414646092 shutdown_connections 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 --2- 172.21.15.73:0/2414646092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102f10 0x7fa84c10f330 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 --2- 172.21.15.73:0/2414646092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c1084f0 0x7fa84c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 --2- 172.21.15.73:0/2414646092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c102550 0x7fa84c1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 -- 172.21.15.73:0/2414646092 >> 172.21.15.73:0/2414646092 conn(0x7fa84c079650 msgr2=0x7fa84c079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 -- 172.21.15.73:0/2414646092 shutdown_connections 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 -- 172.21.15.73:0/2414646092 wait complete. 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.276+0000 7fa851628700 1 Processor -- start 2024-06-06T06:56:17.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 -- start start 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c102f10 0x7fa84c19eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c1084f0 0x7fa84c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112700 con 0x7fa84c102550 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112580 con 0x7fa84c102f10 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa851628700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa84c112880 con 0x7fa84c1084f0 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46812/0 (socket says 172.21.15.73:46812) 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 -- 172.21.15.73:0/93202060 learned_addr learned my addr 172.21.15.73:0/93202060 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84b7fe700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c1084f0 0x7fa84c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84a7fc700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c102f10 0x7fa84c19eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:17.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 -- 172.21.15.73:0/93202060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c1084f0 msgr2=0x7fa84c198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c1084f0 0x7fa84c198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 -- 172.21.15.73:0/93202060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c102f10 msgr2=0x7fa84c19eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c102f10 0x7fa84c19eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 -- 172.21.15.73:0/93202060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84c199180 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.277+0000 7fa84affd700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00b530 tx=0x7fa83c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.278+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c015020 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.278+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83c011ce0 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.278+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa84c199470 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.278+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83c010e40 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.278+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa84c1a5730 con 0x7fa84c102550 2024-06-06T06:56:17.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.279+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa84c103ba0 con 0x7fa84c102550 2024-06-06T06:56:17.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.280+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa83c018020 con 0x7fa84c102550 2024-06-06T06:56:17.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.281+0000 7fa833fff700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 0x7fa8340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:17.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.281+0000 7fa84a7fc700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 0x7fa8340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:17.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.281+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa83c0ce410 con 0x7fa84c102550 2024-06-06T06:56:17.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.281+0000 7fa84a7fc700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 0x7fa8340a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa840006fd0 tx=0x7fa840008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:17.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.285+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa83c01f020 con 0x7fa84c102550 2024-06-06T06:56:17.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.533+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa84c199f50 con 0x7fa84c102550 2024-06-06T06:56:17.534 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.534+0000 7fa833fff700 1 -- 172.21.15.73:0/93202060 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7fa83c010380 con 0x7fa84c102550 2024-06-06T06:56:17.535 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:17.535 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 msgr2=0x7fa8340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 0x7fa8340a50e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa840006fd0 tx=0x7fa840008040 comp rx=0 tx=0).stop 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 msgr2=0x7fa84c19e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa83c00b530 tx=0x7fa83c00b840 comp rx=0 tx=0).stop 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 shutdown_connections 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fa8340a2c20 0x7fa8340a50e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa84c1084f0 0x7fa84c198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa84c102550 0x7fa84c19e860 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.537 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 --2- 172.21.15.73:0/93202060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa84c102f10 0x7fa84c19eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:17.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 >> 172.21.15.73:0/93202060 conn(0x7fa84c079650 msgr2=0x7fa84c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:17.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 shutdown_connections 2024-06-06T06:56:17.538 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:17.536+0000 7fa851628700 1 -- 172.21.15.73:0/93202060 wait complete. 2024-06-06T06:56:17.538 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 2 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 podman[66027]: 2024-06-06 06:56:17.257701237 +0000 UTC m=+0.109847079 container create badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 podman[66027]: 2024-06-06 06:56:17.165140294 +0000 UTC m=+0.017286152 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 podman[66027]: 2024-06-06 06:56:17.477363333 +0000 UTC m=+0.329509187 container init badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 podman[66027]: 2024-06-06 06:56:17.48031142 +0000 UTC m=+0.332457279 container start badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True) 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 ceph-mds[66046]: -- 172.21.15.170:0/599432418 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563df3a85a40 con 0x563df3a59000 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l[66042]: starting mds.l at 2024-06-06T06:56:17.578 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 bash[66027]: badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 2024-06-06T06:56:17.847 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 06:56:17 smithi170 systemd[1]: Started Ceph mds.l for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:18.651 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:18.651 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6832/3776485988,v1:172.21.15.170:6833/3776485988] up:boot 2024-06-06T06:56:18.651 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: fsmap 1 up:standby 2024-06-06T06:56:18.651 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T06:56:18.651 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/93202060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:18.652 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:18 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6832/3776485988,v1:172.21.15.170:6833/3776485988] up:boot 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: fsmap 1 up:standby 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/93202060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:18.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:18 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6832/3776485988,v1:172.21.15.170:6833/3776485988] up:boot 2024-06-06T06:56:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: fsmap 1 up:standby 2024-06-06T06:56:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/93202060' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:18 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:19.698 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:19 smithi136 ceph-mon[35901]: Deploying daemon mds.k on smithi136 2024-06-06T06:56:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:19 smithi073 ceph-mon[32303]: Deploying daemon mds.k on smithi136 2024-06-06T06:56:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:19 smithi170 ceph-mon[36108]: Deploying daemon mds.k on smithi136 2024-06-06T06:56:20.052 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:19 smithi136 systemd[1]: Starting Ceph mds.k for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:20.347 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 podman[65023]: 2024-06-06 06:56:20.092429496 +0000 UTC m=+0.112597019 container create 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T06:56:20.347 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 podman[65023]: 2024-06-06 06:56:19.997207746 +0000 UTC m=+0.017375247 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:20.347 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 podman[65023]: 2024-06-06 06:56:20.346439706 +0000 UTC m=+0.366607219 container init 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:56:20.362 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:20.619 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:20.742 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:20 smithi073 ceph-mon[32303]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:20 smithi136 ceph-mon[35901]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 podman[65023]: 2024-06-06 06:56:20.349471371 +0000 UTC m=+0.369638871 container start 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 ceph-mds[65042]: -- 172.21.15.136:0/588289799 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ded1a85a40 con 0x55ded1a58800 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k[65038]: starting mds.k at 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 bash[65023]: 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 2024-06-06T06:56:20.802 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 06:56:20 smithi136 systemd[1]: Started Ceph mds.k for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:20 smithi170 ceph-mon[36108]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:21.748 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: Deploying daemon mds.j on smithi073 2024-06-06T06:56:21.749 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:21.749 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6832/464039391,v1:172.21.15.136:6833/464039391] up:boot 2024-06-06T06:56:21.749 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: fsmap 2 up:standby 2024-06-06T06:56:21.749 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:21 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T06:56:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: Deploying daemon mds.j on smithi073 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6832/464039391,v1:172.21.15.136:6833/464039391] up:boot 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: fsmap 2 up:standby 2024-06-06T06:56:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:21 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: Deploying daemon mds.j on smithi073 2024-06-06T06:56:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6832/464039391,v1:172.21.15.136:6833/464039391] up:boot 2024-06-06T06:56:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: fsmap 2 up:standby 2024-06-06T06:56:21.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:21 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 -- 172.21.15.73:0/2937471912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388101440 msgr2=0x7f938810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 --2- 172.21.15.73:0/2937471912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388101440 0x7f938810f330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f938000b0a0 tx=0x7f938000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 -- 172.21.15.73:0/2937471912 shutdown_connections 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 --2- 172.21.15.73:0/2937471912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388108510 0x7f93881088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 --2- 172.21.15.73:0/2937471912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388101440 0x7f938810f330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 --2- 172.21.15.73:0/2937471912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 0x7f9388100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 -- 172.21.15.73:0/2937471912 >> 172.21.15.73:0/2937471912 conn(0x7f9388077a00 msgr2=0x7f93880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 -- 172.21.15.73:0/2937471912 shutdown_connections 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.815+0000 7f938ff16700 1 -- 172.21.15.73:0/2937471912 wait complete. 2024-06-06T06:56:23.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 Processor -- start 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 -- start start 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 0x7f938819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388101440 0x7f938819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9388112740 con 0x7f9388108510 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f93881125c0 con 0x7f9388100a80 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.816+0000 7f938ff16700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93881128c0 con 0x7f9388101440 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46838/0 (socket says 172.21.15.73:46838) 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 -- 172.21.15.73:0/4138514063 learned_addr learned my addr 172.21.15.73:0/4138514063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938dcb2700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 0x7f938819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938d4b1700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388101440 0x7f938819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:23.821 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 -- 172.21.15.73:0/4138514063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388101440 msgr2=0x7f938819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388101440 0x7f938819ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 -- 172.21.15.73:0/4138514063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 msgr2=0x7f938819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 0x7f938819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 -- 172.21.15.73:0/4138514063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93881991e0 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938e4b3700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9380006070 tx=0x7f9380004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380021070 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9388199470 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9380007df0 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938001d450 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.817+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a5790 con 0x7f9388108510 2024-06-06T06:56:23.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.818+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938810a9f0 con 0x7f9388108510 2024-06-06T06:56:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.820+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f93800040b0 con 0x7f9388108510 2024-06-06T06:56:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.821+0000 7f937effd700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 0x7f93740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.821+0000 7f938dcb2700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 0x7f93740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.821+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f938000f020 con 0x7f9388108510 2024-06-06T06:56:23.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.821+0000 7f938dcb2700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 0x7f93740a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9388199bc0 tx=0x7f9378008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:23.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:23.824+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f938002a020 con 0x7f9388108510 2024-06-06T06:56:24.092 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:24 smithi073 systemd[1]: Starting Ceph mds.j for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:24.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.090+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9388069180 con 0x7f9388108510 2024-06-06T06:56:24.093 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:24.093 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T06:56:24.093 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.091+0000 7f937effd700 1 -- 172.21.15.73:0/4138514063 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f938009db10 con 0x7f9388108510 2024-06-06T06:56:24.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 msgr2=0x7f93740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:24.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 0x7f93740a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9388199bc0 tx=0x7f9378008040 comp rx=0 tx=0).stop 2024-06-06T06:56:24.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 msgr2=0x7f9388198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9380006070 tx=0x7f9380004ef0 comp rx=0 tx=0).stop 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 shutdown_connections 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9388108510 0x7f9388198a80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9388100a80 0x7f938819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f93740a2c20 0x7f93740a50e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 --2- 172.21.15.73:0/4138514063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9388101440 0x7f938819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 >> 172.21.15.73:0/4138514063 conn(0x7f9388077a00 msgr2=0x7f93880fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 shutdown_connections 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:24.093+0000 7f938ff16700 1 -- 172.21.15.73:0/4138514063 wait complete. 2024-06-06T06:56:24.095 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 3 2024-06-06T06:56:24.448 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:24 smithi073 ceph-mon[32303]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:24.448 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:24 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4138514063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:24 smithi170 ceph-mon[36108]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:24.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:24 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4138514063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:24 smithi136 ceph-mon[35901]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:24.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:24 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4138514063' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:25.231 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 podman[72691]: 2024-06-06 06:56:25.005140249 +0000 UTC m=+0.017306241 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:25.231 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 podman[72691]: 2024-06-06 06:56:25.116975499 +0000 UTC m=+0.129141484 container create 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5) 2024-06-06T06:56:25.775 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 podman[72691]: 2024-06-06 06:56:25.336691415 +0000 UTC m=+0.348857407 container init 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True) 2024-06-06T06:56:25.775 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 podman[72691]: 2024-06-06 06:56:25.339395245 +0000 UTC m=+0.351561238 container start 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-06-06T06:56:25.775 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 ceph-mds[72711]: -- 172.21.15.73:0/3629707076 <== mon.0 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5564a8003a40 con 0x5564a7fd6800 2024-06-06T06:56:25.776 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j[72707]: starting mds.j at 2024-06-06T06:56:25.776 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 bash[72691]: 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 2024-06-06T06:56:25.776 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 06:56:25 smithi073 systemd[1]: Started Ceph mds.j for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:26.415 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:26.415 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.416 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.416 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.416 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:26.416 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:26.416 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:26 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:26.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:26.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:26 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:26.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:26 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:26.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:27.334 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:27 smithi073 ceph-mon[32303]: Deploying daemon mds.d on smithi073 2024-06-06T06:56:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:27 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.73:6834/2227329730,v1:172.21.15.73:6835/2227329730] up:boot 2024-06-06T06:56:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:27 smithi073 ceph-mon[32303]: fsmap 3 up:standby 2024-06-06T06:56:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:27 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T06:56:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:27 smithi136 ceph-mon[35901]: Deploying daemon mds.d on smithi073 2024-06-06T06:56:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:27 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.73:6834/2227329730,v1:172.21.15.73:6835/2227329730] up:boot 2024-06-06T06:56:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:27 smithi136 ceph-mon[35901]: fsmap 3 up:standby 2024-06-06T06:56:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:27 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T06:56:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:27 smithi170 ceph-mon[36108]: Deploying daemon mds.d on smithi073 2024-06-06T06:56:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:27 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.73:6834/2227329730,v1:172.21.15.73:6835/2227329730] up:boot 2024-06-06T06:56:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:27 smithi170 ceph-mon[36108]: fsmap 3 up:standby 2024-06-06T06:56:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:27 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T06:56:28.706 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:28 smithi073 ceph-mon[32303]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:28.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:28 smithi136 ceph-mon[35901]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:28 smithi170 ceph-mon[36108]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:29.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 -- 172.21.15.73:0/1593599112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 msgr2=0x7f437410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:29.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 --2- 172.21.15.73:0/1593599112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437410e210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f436c00b0a0 tx=0x7f436c00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 -- 172.21.15.73:0/1593599112 shutdown_connections 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 --2- 172.21.15.73:0/1593599112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437410e210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 --2- 172.21.15.73:0/1593599112 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f43741077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 --2- 172.21.15.73:0/1593599112 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 0x7f43741017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 -- 172.21.15.73:0/1593599112 >> 172.21.15.73:0/1593599112 conn(0x7f43740782c0 msgr2=0x7f43740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 -- 172.21.15.73:0/1593599112 shutdown_connections 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.614+0000 7f437bccb700 1 -- 172.21.15.73:0/1593599112 wait complete. 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 Processor -- start 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 -- start start 2024-06-06T06:56:29.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 0x7f437419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f4374198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f43741114d0 con 0x7f4374101d30 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4374111350 con 0x7f43741073d0 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f437bccb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4374111650 con 0x7f4374101390 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f4379266700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f4379266700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48894/0 (socket says 172.21.15.73:48894) 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.615+0000 7f4379266700 1 -- 172.21.15.73:0/2444693261 learned_addr learned my addr 172.21.15.73:0/2444693261 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379a67700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 0x7f437419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 -- 172.21.15.73:0/2444693261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 msgr2=0x7f437419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:29.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 0x7f437419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 -- 172.21.15.73:0/2444693261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 msgr2=0x7f4374198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f437a268700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f4374198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f4374198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 -- 172.21.15.73:0/2444693261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4374198fe0 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f437a268700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f4374198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f4379266700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f436400c920 tx=0x7f436400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364007b00 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43741992d0 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4364004ce0 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4364005560 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.616+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43741a5590 con 0x7f4374101d30 2024-06-06T06:56:29.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.617+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4374069180 con 0x7f4374101d30 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.618+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4364004750 con 0x7f4374101d30 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.619+0000 7f436affd700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 0x7f43600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.619+0000 7f4379a67700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 0x7f43600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.619+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f43640cec00 con 0x7f4374101d30 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.619+0000 7f4379a67700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 0x7f43600a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43741999c0 tx=0x7f437000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:29.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.622+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4364014030 con 0x7f4374101d30 2024-06-06T06:56:29.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.866+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4374199db0 con 0x7f4374101d30 2024-06-06T06:56:29.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.867+0000 7f436affd700 1 -- 172.21.15.73:0/2444693261 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f436409c000 con 0x7f4374101d30 2024-06-06T06:56:29.868 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:29.868 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.869+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 msgr2=0x7f43600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.869+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 0x7f43600a50e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f43741999c0 tx=0x7f437000b5b0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.869+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 msgr2=0x7f437419ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.869+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f436400c920 tx=0x7f436400cce0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 shutdown_connections 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f43600a2c20 0x7f43600a50e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4374101390 0x7f437419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f43741073d0 0x7f4374198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 --2- 172.21.15.73:0/2444693261 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4374101d30 0x7f437419ec00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 >> 172.21.15.73:0/2444693261 conn(0x7f43740782c0 msgr2=0x7f43741006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:29.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 shutdown_connections 2024-06-06T06:56:29.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:29.870+0000 7f437bccb700 1 -- 172.21.15.73:0/2444693261 wait complete. 2024-06-06T06:56:29.872 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 4 2024-06-06T06:56:30.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2444693261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:30 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2444693261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:30.687 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:30 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2444693261' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:30.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:30 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:31.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:31 smithi136 ceph-mon[35901]: Deploying daemon mds.e on smithi136 2024-06-06T06:56:31.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:31 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.73:6836/38418017,v1:172.21.15.73:6837/38418017] up:boot 2024-06-06T06:56:31.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:31 smithi136 ceph-mon[35901]: fsmap 4 up:standby 2024-06-06T06:56:31.657 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:31 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T06:56:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:31 smithi073 ceph-mon[32303]: Deploying daemon mds.e on smithi136 2024-06-06T06:56:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:31 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.73:6836/38418017,v1:172.21.15.73:6837/38418017] up:boot 2024-06-06T06:56:31.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:31 smithi073 ceph-mon[32303]: fsmap 4 up:standby 2024-06-06T06:56:31.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:31 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T06:56:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:31 smithi170 ceph-mon[36108]: Deploying daemon mds.e on smithi136 2024-06-06T06:56:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:31 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.73:6836/38418017,v1:172.21.15.73:6837/38418017] up:boot 2024-06-06T06:56:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:31 smithi170 ceph-mon[36108]: fsmap 4 up:standby 2024-06-06T06:56:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:31 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T06:56:32.190 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 systemd[1]: Starting Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:32.460 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:32 smithi136 ceph-mon[35901]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:32.460 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 podman[65333]: 2024-06-06 06:56:32.237774954 +0000 UTC m=+0.017219600 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:32.460 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 podman[65333]: 2024-06-06 06:56:32.341731743 +0000 UTC m=+0.121176382 container create b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T06:56:32.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:32.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:32 smithi170 ceph-mon[36108]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:32 smithi073 ceph-mon[32303]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 podman[65333]: 2024-06-06 06:56:32.569463534 +0000 UTC m=+0.348908180 container init b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True) 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 podman[65333]: 2024-06-06 06:56:32.572262258 +0000 UTC m=+0.351706893 container start b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 ceph-mds[65352]: -- 172.21.15.136:0/1594079709 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556c19a5fa40 con 0x556c19a32800 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: starting mds.e at 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 bash[65333]: b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 2024-06-06T06:56:33.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 06:56:32 smithi136 systemd[1]: Started Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: Deploying daemon mds.f on smithi170 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:33.972 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/687158910,v1:172.21.15.136:6835/687158910] up:boot 2024-06-06T06:56:33.973 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: fsmap 5 up:standby 2024-06-06T06:56:33.973 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:33 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: Deploying daemon mds.f on smithi170 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:33.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/687158910,v1:172.21.15.136:6835/687158910] up:boot 2024-06-06T06:56:33.999 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: fsmap 5 up:standby 2024-06-06T06:56:33.999 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:33 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T06:56:34.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: Deploying daemon mds.f on smithi170 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/687158910,v1:172.21.15.136:6835/687158910] up:boot 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: fsmap 5 up:standby 2024-06-06T06:56:34.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:33 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T06:56:34.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 -- 172.21.15.73:0/2830923919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 msgr2=0x7f5808107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 --2- 172.21.15.73:0/2830923919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f5808107110 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f57fc0099c0 tx=0x7f57fc009cd0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 -- 172.21.15.73:0/2830923919 shutdown_connections 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 --2- 172.21.15.73:0/2830923919 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 0x7f580810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 --2- 172.21.15.73:0/2830923919 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f5808102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 --2- 172.21.15.73:0/2830923919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f5808107110 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 -- 172.21.15.73:0/2830923919 >> 172.21.15.73:0/2830923919 conn(0x7f58080795f0 msgr2=0x7f5808079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 -- 172.21.15.73:0/2830923919 shutdown_connections 2024-06-06T06:56:34.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.553+0000 7f580e65c700 1 -- 172.21.15.73:0/2830923919 wait complete. 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 Processor -- start 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 -- start start 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f580819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 0x7f58081988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5808112620 con 0x7f58081030f0 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f58081124a0 con 0x7f5808102740 2024-06-06T06:56:34.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f580e65c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58081127a0 con 0x7f5808107840 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.554+0000 7f58077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48910/0 (socket says 172.21.15.73:48910) 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 -- 172.21.15.73:0/2892975384 learned_addr learned my addr 172.21.15.73:0/2892975384 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f580cbf9700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 0x7f58081988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f5807fff700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f580819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 -- 172.21.15.73:0/2892975384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 msgr2=0x7f58081988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 0x7f58081988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 -- 172.21.15.73:0/2892975384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 msgr2=0x7f580819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:34.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f580819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 -- 172.21.15.73:0/2892975384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5808199040 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f5807fff700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f580819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58077fe700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006010 tx=0x7f57fc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc026070 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57fc00f460 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57fc009e60 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58081992d0 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.555+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58081a55f0 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.556+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5808069180 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.557+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f57fc029070 con 0x7f58081030f0 2024-06-06T06:56:34.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.558+0000 7f58057fa700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 0x7f57f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:34.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.558+0000 7f5807fff700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 0x7f57f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:34.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.558+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f57fc0cf670 con 0x7f58081030f0 2024-06-06T06:56:34.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.558+0000 7f5807fff700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 0x7f57f00a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005950 tx=0x7f57f8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:34.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.561+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f57fc030080 con 0x7f58081030f0 2024-06-06T06:56:34.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.807+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f580802dca0 con 0x7f58081030f0 2024-06-06T06:56:34.808 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.808+0000 7f58057fa700 1 -- 172.21.15.73:0/2892975384 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4690 (secure 0 0 0) 0x7f57fc013070 con 0x7f58081030f0 2024-06-06T06:56:34.809 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:34.809 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 msgr2=0x7f57f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 0x7f57f00a50e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f57f8005950 tx=0x7f57f8005f50 comp rx=0 tx=0).stop 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 msgr2=0x7f580819ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f57fc006010 tx=0x7f57fc0045e0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 shutdown_connections 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f57f00a2c20 0x7f57f00a50e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5808107840 0x7f58081988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.811 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5808102740 0x7f580819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 --2- 172.21.15.73:0/2892975384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58081030f0 0x7f580819ec60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 >> 172.21.15.73:0/2892975384 conn(0x7f58080795f0 msgr2=0x7f5808108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 shutdown_connections 2024-06-06T06:56:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:34.810+0000 7f580e65c700 1 -- 172.21.15.73:0/2892975384 wait complete. 2024-06-06T06:56:34.812 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2024-06-06T06:56:35.066 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:34 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2892975384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:35.182 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:34 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2892975384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:35.221 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:34 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2892975384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:35.847 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 06:56:35 smithi170 systemd[1]: Started Ceph mds.f for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:36.158 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:36.159 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:35 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:36.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:36.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.167 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:36.168 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:36.168 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:35 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:36.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:36.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:35 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:37.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:36 smithi073 ceph-mon[32303]: Deploying daemon mds.c on smithi170 2024-06-06T06:56:37.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:36 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:boot 2024-06-06T06:56:37.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:36 smithi073 ceph-mon[32303]: fsmap 6 up:standby 2024-06-06T06:56:37.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:36 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T06:56:37.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:36 smithi170 ceph-mon[36108]: Deploying daemon mds.c on smithi170 2024-06-06T06:56:37.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:36 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:boot 2024-06-06T06:56:37.181 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:36 smithi170 ceph-mon[36108]: fsmap 6 up:standby 2024-06-06T06:56:37.182 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:36 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T06:56:37.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:36 smithi136 ceph-mon[35901]: Deploying daemon mds.c on smithi170 2024-06-06T06:56:37.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:36 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:boot 2024-06-06T06:56:37.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:36 smithi136 ceph-mon[35901]: fsmap 6 up:standby 2024-06-06T06:56:37.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:36 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T06:56:37.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:37.771 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:37.839 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:37 smithi170 systemd[1]: Starting Ceph mds.c for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:38.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:37 smithi170 ceph-mon[36108]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:38.095 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:37 smithi170 podman[66648]: 2024-06-06 06:56:37.741476017 +0000 UTC m=+0.017348366 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:38.095 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:37 smithi170 podman[66648]: 2024-06-06 06:56:37.858499531 +0000 UTC m=+0.134371904 container create da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T06:56:38.095 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 podman[66648]: 2024-06-06 06:56:38.094374258 +0000 UTC m=+0.370246588 container init da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:56:38.161 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:37 smithi073 ceph-mon[32303]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:38.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:37 smithi136 ceph-mon[35901]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:38.347 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 podman[66648]: 2024-06-06 06:56:38.097483253 +0000 UTC m=+0.373355584 container start da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-06T06:56:38.347 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 ceph-mds[66668]: -- 172.21.15.170:0/3063687462 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559f29849a40 con 0x559f2981c800 2024-06-06T06:56:38.347 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: starting mds.c at 2024-06-06T06:56:38.347 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 bash[66648]: da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 2024-06-06T06:56:38.348 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 06:56:38 smithi170 systemd[1]: Started Ceph mds.c for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 -- 172.21.15.73:0/1690536338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 msgr2=0x7f98f01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 --2- 172.21.15.73:0/1690536338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f01077e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f98e0009990 tx=0x7f98e0009ca0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 -- 172.21.15.73:0/1690536338 shutdown_connections 2024-06-06T06:56:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 --2- 172.21.15.73:0/1690536338 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 0x7f98f010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 --2- 172.21.15.73:0/1690536338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f01077e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 --2- 172.21.15.73:0/1690536338 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 -- 172.21.15.73:0/1690536338 >> 172.21.15.73:0/1690536338 conn(0x7f98f0078340 msgr2=0x7f98f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 -- 172.21.15.73:0/1690536338 shutdown_connections 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.439+0000 7f98f6434700 1 -- 172.21.15.73:0/1690536338 wait complete. 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 Processor -- start 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 -- start start 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 0x7f98f019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0111520 con 0x7f98f0107400 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.440+0000 7f98f6434700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01113a0 con 0x7f98f0101d50 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f6434700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98f01116a0 con 0x7f98f0101390 2024-06-06T06:56:39.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54172/0 (socket says 172.21.15.73:54172) 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 -- 172.21.15.73:0/1146825863 learned_addr learned my addr 172.21.15.73:0/1146825863 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98ef7fe700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 0x7f98f019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98effff700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 -- 172.21.15.73:0/1146825863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 msgr2=0x7f98f019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 -- 172.21.15.73:0/1146825863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 msgr2=0x7f98f019ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 0x7f98f019ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 -- 172.21.15.73:0/1146825863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f0198fc0 con 0x7f98f0107400 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98effff700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:56:39.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f49d1700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98e800ef30 tx=0x7f98e800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e80092e0 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f98e8004bb0 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f0199250 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8004060 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.441+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f01a5570 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.442+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f0069180 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.443+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f98e801b030 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.444+0000 7f98ed7fa700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 0x7f98d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.444+0000 7f98effff700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 0x7f98d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.444+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f98e80cefe0 con 0x7f98f0107400 2024-06-06T06:56:39.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.444+0000 7f98effff700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 0x7f98d80a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098e0 tx=0x7f98e001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:39.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.448+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f98e8022020 con 0x7f98f0107400 2024-06-06T06:56:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: Deploying daemon mds.h on smithi136 2024-06-06T06:56:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:boot 2024-06-06T06:56:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: fsmap 7 up:standby 2024-06-06T06:56:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:39 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: Deploying daemon mds.h on smithi136 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:boot 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: fsmap 7 up:standby 2024-06-06T06:56:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:39 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T06:56:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: Deploying daemon mds.h on smithi136 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:boot 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: fsmap 7 up:standby 2024-06-06T06:56:39.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:39 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T06:56:39.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.684+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98f01a5740 con 0x7f98f0107400 2024-06-06T06:56:39.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.685+0000 7f98ed7fa700 1 -- 172.21.15.73:0/1146825863 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6346 (secure 0 0 0) 0x7f98e8010040 con 0x7f98f0107400 2024-06-06T06:56:39.687 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:39.687 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 msgr2=0x7f98d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 0x7f98d80a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f98e00098e0 tx=0x7f98e001a040 comp rx=0 tx=0).stop 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 msgr2=0x7f98f0198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f98e800ef30 tx=0x7f98e800c3a0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 shutdown_connections 2024-06-06T06:56:39.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f98d80a2c20 0x7f98d80a50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98f0101390 0x7f98f019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98f0107400 0x7f98f0198860 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 --2- 172.21.15.73:0/1146825863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98f0101d50 0x7f98f019ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 >> 172.21.15.73:0/1146825863 conn(0x7f98f0078340 msgr2=0x7f98f01006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 shutdown_connections 2024-06-06T06:56:39.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:39.687+0000 7f98f6434700 1 -- 172.21.15.73:0/1146825863 wait complete. 2024-06-06T06:56:39.692 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 8 2024-06-06T06:56:40.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:40 smithi073 ceph-mon[32303]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:40.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:40 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1146825863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:40.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:40 smithi136 ceph-mon[35901]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:40.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:40 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1146825863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:40.552 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 systemd[1]: Starting Ceph mds.h for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:40 smithi170 ceph-mon[36108]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:40.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:40 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1146825863' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:40.899 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 podman[65645]: 2024-06-06 06:56:40.480291268 +0000 UTC m=+0.017009795 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:40.899 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 podman[65645]: 2024-06-06 06:56:40.604640752 +0000 UTC m=+0.141359254 container create a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T06:56:40.899 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 podman[65645]: 2024-06-06 06:56:40.898956436 +0000 UTC m=+0.435674940 container init a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc) 2024-06-06T06:56:41.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 podman[65645]: 2024-06-06 06:56:40.901783153 +0000 UTC m=+0.438501662 container start a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-06T06:56:41.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 ceph-mds[65664]: -- 172.21.15.136:0/2731278326 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5647ceb83a40 con 0x5647ceb57000 2024-06-06T06:56:41.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h[65660]: starting mds.h at 2024-06-06T06:56:41.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:40 smithi136 bash[65645]: a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a 2024-06-06T06:56:41.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 06:56:41 smithi136 systemd[1]: Started Ceph mds.h for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: Deploying daemon mds.i on smithi170 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:boot 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: fsmap 8 up:standby 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T06:56:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:42 smithi136 ceph-mon[35901]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: Deploying daemon mds.i on smithi170 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:boot 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: fsmap 8 up:standby 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T06:56:42.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:42 smithi170 ceph-mon[36108]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:42.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:42.385 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.385 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.385 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:42.385 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: Deploying daemon mds.i on smithi170 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:boot 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: fsmap 8 up:standby 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T06:56:42.386 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:42 smithi073 ceph-mon[32303]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:42.607 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:43.347 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 systemd[1]: Starting Ceph mds.i for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:43.702 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 podman[66950]: 2024-06-06 06:56:43.326248472 +0000 UTC m=+0.017393618 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:43.702 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 podman[66950]: 2024-06-06 06:56:43.440427768 +0000 UTC m=+0.131572948 container create 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-06T06:56:43.702 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 podman[66950]: 2024-06-06 06:56:43.701758677 +0000 UTC m=+0.392903805 container init 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-06T06:56:44.059 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 podman[66950]: 2024-06-06 06:56:43.70487877 +0000 UTC m=+0.396023898 container start 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T06:56:44.059 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 ceph-mds[66968]: -- 172.21.15.170:0/3667805410 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c8e083a40 con 0x559c8e056c00 2024-06-06T06:56:44.059 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: starting mds.i at 2024-06-06T06:56:44.059 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 bash[66950]: 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe 2024-06-06T06:56:44.059 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:56:43 smithi170 systemd[1]: Started Ceph mds.i for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 -- 172.21.15.73:0/2213802095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec106750 msgr2=0x7f72ec10e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 --2- 172.21.15.73:0/2213802095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec106750 0x7f72ec10e2d0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f72e400b0d0 tx=0x7f72e400b3e0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 -- 172.21.15.73:0/2213802095 shutdown_connections 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 --2- 172.21.15.73:0/2213802095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec106750 0x7f72ec10e2d0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 --2- 172.21.15.73:0/2213802095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec101df0 0x7f72ec1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 --2- 172.21.15.73:0/2213802095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec1027a0 0x7f72ec106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 -- 172.21.15.73:0/2213802095 >> 172.21.15.73:0/2213802095 conn(0x7f72ec077a00 msgr2=0x7f72ec0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 -- 172.21.15.73:0/2213802095 shutdown_connections 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.369+0000 7f72f137b700 1 -- 172.21.15.73:0/2213802095 wait complete. 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 Processor -- start 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 -- start start 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec101df0 0x7f72ec1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec106750 0x7f72ec1a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec1115b0 con 0x7f72ec1027a0 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec111430 con 0x7f72ec106750 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72f137b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f72ec111730 con 0x7f72ec101df0 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72ea7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:44.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72ea7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54196/0 (socket says 172.21.15.73:54196) 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72ea7fc700 1 -- 172.21.15.73:0/4250465649 learned_addr learned my addr 172.21.15.73:0/4250465649 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.370+0000 7f72eaffd700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec101df0 0x7f72ec1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 -- 172.21.15.73:0/4250465649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec101df0 msgr2=0x7f72ec1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec101df0 0x7f72ec1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72eb7fe700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec106750 0x7f72ec1a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 -- 172.21.15.73:0/4250465649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec106750 msgr2=0x7f72ec1a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec106750 0x7f72ec1a4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 -- 172.21.15.73:0/4250465649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72ec1a4e90 con 0x7f72ec1027a0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72ea7fc700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b550 tx=0x7f72dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc015020 con 0x7f72ec1027a0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f72dc011ce0 con 0x7f72ec1027a0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72ec1a50c0 con 0x7f72ec1027a0 2024-06-06T06:56:44.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc010d60 con 0x7f72ec1027a0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.371+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72ec1a55b0 con 0x7f72ec1027a0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.372+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72ec101090 con 0x7f72ec1027a0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.373+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f72dc018020 con 0x7f72ec1027a0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.374+0000 7f72d3fff700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 0x7f72d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.374+0000 7f72eaffd700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 0x7f72d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.374+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f72dc0ce9d0 con 0x7f72ec1027a0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.374+0000 7f72eaffd700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 0x7f72d40a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005ea0 tx=0x7f72e0005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:44.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.377+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f72dc01f020 con 0x7f72ec1027a0 2024-06-06T06:56:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:44.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:44.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:44 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:44.602 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:44.603 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:44 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:44.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.632+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f72ec02dca0 con 0x7f72ec1027a0 2024-06-06T06:56:44.633 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.633+0000 7f72d3fff700 1 -- 172.21.15.73:0/4250465649 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7f72dc09bdd0 con 0x7f72ec1027a0 2024-06-06T06:56:44.633 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:44.634 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24404,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 msgr2=0x7f72d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 0x7f72d40a50e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005ea0 tx=0x7f72e0005e30 comp rx=0 tx=0).stop 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 msgr2=0x7f72ec199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00b550 tx=0x7f72dc00b910 comp rx=0 tx=0).stop 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 shutdown_connections 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f72d40a2c20 0x7f72d40a50e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f72ec101df0 0x7f72ec1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f72ec106750 0x7f72ec1a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 --2- 172.21.15.73:0/4250465649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72ec1027a0 0x7f72ec199e10 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:44.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 >> 172.21.15.73:0/4250465649 conn(0x7f72ec077a00 msgr2=0x7f72ec104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:44.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 shutdown_connections 2024-06-06T06:56:44.637 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:44.635+0000 7f72f137b700 1 -- 172.21.15.73:0/4250465649 wait complete. 2024-06-06T06:56:44.637 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 10 2024-06-06T06:56:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:44 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:45.593 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:45 smithi073 ceph-mon[32303]: Deploying daemon mds.g on smithi073 2024-06-06T06:56:45.593 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:45 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:boot 2024-06-06T06:56:45.593 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:45 smithi073 ceph-mon[32303]: fsmap 9 up:standby 2024-06-06T06:56:45.593 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:45 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T06:56:45.593 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:45 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4250465649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:45 smithi170 ceph-mon[36108]: Deploying daemon mds.g on smithi073 2024-06-06T06:56:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:45 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:boot 2024-06-06T06:56:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:45 smithi170 ceph-mon[36108]: fsmap 9 up:standby 2024-06-06T06:56:45.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:45 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T06:56:45.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:45 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4250465649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:45.658 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:45 smithi136 ceph-mon[35901]: Deploying daemon mds.g on smithi073 2024-06-06T06:56:45.658 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:45 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:boot 2024-06-06T06:56:45.659 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:45 smithi136 ceph-mon[35901]: fsmap 9 up:standby 2024-06-06T06:56:45.659 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:45 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T06:56:45.659 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:45 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4250465649' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:46.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:46 smithi073 ceph-mon[32303]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:46.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:46 smithi136 ceph-mon[35901]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:46.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:46 smithi170 ceph-mon[36108]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:47.025 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:46 smithi073 systemd[1]: Starting Ceph mds.g for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:47.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:47.389 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 podman[73930]: 2024-06-06 06:56:47.005876888 +0000 UTC m=+0.016945118 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:47.389 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 podman[73930]: 2024-06-06 06:56:47.132691712 +0000 UTC m=+0.143759928 container create f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T06:56:47.626 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:47.749 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 podman[73930]: 2024-06-06 06:56:47.39374326 +0000 UTC m=+0.404811485 container init f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-06T06:56:47.749 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 podman[73930]: 2024-06-06 06:56:47.396730894 +0000 UTC m=+0.407799128 container start f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T06:56:47.749 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 ceph-mds[73966]: -- 172.21.15.73:0/3346021206 <== mon.0 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5590da667a40 con 0x5590da63b000 2024-06-06T06:56:47.749 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g[73944]: starting mds.g at 2024-06-06T06:56:47.749 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 bash[73930]: f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 2024-06-06T06:56:47.750 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 06:56:47 smithi073 systemd[1]: Started Ceph mds.g for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:48.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:48 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:48.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:48 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:48.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:48.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:48.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:48 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:49 smithi073 ceph-mon[32303]: Deploying daemon mds.a on smithi073 2024-06-06T06:56:49.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:49 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.73:6838/2070842628,v1:172.21.15.73:6839/2070842628] up:boot 2024-06-06T06:56:49.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:49 smithi073 ceph-mon[32303]: fsmap 10 up:standby 2024-06-06T06:56:49.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:49 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T06:56:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:49 smithi136 ceph-mon[35901]: Deploying daemon mds.a on smithi073 2024-06-06T06:56:49.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:49 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.73:6838/2070842628,v1:172.21.15.73:6839/2070842628] up:boot 2024-06-06T06:56:49.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:49 smithi136 ceph-mon[35901]: fsmap 10 up:standby 2024-06-06T06:56:49.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:49 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T06:56:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:49 smithi170 ceph-mon[36108]: Deploying daemon mds.a on smithi073 2024-06-06T06:56:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:49 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.73:6838/2070842628,v1:172.21.15.73:6839/2070842628] up:boot 2024-06-06T06:56:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:49 smithi170 ceph-mon[36108]: fsmap 10 up:standby 2024-06-06T06:56:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:49 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T06:56:50.414 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:50 smithi073 ceph-mon[32303]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:50.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 -- 172.21.15.73:0/370423054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc1013c0 msgr2=0x7f90bc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:50.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 --2- 172.21.15.73:0/370423054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc1013c0 0x7f90bc101840 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f90b0009990 tx=0x7f90b0009ca0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 -- 172.21.15.73:0/370423054 shutdown_connections 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 --2- 172.21.15.73:0/370423054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc107440 0x7f90bc107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 --2- 172.21.15.73:0/370423054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc101d80 0x7f90bc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 --2- 172.21.15.73:0/370423054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc1013c0 0x7f90bc101840 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 -- 172.21.15.73:0/370423054 >> 172.21.15.73:0/370423054 conn(0x7f90bc077400 msgr2=0x7f90bc077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 -- 172.21.15.73:0/370423054 shutdown_connections 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.433+0000 7f90c180d700 1 -- 172.21.15.73:0/370423054 wait complete. 2024-06-06T06:56:50.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.434+0000 7f90c180d700 1 Processor -- start 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.434+0000 7f90c180d700 1 -- start start 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 0x7f90bc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 0x7f90bc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111510 con 0x7f90bc107440 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111390 con 0x7f90bc1013c0 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c180d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90bc111690 con 0x7f90bc101d80 2024-06-06T06:56:50.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90baffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 0x7f90bc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40814/0 (socket says 172.21.15.73:40814) 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90baffd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 0x7f90bc19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55614/0 (socket says 172.21.15.73:55614) 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 -- 172.21.15.73:0/2814347921 learned_addr learned my addr 172.21.15.73:0/2814347921 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90ba7fc700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 0x7f90bc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 -- 172.21.15.73:0/2814347921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 msgr2=0x7f90bc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:50.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 0x7f90bc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 -- 172.21.15.73:0/2814347921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 msgr2=0x7f90bc19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 0x7f90bc19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 -- 172.21.15.73:0/2814347921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90bc199060 con 0x7f90bc107440 2024-06-06T06:56:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90ba7fc700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 0x7f90bc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:56:50.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90bb7fe700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90b4007ee0 tx=0x7f90b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.435+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b400dbd0 con 0x7f90bc107440 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.436+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90b400dd70 con 0x7f90bc107440 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.436+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90bc199350 con 0x7f90bc107440 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.436+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90b4005990 con 0x7f90bc107440 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.436+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90bc1a5610 con 0x7f90bc107440 2024-06-06T06:56:50.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.437+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90bc069180 con 0x7f90bc107440 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.438+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f90b401a020 con 0x7f90bc107440 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.438+0000 7f90c080b700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.438+0000 7f90baffd700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.438+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f90b40ce9a0 con 0x7f90bc107440 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.439+0000 7f90baffd700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 0x7f90a40a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90bc199a40 tx=0x7f90ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:50.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.442+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f90b4010070 con 0x7f90bc107440 2024-06-06T06:56:50.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.729+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f90bc1a57e0 con 0x7f90bc107440 2024-06-06T06:56:50.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.730+0000 7f90c080b700 1 -- 172.21.15.73:0/2814347921 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8827 (secure 0 0 0) 0x7f90b409bda0 con 0x7f90bc107440 2024-06-06T06:56:50.731 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:50.731 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24404,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T06:56:50.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 msgr2=0x7f90a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:50.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 0x7f90a40a50e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f90bc199a40 tx=0x7f90ac00b410 comp rx=0 tx=0).stop 2024-06-06T06:56:50.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 msgr2=0x7f90bc198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:50.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90b4007ee0 tx=0x7f90b400d3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 shutdown_connections 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90bc107440 0x7f90bc198900 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90bc1013c0 0x7f90bc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90a40a2c20 0x7f90a40a50e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 --2- 172.21.15.73:0/2814347921 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90bc101d80 0x7f90bc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.732+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 >> 172.21.15.73:0/2814347921 conn(0x7f90bc077400 msgr2=0x7f90bc06ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:50.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.733+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 shutdown_connections 2024-06-06T06:56:50.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:50.733+0000 7f90c180d700 1 -- 172.21.15.73:0/2814347921 wait complete. 2024-06-06T06:56:50.736 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 11 2024-06-06T06:56:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:50 smithi136 ceph-mon[35901]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:50 smithi170 ceph-mon[36108]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:51.742 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:51 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2814347921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:51 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2814347921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:51 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2814347921' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:52.274 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 06:56:51 smithi073 systemd[1]: Started Ceph mds.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:52.649 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:52.649 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.649 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.649 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.649 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:52.650 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:52.650 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:52.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:52.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-06T06:56:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-06T06:56:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:53.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:56:53.685 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:53 smithi136 ceph-mon[35901]: Deploying daemon mds.b on smithi136 2024-06-06T06:56:53.685 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:53 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.73:6840/552212635,v1:172.21.15.73:6841/552212635] up:boot 2024-06-06T06:56:53.685 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:53 smithi136 ceph-mon[35901]: fsmap 11 up:standby 2024-06-06T06:56:53.685 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T06:56:53.701 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:53 smithi073 ceph-mon[32303]: Deploying daemon mds.b on smithi136 2024-06-06T06:56:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:53 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.73:6840/552212635,v1:172.21.15.73:6841/552212635] up:boot 2024-06-06T06:56:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:53 smithi073 ceph-mon[32303]: fsmap 11 up:standby 2024-06-06T06:56:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T06:56:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:53 smithi170 ceph-mon[36108]: Deploying daemon mds.b on smithi136 2024-06-06T06:56:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:53 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.73:6840/552212635,v1:172.21.15.73:6841/552212635] up:boot 2024-06-06T06:56:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:53 smithi170 ceph-mon[36108]: fsmap 11 up:standby 2024-06-06T06:56:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T06:56:54.301 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 systemd[1]: Starting Ceph mds.b for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T06:56:54.677 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:54 smithi136 ceph-mon[35901]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:54.678 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 podman[65948]: 2024-06-06 06:56:54.330777466 +0000 UTC m=+0.017327995 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T06:56:54.678 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 podman[65948]: 2024-06-06 06:56:54.441398525 +0000 UTC m=+0.127949063 container create fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-06T06:56:54.678 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 podman[65948]: 2024-06-06 06:56:54.677651514 +0000 UTC m=+0.364202046 container init fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc) 2024-06-06T06:56:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:54 smithi073 ceph-mon[32303]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:54 smithi170 ceph-mon[36108]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:55.052 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 podman[65948]: 2024-06-06 06:56:54.680556825 +0000 UTC m=+0.367107357 container start fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T06:56:55.052 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 ceph-mds[65967]: -- 172.21.15.136:0/913114898 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c056511a40 con 0x55c0564e4c00 2024-06-06T06:56:55.052 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b[65963]: starting mds.b at 2024-06-06T06:56:55.052 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 bash[65948]: fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 2024-06-06T06:56:55.052 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 06:56:54 smithi136 systemd[1]: Started Ceph mds.b for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T06:56:55.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 -- 172.21.15.73:0/1967322202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 msgr2=0x7ff69c10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:55.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 --2- 172.21.15.73:0/1967322202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c10f320 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff69400b0a0 tx=0x7ff69400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 -- 172.21.15.73:0/1967322202 shutdown_connections 2024-06-06T06:56:55.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 --2- 172.21.15.73:0/1967322202 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 0x7ff69c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 --2- 172.21.15.73:0/1967322202 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c10f320 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 --2- 172.21.15.73:0/1967322202 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 0x7ff69c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 -- 172.21.15.73:0/1967322202 >> 172.21.15.73:0/1967322202 conn(0x7ff69c078340 msgr2=0x7ff69c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 -- 172.21.15.73:0/1967322202 shutdown_connections 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.461+0000 7ff6a2530700 1 -- 172.21.15.73:0/1967322202 wait complete. 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.462+0000 7ff6a2530700 1 Processor -- start 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.462+0000 7ff6a2530700 1 -- start start 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.462+0000 7ff6a2530700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 0x7ff69c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a2530700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a2530700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 0x7ff69c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a2530700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c112630 con 0x7ff69c06ab30 2024-06-06T06:56:55.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a2530700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1124b0 con 0x7ff69c06a170 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a2530700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff69c1127b0 con 0x7ff69c108500 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38074/0 (socket says 172.21.15.73:38074) 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 -- 172.21.15.73:0/79105313 learned_addr learned my addr 172.21.15.73:0/79105313 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69bfff700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 0x7ff69c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6a0acd700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 0x7ff69c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 -- 172.21.15.73:0/79105313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 msgr2=0x7ff69c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 0x7ff69c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 -- 172.21.15.73:0/79105313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 msgr2=0x7ff69c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 0x7ff69c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 -- 172.21.15.73:0/79105313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff69c1990d0 con 0x7ff69c06ab30 2024-06-06T06:56:55.466 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff69b7fe700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.463+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff69000ca10 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.464+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff69000cbb0 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.464+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff69c1993c0 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.464+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff690018680 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.464+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff69c1a5680 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.464+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff69c10a9e0 con 0x7ff69c06ab30 2024-06-06T06:56:55.467 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.467+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff690010430 con 0x7ff69c06ab30 2024-06-06T06:56:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.467+0000 7ff6997fa700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 0x7ff6840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.467+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ff6900cef60 con 0x7ff69c06ab30 2024-06-06T06:56:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.467+0000 7ff69bfff700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 0x7ff6840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:55.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.468+0000 7ff69bfff700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 0x7ff6840a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff69c199ab0 tx=0x7ff68c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:55.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.471+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff69009c360 con 0x7ff69c06ab30 2024-06-06T06:56:55.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.735+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff69c199ea0 con 0x7ff69c06ab30 2024-06-06T06:56:55.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.735+0000 7ff6997fa700 1 -- 172.21.15.73:0/79105313 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10479 (secure 0 0 0) 0x7ff69009bc60 con 0x7ff69c06ab30 2024-06-06T06:56:55.737 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:56:55.737 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24404,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 msgr2=0x7ff6840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 0x7ff6840a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff69c199ab0 tx=0x7ff68c009260 comp rx=0 tx=0).stop 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 msgr2=0x7ff69c19ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff69000e9a0 tx=0x7ff69000ecb0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 shutdown_connections 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff6840a2c20 0x7ff6840a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff69c108500 0x7ff69c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff69c06a170 0x7ff69c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 --2- 172.21.15.73:0/79105313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff69c06ab30 0x7ff69c19ecf0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:55.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 >> 172.21.15.73:0/79105313 conn(0x7ff69c078340 msgr2=0x7ff69c1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:55.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 shutdown_connections 2024-06-06T06:56:55.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:55.738+0000 7ff6a2530700 1 -- 172.21.15.73:0/79105313 wait complete. 2024-06-06T06:56:55.740 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 13 2024-06-06T06:56:56.014 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:boot 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: fsmap 12 up:standby 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T06:56:56.015 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:55 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/79105313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:boot 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: fsmap 12 up:standby 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T06:56:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:55 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/79105313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:56.123 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.123 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.123 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.123 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.123 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:boot 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: fsmap 12 up:standby 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T06:56:56.124 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:55 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/79105313' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:56:57.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-06T06:56:57.448 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:56:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:56:58 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:56:58 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:56:58.613 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:56:58 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 -- 172.21.15.73:0/1607538465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 msgr2=0x7f47d006b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 --2- 172.21.15.73:0/1607538465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d006b5d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009990 tx=0x7f47c0009ca0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 -- 172.21.15.73:0/1607538465 shutdown_connections 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 --2- 172.21.15.73:0/1607538465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d0104ed0 0x7f47d0105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 --2- 172.21.15.73:0/1607538465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d01073c0 0x7f47d006ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 --2- 172.21.15.73:0/1607538465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d006b5d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 -- 172.21.15.73:0/1607538465 >> 172.21.15.73:0/1607538465 conn(0x7f47d0078340 msgr2=0x7f47d0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 -- 172.21.15.73:0/1607538465 shutdown_connections 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.941+0000 7f47d7f04700 1 -- 172.21.15.73:0/1607538465 wait complete. 2024-06-06T06:56:59.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 Processor -- start 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 -- start start 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0104ed0 0x7f47d019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d01073c0 0x7f47d01a3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0118ca0 con 0x7f47d006b170 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0118b20 con 0x7f47d0104ed0 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d7f04700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47d0118e20 con 0x7f47d01073c0 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38090/0 (socket says 172.21.15.73:38090) 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 -- 172.21.15.73:0/3265073180 learned_addr learned my addr 172.21.15.73:0/3265073180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d549f700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0104ed0 0x7f47d019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:59.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d64a1700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d01073c0 0x7f47d01a3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 -- 172.21.15.73:0/3265073180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d01073c0 msgr2=0x7f47d01a3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d01073c0 0x7f47d01a3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 -- 172.21.15.73:0/3265073180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0104ed0 msgr2=0x7f47d019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0104ed0 0x7f47d019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.942+0000 7f47d5ca0700 1 -- 172.21.15.73:0/3265073180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47d01a4260 con 0x7f47d006b170 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47d5ca0700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ba50 tx=0x7f47cc00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc019070 con 0x7f47d006b170 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47cc00cde0 con 0x7f47d006b170 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47d01a4550 con 0x7f47d006b170 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47cc007370 con 0x7f47d006b170 2024-06-06T06:56:59.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.943+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47d01abdd0 con 0x7f47d006b170 2024-06-06T06:56:59.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.944+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47d01019b0 con 0x7f47d006b170 2024-06-06T06:56:59.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.945+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f47cc01c070 con 0x7f47d006b170 2024-06-06T06:56:59.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.945+0000 7f47c6ffd700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 0x7f47bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:56:59.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.945+0000 7f47d549f700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 0x7f47bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:56:59.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.946+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f47cc0ce780 con 0x7f47d006b170 2024-06-06T06:56:59.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.946+0000 7f47d549f700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 0x7f47bc0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f47c000b5c0 tx=0x7f47c0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:56:59.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:56:59.949+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f47cc023070 con 0x7f47d006b170 2024-06-06T06:57:00.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.160+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f47d0069180 con 0x7f47d006b170 2024-06-06T06:57:00.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.160+0000 7f47c6ffd700 1 -- 172.21.15.73:0/3265073180 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f47cc09bb80 con 0x7f47d006b170 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": ".mgr" 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T06:57:00.162 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-06-06T06:57:00.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 msgr2=0x7f47bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:00.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 0x7f47bc0a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f47c000b5c0 tx=0x7f47c0005960 comp rx=0 tx=0).stop 2024-06-06T06:57:00.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 msgr2=0x7f47d019f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:00.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f47cc00ba50 tx=0x7f47cc00bd60 comp rx=0 tx=0).stop 2024-06-06T06:57:00.165 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 shutdown_connections 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f47bc0a2c20 0x7f47bc0a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47d01073c0 0x7f47d01a3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47d006b170 0x7f47d019f250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 --2- 172.21.15.73:0/3265073180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47d0104ed0 0x7f47d019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 >> 172.21.15.73:0/3265073180 conn(0x7f47d0078340 msgr2=0x7f47d0103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 shutdown_connections 2024-06-06T06:57:00.166 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:00.164+0000 7f47d7f04700 1 -- 172.21.15.73:0/3265073180 wait complete. 2024-06-06T06:57:00.478 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:00.478 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.478 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:00 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3265073180' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:00 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3265073180' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T06:57:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:00.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:00.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:00 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3265073180' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-06T06:57:00.815 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-06T06:57:00.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-06T06:57:01.075 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:02.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:57:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:57:02.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:02.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:57:02.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:57:02.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:57:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- 172.21.15.73:0/3390405561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 msgr2=0x7f366c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- 172.21.15.73:0/3390405561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c10e240 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f366400b0a0 tx=0x7f366400b3b0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- 172.21.15.73:0/3390405561 shutdown_connections 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- 172.21.15.73:0/3390405561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- 172.21.15.73:0/3390405561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c10e240 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- 172.21.15.73:0/3390405561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- 172.21.15.73:0/3390405561 >> 172.21.15.73:0/3390405561 conn(0x7f366c077400 msgr2=0x7f366c077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- 172.21.15.73:0/3390405561 shutdown_connections 2024-06-06T06:57:02.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- 172.21.15.73:0/3390405561 wait complete. 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 Processor -- start 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 -- start start 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.779+0000 7f3672a72700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f366c111530 con 0x7f366c101d80 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1113b0 con 0x7f366c1013c0 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f366c1116b0 con 0x7f366c107440 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38104/0 (socket says 172.21.15.73:38104) 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 -- 172.21.15.73:0/52444592 learned_addr learned my addr 172.21.15.73:0/52444592 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f367100f700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:02.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f367080e700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 -- 172.21.15.73:0/52444592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 msgr2=0x7f366c103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 -- 172.21.15.73:0/52444592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 msgr2=0x7f366c103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 -- 172.21.15.73:0/52444592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366c1a6ff0 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f367080e700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f367100f700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f366bfff700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f365800d950 tx=0x7f365800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658016070 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3658010460 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366c1a72e0 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800f620 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.780+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f366c1a7870 con 0x7f366c101d80 2024-06-06T06:57:02.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.781+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f366c102ab0 con 0x7f366c101d80 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.782+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3658019070 con 0x7f366c101d80 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.783+0000 7f3669ffb700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 0x7f365c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.783+0000 7f367080e700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 0x7f365c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.783+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f36580ce7a0 con 0x7f366c101d80 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.783+0000 7f367080e700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 0x7f365c0a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3660005e10 tx=0x7f3660005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:02.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.787+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f365809bba0 con 0x7f366c101d80 2024-06-06T06:57:02.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:02.983+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f366c1a7ad0 con 0x7f366c101d80 2024-06-06T06:57:03.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.355+0000 7f3669ffb700 1 -- 172.21.15.73:0/52444592 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7f365809b4a0 con 0x7f366c101d80 2024-06-06T06:57:03.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 msgr2=0x7f365c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 0x7f365c0a5160 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3660005e10 tx=0x7f3660005d80 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 msgr2=0x7f366c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f365800d950 tx=0x7f365800dd10 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 shutdown_connections 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f365c0a2ca0 0x7f365c0a5160 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f366c107440 0x7f366c103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f366c1013c0 0x7f366c103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.357+0000 7f3672a72700 1 --2- 172.21.15.73:0/52444592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f366c101d80 0x7f366c1035b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:03.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.358+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 >> 172.21.15.73:0/52444592 conn(0x7f366c077400 msgr2=0x7f366c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:03.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.358+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 shutdown_connections 2024-06-06T06:57:03.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:03.359+0000 7f3672a72700 1 -- 172.21.15.73:0/52444592 wait complete. 2024-06-06T06:57:03.360 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2024-06-06T06:57:03.618 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:03 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T06:57:03.668 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:03 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T06:57:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:03 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-06T06:57:04.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-06-06T06:57:04.257 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:04.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:04 smithi073 ceph-mon[32303]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:04.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:04 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T06:57:04.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:04 smithi073 ceph-mon[32303]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T06:57:04.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:04 smithi136 ceph-mon[35901]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:04.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:04 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T06:57:04.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:04 smithi136 ceph-mon[35901]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T06:57:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:04 smithi170 ceph-mon[36108]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:04 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/52444592' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-06T06:57:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:04 smithi170 ceph-mon[36108]: osdmap e83: 12 total, 12 up, 12 in 2024-06-06T06:57:05.443 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:05 smithi073 ceph-mon[32303]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T06:57:05.714 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:05 smithi136 ceph-mon[35901]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T06:57:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:05 smithi170 ceph-mon[36108]: osdmap e84: 12 total, 12 up, 12 in 2024-06-06T06:57:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 -- 172.21.15.73:0/4253431016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 msgr2=0x7ff1b8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 --2- 172.21.15.73:0/4253431016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b8101810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8009990 tx=0x7ff1a8009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 -- 172.21.15.73:0/4253431016 shutdown_connections 2024-06-06T06:57:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 --2- 172.21.15.73:0/4253431016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8107400 0x7ff1b81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 --2- 172.21.15.73:0/4253431016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8101d50 0x7ff1b810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 --2- 172.21.15.73:0/4253431016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b8101810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 -- 172.21.15.73:0/4253431016 >> 172.21.15.73:0/4253431016 conn(0x7ff1b8078340 msgr2=0x7ff1b8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 -- 172.21.15.73:0/4253431016 shutdown_connections 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.021+0000 7ff1c028d700 1 -- 172.21.15.73:0/4253431016 wait complete. 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 Processor -- start 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 -- start start 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8101d50 0x7ff1b819ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8107400 0x7ff1b81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b81115c0 con 0x7ff1b8101390 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8111440 con 0x7ff1b8107400 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1c028d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1b8111740 con 0x7ff1b8101d50 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1be029700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1be029700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47884/0 (socket says 172.21.15.73:47884) 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.022+0000 7ff1be029700 1 -- 172.21.15.73:0/454759013 learned_addr learned my addr 172.21.15.73:0/454759013 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1bd828700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8101d50 0x7ff1b819ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be82a700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8107400 0x7ff1b81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 -- 172.21.15.73:0/454759013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8101d50 msgr2=0x7ff1b819ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8101d50 0x7ff1b819ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 -- 172.21.15.73:0/454759013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8107400 msgr2=0x7ff1b81989d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8107400 0x7ff1b81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 -- 172.21.15.73:0/454759013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1b8199130 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1be029700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400d980 tx=0x7ff1b400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4017070 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1b8199420 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b4004830 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4005440 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.023+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1b81a56e0 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.024+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1b8069180 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.025+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1b40055e0 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.026+0000 7ff1af7fe700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 0x7ff1a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.026+0000 7ff1bd828700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 0x7ff1a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.026+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7ff1b40cec70 con 0x7ff1b8101390 2024-06-06T06:57:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.027+0000 7ff1bd828700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 0x7ff1a40a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006010 tx=0x7ff1a801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.031+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1b4020020 con 0x7ff1b8101390 2024-06-06T06:57:06.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.231+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7ff1b81a5910 con 0x7ff1b8101390 2024-06-06T06:57:06.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.527+0000 7ff1af7fe700 1 -- 172.21.15.73:0/454759013 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v86) ==== 171+0+0 (secure 0 0 0) 0x7ff1b409bf00 con 0x7ff1b8101390 2024-06-06T06:57:06.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 msgr2=0x7ff1a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:06.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 0x7ff1a40a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff1a8006010 tx=0x7ff1a801a040 comp rx=0 tx=0).stop 2024-06-06T06:57:06.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 msgr2=0x7ff1b819e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff1b400d980 tx=0x7ff1b400dd40 comp rx=0 tx=0).stop 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 shutdown_connections 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1b8107400 0x7ff1b81989d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1b8101390 0x7ff1b819e810 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff1a40a2c20 0x7ff1a40a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 --2- 172.21.15.73:0/454759013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1b8101d50 0x7ff1b819ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 >> 172.21.15.73:0/454759013 conn(0x7ff1b8078340 msgr2=0x7ff1b8100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 shutdown_connections 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:06.530+0000 7ff1c028d700 1 -- 172.21.15.73:0/454759013 wait complete. 2024-06-06T06:57:06.532 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2024-06-06T06:57:06.549 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:06 smithi073 ceph-mon[32303]: pgmap v291: 33 pgs: 3 creating+activating, 9 active+clean, 21 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:06.549 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:06 smithi073 ceph-mon[32303]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:06.549 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:06 smithi073 ceph-mon[32303]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T06:57:06.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:06 smithi136 ceph-mon[35901]: pgmap v291: 33 pgs: 3 creating+activating, 9 active+clean, 21 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:06.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:06 smithi136 ceph-mon[35901]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:06.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:06 smithi136 ceph-mon[35901]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T06:57:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:06 smithi170 ceph-mon[36108]: pgmap v291: 33 pgs: 3 creating+activating, 9 active+clean, 21 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:06 smithi170 ceph-mon[36108]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:06.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:06 smithi170 ceph-mon[36108]: osdmap e85: 12 total, 12 up, 12 in 2024-06-06T06:57:07.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-06T06:57:07.472 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:07.738 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:07 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T06:57:07.738 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:07 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T06:57:07.738 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:07 smithi073 ceph-mon[32303]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T06:57:07.738 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:07 smithi073 ceph-mon[32303]: pgmap v294: 97 pgs: 12 creating+peering, 3 creating+activating, 29 active+clean, 53 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:07 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T06:57:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:07 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T06:57:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:07 smithi136 ceph-mon[35901]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T06:57:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:07 smithi136 ceph-mon[35901]: pgmap v294: 97 pgs: 12 creating+peering, 3 creating+activating, 29 active+clean, 53 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:07 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-06-06T06:57:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:07 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/454759013' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-06-06T06:57:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:07 smithi170 ceph-mon[36108]: osdmap e86: 12 total, 12 up, 12 in 2024-06-06T06:57:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:07 smithi170 ceph-mon[36108]: pgmap v294: 97 pgs: 12 creating+peering, 3 creating+activating, 29 active+clean, 53 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:08.608 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:08 smithi073 ceph-mon[32303]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T06:57:08.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:08 smithi136 ceph-mon[35901]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T06:57:08.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:08 smithi170 ceph-mon[36108]: osdmap e87: 12 total, 12 up, 12 in 2024-06-06T06:57:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 -- 172.21.15.73:0/913199502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 msgr2=0x7f14bc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 --2- 172.21.15.73:0/913199502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc101a00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0099c0 tx=0x7f14ac009cd0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 -- 172.21.15.73:0/913199502 shutdown_connections 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 --2- 172.21.15.73:0/913199502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc1066b0 0x7f14bc10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 --2- 172.21.15.73:0/913199502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc101a00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 --2- 172.21.15.73:0/913199502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc101fd0 0x7f14bc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 -- 172.21.15.73:0/913199502 >> 172.21.15.73:0/913199502 conn(0x7f14bc078360 msgr2=0x7f14bc078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 -- 172.21.15.73:0/913199502 shutdown_connections 2024-06-06T06:57:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.183+0000 7f14c115d700 1 -- 172.21.15.73:0/913199502 wait complete. 2024-06-06T06:57:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.184+0000 7f14c115d700 1 Processor -- start 2024-06-06T06:57:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 -- start start 2024-06-06T06:57:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 0x7f14bc19ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 0x7f14bc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc111540 con 0x7f14bc101620 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1113c0 con 0x7f14bc101fd0 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14c115d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f14bc1116c0 con 0x7f14bc1066b0 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47908/0 (socket says 172.21.15.73:47908) 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 -- 172.21.15.73:0/919052954 learned_addr learned my addr 172.21.15.73:0/919052954 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14ba59c700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 0x7f14bc19ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bb59e700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 0x7f14bc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 -- 172.21.15.73:0/919052954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 msgr2=0x7f14bc198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 0x7f14bc198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 -- 172.21.15.73:0/919052954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 msgr2=0x7f14bc19ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 0x7f14bc19ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bad9d700 1 -- 172.21.15.73:0/919052954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14bc1990b0 con 0x7f14bc101620 2024-06-06T06:57:09.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.185+0000 7f14bb59e700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 0x7f14bc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14ba59c700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 0x7f14bc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14bad9d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009fd0 tx=0x7f14ac0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac027070 con 0x7f14bc101620 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14ac004140 con 0x7f14bc101620 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14bc199340 con 0x7f14bc101620 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14ac022e60 con 0x7f14bc101620 2024-06-06T06:57:09.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.186+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14bc103830 con 0x7f14bc101620 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.187+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14bc102ab0 con 0x7f14bc101620 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.188+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f14ac02a070 con 0x7f14bc101620 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.189+0000 7f14a3fff700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 0x7f14a40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.189+0000 7f14ba59c700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 0x7f14a40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.189+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f14ac0d0130 con 0x7f14bc101620 2024-06-06T06:57:09.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.189+0000 7f14ba59c700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 0x7f14a40a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14bc105c60 tx=0x7f14b0005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:09.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.193+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f14ac013070 con 0x7f14bc101620 2024-06-06T06:57:09.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.431+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f14bc02dca0 con 0x7f14bc101620 2024-06-06T06:57:09.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.541+0000 7f14a3fff700 1 -- 172.21.15.73:0/919052954 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f14ac09d2d0 con 0x7f14bc101620 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 msgr2=0x7f14a40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 0x7f14a40a5160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f14bc105c60 tx=0x7f14b0005f50 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 msgr2=0x7f14bc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f14ac009fd0 tx=0x7f14ac0037a0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 shutdown_connections 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f14a40a2ca0 0x7f14a40a5160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14bc1066b0 0x7f14bc198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14bc101620 0x7f14bc19e790 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 --2- 172.21.15.73:0/919052954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f14bc101fd0 0x7f14bc19ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 >> 172.21.15.73:0/919052954 conn(0x7f14bc078360 msgr2=0x7f14bc0ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:09.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 shutdown_connections 2024-06-06T06:57:09.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:09.543+0000 7f14c115d700 1 -- 172.21.15.73:0/919052954 wait complete. 2024-06-06T06:57:09.545 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 2 and data pool 3 2024-06-06T06:57:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:09 smithi170 ceph-mon[36108]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T06:57:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:09 smithi170 ceph-mon[36108]: pgmap v297: 97 pgs: 18 creating+peering, 3 creating+activating, 38 active+clean, 38 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:09 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T06:57:09.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:09 smithi170 ceph-mon[36108]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-06T06:57:09.874 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:09 smithi073 ceph-mon[32303]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T06:57:09.874 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:09 smithi073 ceph-mon[32303]: pgmap v297: 97 pgs: 18 creating+peering, 3 creating+activating, 38 active+clean, 38 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:09.874 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:09 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T06:57:09.874 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:09 smithi073 ceph-mon[32303]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-06T06:57:10.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:09 smithi136 ceph-mon[35901]: osdmap e88: 12 total, 12 up, 12 in 2024-06-06T06:57:10.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:09 smithi136 ceph-mon[35901]: pgmap v297: 97 pgs: 18 creating+peering, 3 creating+activating, 38 active+clean, 38 unknown; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:10.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:09 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-06T06:57:10.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:09 smithi136 ceph-mon[35901]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-06-06T06:57:10.221 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-06-06T06:57:10.221 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-06-06T06:57:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:10.827 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:10 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T06:57:10.828 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:10 smithi073 ceph-mon[32303]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T06:57:10.828 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:10 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-06T06:57:10.828 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:10 smithi073 ceph-mon[32303]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-06T06:57:10.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:10 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T06:57:10.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:10 smithi170 ceph-mon[36108]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T06:57:10.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:10 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-06T06:57:10.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:10 smithi170 ceph-mon[36108]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-06T06:57:11.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:10 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/919052954' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-06T06:57:11.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:10 smithi136 ceph-mon[35901]: osdmap e89: 12 total, 12 up, 12 in 2024-06-06T06:57:11.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:10 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-06-06T06:57:11.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:10 smithi136 ceph-mon[35901]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-06-06T06:57:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:11 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:active 2024-06-06T06:57:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:11 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:11 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:11 smithi170 ceph-mon[36108]: pgmap v299: 97 pgs: 18 creating+peering, 73 active+clean, 6 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-06-06T06:57:11.911 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:11 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:active 2024-06-06T06:57:11.911 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:11 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:11.911 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:11 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:11.911 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:11 smithi073 ceph-mon[32303]: pgmap v299: 97 pgs: 18 creating+peering, 73 active+clean, 6 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-06-06T06:57:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:11 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] up:active 2024-06-06T06:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:11 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:11 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:12.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:11 smithi136 ceph-mon[35901]: pgmap v299: 97 pgs: 18 creating+peering, 73 active+clean, 6 unknown; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-06-06T06:57:12.263 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- 172.21.15.73:0/3667637706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 msgr2=0x7fef8010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 --2- 172.21.15.73:0/3667637706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef8010e230 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef7800b0a0 tx=0x7fef7800b3b0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- 172.21.15.73:0/3667637706 shutdown_connections 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 --2- 172.21.15.73:0/3667637706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef801073f0 0x7fef801077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 --2- 172.21.15.73:0/3667637706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef8010e230 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 --2- 172.21.15.73:0/3667637706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef80101380 0x7fef80101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- 172.21.15.73:0/3667637706 >> 172.21.15.73:0/3667637706 conn(0x7fef80078300 msgr2=0x7fef80078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- 172.21.15.73:0/3667637706 shutdown_connections 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- 172.21.15.73:0/3667637706 wait complete. 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 Processor -- start 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.261+0000 7fef86c71700 1 -- start start 2024-06-06T06:57:12.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef80101380 0x7fef80103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 0x7fef80103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111610 con 0x7fef80101d40 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111490 con 0x7fef801073f0 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef80111790 con 0x7fef80101380 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47928/0 (socket says 172.21.15.73:47928) 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 -- 172.21.15.73:0/4253583849 learned_addr learned my addr 172.21.15.73:0/4253583849 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef8520e700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 0x7fef80103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef84a0d700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef80101380 0x7fef80103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 -- 172.21.15.73:0/4253583849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef80101380 msgr2=0x7fef80103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:12.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef80101380 0x7fef80103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 -- 172.21.15.73:0/4253583849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 msgr2=0x7fef80103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 0x7fef80103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 -- 172.21.15.73:0/4253583849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef801a7150 con 0x7fef80101d40 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef8520e700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 0x7fef80103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7ffff700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b530 tx=0x7fef6c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c015020 con 0x7fef80101d40 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef6c011ce0 con 0x7fef80101d40 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef801a7440 con 0x7fef80101d40 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.262+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c010d60 con 0x7fef80101d40 2024-06-06T06:57:12.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.263+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef801a7980 con 0x7fef80101d40 2024-06-06T06:57:12.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.264+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef80069180 con 0x7fef80101d40 2024-06-06T06:57:12.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.265+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fef6c018020 con 0x7fef80101d40 2024-06-06T06:57:12.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.266+0000 7fef7dffb700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 0x7fef700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:12.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.266+0000 7fef84a0d700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 0x7fef700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:12.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.266+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fef6c0cece0 con 0x7fef80101d40 2024-06-06T06:57:12.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.266+0000 7fef84a0d700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 0x7fef700a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef74006fd0 tx=0x7fef74008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:12.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.270+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef6c01f020 con 0x7fef80101d40 2024-06-06T06:57:12.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.478+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fef80104590 con 0x7fef80101d40 2024-06-06T06:57:12.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.564+0000 7fef7dffb700 1 -- 172.21.15.73:0/4253583849 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v90) ==== 157+0+0 (secure 0 0 0) 0x7fef80104590 con 0x7fef80101d40 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 msgr2=0x7fef700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 0x7fef700a50e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fef74006fd0 tx=0x7fef74008040 comp rx=0 tx=0).stop 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 msgr2=0x7fef801035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00b530 tx=0x7fef6c00b8f0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 shutdown_connections 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fef700a2c20 0x7fef700a50e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef80101380 0x7fef80103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef801073f0 0x7fef80103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 --2- 172.21.15.73:0/4253583849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef80101d40 0x7fef801035a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:12.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 >> 172.21.15.73:0/4253583849 conn(0x7fef80078300 msgr2=0x7fef80100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:12.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 shutdown_connections 2024-06-06T06:57:12.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:12.567+0000 7fef86c71700 1 -- 172.21.15.73:0/4253583849 wait complete. 2024-06-06T06:57:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:12 smithi170 ceph-mon[36108]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T06:57:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:12 smithi170 ceph-mon[36108]: Cluster is now healthy 2024-06-06T06:57:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:12 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T06:57:12.931 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:12 smithi073 ceph-mon[32303]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T06:57:12.931 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:12 smithi073 ceph-mon[32303]: Cluster is now healthy 2024-06-06T06:57:12.931 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:12 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T06:57:13.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:12 smithi136 ceph-mon[35901]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-06T06:57:13.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:12 smithi136 ceph-mon[35901]: Cluster is now healthy 2024-06-06T06:57:13.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:12 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-06-06T06:57:13.417 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-06-06T06:57:13.659 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:13 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T06:57:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:13 smithi073 ceph-mon[32303]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T06:57:13.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:13 smithi073 ceph-mon[32303]: pgmap v301: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-06T06:57:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:13 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T06:57:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:13 smithi170 ceph-mon[36108]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T06:57:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:13 smithi170 ceph-mon[36108]: pgmap v301: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-06T06:57:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:13 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4253583849' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-06-06T06:57:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:13 smithi136 ceph-mon[35901]: osdmap e90: 12 total, 12 up, 12 in 2024-06-06T06:57:14.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:13 smithi136 ceph-mon[35901]: pgmap v301: 97 pgs: 97 active+clean; 578 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 6 op/s 2024-06-06T06:57:14.621 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:57:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T06:57:14.354+0000 7f514b1c7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 -- 172.21.15.73:0/3548961787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc107460 msgr2=0x7f55dc107840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/3548961787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc107460 0x7f55dc107840 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f55d0009990 tx=0x7f55d0009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 -- 172.21.15.73:0/3548961787 shutdown_connections 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/3548961787 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc101d70 0x7f55dc10e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/3548961787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc107460 0x7f55dc107840 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/3548961787 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 0x7f55dc101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 -- 172.21.15.73:0/3548961787 >> 172.21.15.73:0/3548961787 conn(0x7f55dc0783b0 msgr2=0x7f55dc0787c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 -- 172.21.15.73:0/3548961787 shutdown_connections 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 -- 172.21.15.73:0/3548961787 wait complete. 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.564+0000 7f55e2ef0700 1 Processor -- start 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 -- start start 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 0x7f55dc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc107460 0x7f55dc198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc111550 con 0x7f55dc101d70 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1113d0 con 0x7f55dc107460 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e2ef0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f55dc1116d0 con 0x7f55dc1013b0 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:15.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56704/0 (socket says 172.21.15.73:56704) 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 -- 172.21.15.73:0/2603474628 learned_addr learned my addr 172.21.15.73:0/2603474628 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e0c8c700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 0x7f55dc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55e148d700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc107460 0x7f55dc198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 -- 172.21.15.73:0/2603474628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 msgr2=0x7f55dc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 0x7f55dc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 -- 172.21.15.73:0/2603474628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc107460 msgr2=0x7f55dc198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc107460 0x7f55dc198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 -- 172.21.15.73:0/2603474628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55dc1990a0 con 0x7f55dc101d70 2024-06-06T06:57:15.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.565+0000 7f55dbfff700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55c800d980 tx=0x7f55c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:15.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.566+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8017070 con 0x7f55dc101d70 2024-06-06T06:57:15.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.566+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55c8005020 con 0x7f55dc101d70 2024-06-06T06:57:15.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.566+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55dc199390 con 0x7f55dc101d70 2024-06-06T06:57:15.570 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.566+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c800fa90 con 0x7f55dc101d70 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.566+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55dc1a5650 con 0x7f55dc101d70 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.567+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55dc050a00 con 0x7f55dc101d70 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.568+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f55c80051d0 con 0x7f55dc101d70 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.568+0000 7f55d9ffb700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 0x7f55cc0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.569+0000 7f55e0c8c700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 0x7f55cc0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.569+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8277+0+0 (secure 0 0 0) 0x7f55c80cee40 con 0x7f55dc101d70 2024-06-06T06:57:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.569+0000 7f55e0c8c700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 0x7f55cc0a5200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b5c0 tx=0x7f55d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.573+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55c8020020 con 0x7f55dc101d70 2024-06-06T06:57:15.773 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:15.772+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f55dc1a5820 con 0x7f55dc101d70 2024-06-06T06:57:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:16 smithi170 ceph-mon[36108]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T06:57:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:16 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T06:57:17.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:16 smithi073 ceph-mon[32303]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T06:57:17.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:16 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T06:57:17.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:16 smithi136 ceph-mon[35901]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-06-06T06:57:17.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:16 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-06T06:57:17.352 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.350+0000 7f55d9ffb700 1 -- 172.21.15.73:0/2603474628 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v92) ==== 228+0+0 (secure 0 0 0) 0x7f55c809bd70 con 0x7f55dc101d70 2024-06-06T06:57:17.355 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 msgr2=0x7f55cc0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 0x7f55cc0a5200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f55d000b5c0 tx=0x7f55d0005960 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 msgr2=0x7f55dc19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f55c800d980 tx=0x7f55c800dc90 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 shutdown_connections 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f55cc0a2d40 0x7f55cc0a5200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f55dc1013b0 0x7f55dc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f55dc107460 0x7f55dc198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 --2- 172.21.15.73:0/2603474628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc101d70 0x7f55dc19ecc0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 >> 172.21.15.73:0/2603474628 conn(0x7f55dc0783b0 msgr2=0x7f55dc1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 shutdown_connections 2024-06-06T06:57:17.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:17.353+0000 7f55e2ef0700 1 -- 172.21.15.73:0/2603474628 wait complete. 2024-06-06T06:57:17.357 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data_ec' created 2024-06-06T06:57:17.680 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:17 smithi073 ceph-mon[32303]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T06:57:17.680 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:17 smithi073 ceph-mon[32303]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T06:57:17.680 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:17 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T06:57:17.680 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:17 smithi073 ceph-mon[32303]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T06:57:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:17 smithi170 ceph-mon[36108]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T06:57:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:17 smithi170 ceph-mon[36108]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T06:57:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:17 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T06:57:17.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:17 smithi170 ceph-mon[36108]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T06:57:18.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:17 smithi136 ceph-mon[35901]: osdmap e91: 12 total, 12 up, 12 in 2024-06-06T06:57:18.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:17 smithi136 ceph-mon[35901]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-06-06T06:57:18.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:17 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2603474628' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-06T06:57:18.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:17 smithi136 ceph-mon[35901]: osdmap e92: 12 total, 12 up, 12 in 2024-06-06T06:57:18.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-06-06T06:57:18.322 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:19.665 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:19 smithi073 ceph-mon[32303]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T06:57:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:19 smithi136 ceph-mon[35901]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T06:57:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:19 smithi170 ceph-mon[36108]: osdmap e93: 12 total, 12 up, 12 in 2024-06-06T06:57:20.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 -- 172.21.15.73:0/3261107522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438107840 msgr2=0x7f243810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:20.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 --2- 172.21.15.73:0/3261107522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438107840 0x7f243810f3c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f243000b0d0 tx=0x7f243000b3e0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 -- 172.21.15.73:0/3261107522 shutdown_connections 2024-06-06T06:57:20.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 --2- 172.21.15.73:0/3261107522 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438107840 0x7f243810f3c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 --2- 172.21.15.73:0/3261107522 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438102740 0x7f2438102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 --2- 172.21.15.73:0/3261107522 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f2438107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 -- 172.21.15.73:0/3261107522 >> 172.21.15.73:0/3261107522 conn(0x7f24380795f0 msgr2=0x7f2438079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 -- 172.21.15.73:0/3261107522 shutdown_connections 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.239+0000 7f243e65c700 1 -- 172.21.15.73:0/3261107522 wait complete. 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 Processor -- start 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 -- start start 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f243819ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438107840 0x7f24381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2438112680 con 0x7f2438102740 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2438112500 con 0x7f2438107840 2024-06-06T06:57:20.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f243e65c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2438112800 con 0x7f24381030f0 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f2437fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f2437fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56728/0 (socket says 172.21.15.73:56728) 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.240+0000 7f2437fff700 1 -- 172.21.15.73:0/3305786028 learned_addr learned my addr 172.21.15.73:0/3305786028 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f242ffff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f243819ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 -- 172.21.15.73:0/3305786028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 msgr2=0x7f243819ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f243819ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 -- 172.21.15.73:0/3305786028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438107840 msgr2=0x7f24381988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f243cbf9700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438107840 0x7f24381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438107840 0x7f24381988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 -- 172.21.15.73:0/3305786028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2438199020 con 0x7f2438102740 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f242ffff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f243819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:20.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2437fff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f242800c920 tx=0x7f242800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428007b00 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2428004ce0 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2438199310 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2428005600 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.241+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24381a55d0 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.242+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2438069180 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.244+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2428007ca0 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.245+0000 7f2435ffb700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 0x7f24240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.245+0000 7f242ffff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 0x7f24240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.245+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8770+0+0 (secure 0 0 0) 0x7f24280cf2c0 con 0x7f2438102740 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.245+0000 7f242ffff700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 0x7f24240a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2438101ff0 tx=0x7f2420008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:20.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.248+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2428014030 con 0x7f2438102740 2024-06-06T06:57:20.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:20.447+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f243802dca0 con 0x7f2438102740 2024-06-06T06:57:20.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:20 smithi073 ceph-mon[32303]: pgmap v307: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T06:57:20.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:20 smithi073 ceph-mon[32303]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:20.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:20 smithi073 ceph-mon[32303]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T06:57:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:20 smithi136 ceph-mon[35901]: pgmap v307: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T06:57:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:20 smithi136 ceph-mon[35901]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:20 smithi136 ceph-mon[35901]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T06:57:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:20 smithi170 ceph-mon[36108]: pgmap v307: 129 pgs: 2 creating+peering, 30 unknown, 97 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 0 op/s 2024-06-06T06:57:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:20 smithi170 ceph-mon[36108]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-06T06:57:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:20 smithi170 ceph-mon[36108]: osdmap e94: 12 total, 12 up, 12 in 2024-06-06T06:57:21.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.365+0000 7f2435ffb700 1 -- 172.21.15.73:0/3305786028 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v95) ==== 169+0+0 (secure 0 0 0) 0x7f242809c000 con 0x7f2438102740 2024-06-06T06:57:21.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 msgr2=0x7f24240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:21.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 0x7f24240a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2438101ff0 tx=0x7f2420008040 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 msgr2=0x7f243819e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f242800c920 tx=0x7f242800cce0 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 shutdown_connections 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2438107840 0x7f24381988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2438102740 0x7f243819e700 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f24240a2c20 0x7f24240a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 --2- 172.21.15.73:0/3305786028 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f24381030f0 0x7f243819ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.368+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 >> 172.21.15.73:0/3305786028 conn(0x7f24380795f0 msgr2=0x7f2438100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.369+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 shutdown_connections 2024-06-06T06:57:21.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:21.369+0000 7f243e65c700 1 -- 172.21.15.73:0/3305786028 wait complete. 2024-06-06T06:57:21.371 INFO:teuthology.orchestra.run.smithi073.stderr:set pool 4 allow_ec_overwrites to true 2024-06-06T06:57:21.697 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T06:57:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T06:57:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-06-06T06:57:22.033 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-06-06T06:57:22.277 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:22.653 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:22 smithi073 ceph-mon[32303]: pgmap v309: 129 pgs: 2 creating+peering, 6 unknown, 121 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:22.654 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:22 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T06:57:22.654 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:22 smithi073 ceph-mon[32303]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T06:57:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:22 smithi136 ceph-mon[35901]: pgmap v309: 129 pgs: 2 creating+peering, 6 unknown, 121 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:22 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T06:57:22.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:22 smithi136 ceph-mon[35901]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T06:57:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:22 smithi170 ceph-mon[36108]: pgmap v309: 129 pgs: 2 creating+peering, 6 unknown, 121 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:22 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3305786028' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-06-06T06:57:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:22 smithi170 ceph-mon[36108]: osdmap e95: 12 total, 12 up, 12 in 2024-06-06T06:57:23.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.939+0000 7f2aa38b8700 1 -- 172.21.15.73:0/4082342875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101620 msgr2=0x7f2a9c101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:23.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.939+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/4082342875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101620 0x7f2a9c101a00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a980099c0 tx=0x7f2a98009cd0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 -- 172.21.15.73:0/4082342875 shutdown_connections 2024-06-06T06:57:23.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/4082342875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 0x7f2a9c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/4082342875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101620 0x7f2a9c101a00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/4082342875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 -- 172.21.15.73:0/4082342875 >> 172.21.15.73:0/4082342875 conn(0x7f2a9c078340 msgr2=0x7f2a9c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 -- 172.21.15.73:0/4082342875 shutdown_connections 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 -- 172.21.15.73:0/4082342875 wait complete. 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 Processor -- start 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 -- start start 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 0x7f2a9c199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.940+0000 7f2aa38b8700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 0x7f2a9c1a4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa38b8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c1115c0 con 0x7f2a9c101fd0 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa38b8700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c111440 con 0x7f2a9c101620 2024-06-06T06:57:23.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa38b8700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a9c111740 con 0x7f2a9c1066b0 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56746/0 (socket says 172.21.15.73:56746) 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 -- 172.21.15.73:0/2235301618 learned_addr learned my addr 172.21.15.73:0/2235301618 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa1654700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 0x7f2a9c199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa1e55700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 0x7f2a9c1a4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 -- 172.21.15.73:0/2235301618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 msgr2=0x7f2a9c1a4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 0x7f2a9c1a4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 -- 172.21.15.73:0/2235301618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 msgr2=0x7f2a9c199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 0x7f2a9c199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 -- 172.21.15.73:0/2235301618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a9c1a5020 con 0x7f2a9c101fd0 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa1654700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 0x7f2a9c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:23.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa0e53700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c8f0 tx=0x7f2a8c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c007b00 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a8c007ca0 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a9c1a5250 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c018660 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.941+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a9c1a57b0 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.942+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a9c069180 con 0x7f2a9c101fd0 2024-06-06T06:57:23.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.944+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2a8c00f450 con 0x7f2a9c101fd0 2024-06-06T06:57:23.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.945+0000 7f2a927fc700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 0x7f2a880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:23.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.945+0000 7f2aa1654700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 0x7f2a880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:23.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.945+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8770+0+0 (secure 0 0 0) 0x7f2a8c0cf1f0 con 0x7f2a9c101fd0 2024-06-06T06:57:23.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.945+0000 7f2aa1654700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 0x7f2a880a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009fd0 tx=0x7f2a98005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:23.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:23.948+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a8c014030 con 0x7f2a9c101fd0 2024-06-06T06:57:24.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.197+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f2a9c02dca0 con 0x7f2a9c101fd0 2024-06-06T06:57:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.378+0000 7f2a927fc700 1 -- 172.21.15.73:0/2235301618 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f2a8c09c0e0 con 0x7f2a9c101fd0 2024-06-06T06:57:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 msgr2=0x7f2a880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 0x7f2a880a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009fd0 tx=0x7f2a98005fd0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 msgr2=0x7f2a9c199fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c00c8f0 tx=0x7f2a8c00ccb0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 shutdown_connections 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2a880a2ca0 0x7f2a880a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a9c1066b0 0x7f2a9c1a4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a9c101620 0x7f2a9c199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 --2- 172.21.15.73:0/2235301618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a9c101fd0 0x7f2a9c199fa0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.381+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 >> 172.21.15.73:0/2235301618 conn(0x7f2a9c078340 msgr2=0x7f2a9c0ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.382+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 shutdown_connections 2024-06-06T06:57:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:24.382+0000 7f2aa38b8700 1 -- 172.21.15.73:0/2235301618 wait complete. 2024-06-06T06:57:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:added data pool 4 to fsmap 2024-06-06T06:57:24.757 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:24 smithi073 ceph-mon[32303]: pgmap v311: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:24.757 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:24 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T06:57:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:24 smithi136 ceph-mon[35901]: pgmap v311: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:24 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T06:57:24.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:24 smithi170 ceph-mon[36108]: pgmap v311: 129 pgs: 2 creating+peering, 127 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:24.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:24 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-06-06T06:57:25.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:57:25.618 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:25.741 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:25 smithi073 ceph-mon[32303]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T06:57:25.741 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:25 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T06:57:25.741 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:25 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:25 smithi136 ceph-mon[35901]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T06:57:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:25 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T06:57:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:25 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:25 smithi170 ceph-mon[36108]: osdmap e96: 12 total, 12 up, 12 in 2024-06-06T06:57:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:25 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2235301618' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-06-06T06:57:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:25 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:57:26.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:26 smithi073 ceph-mon[32303]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:26.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:26 smithi073 ceph-mon[32303]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T06:57:26.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:26 smithi073 ceph-mon[32303]: Cluster is now healthy 2024-06-06T06:57:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:26 smithi136 ceph-mon[35901]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:26 smithi136 ceph-mon[35901]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T06:57:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:26 smithi136 ceph-mon[35901]: Cluster is now healthy 2024-06-06T06:57:26.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:26 smithi170 ceph-mon[36108]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:26.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:26 smithi170 ceph-mon[36108]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-06T06:57:26.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:26 smithi170 ceph-mon[36108]: Cluster is now healthy 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 -- 172.21.15.73:0/2478207744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 msgr2=0x7fba1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 --2- 172.21.15.73:0/2478207744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c074aa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fba0c009990 tx=0x7fba0c009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 -- 172.21.15.73:0/2478207744 shutdown_connections 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 --2- 172.21.15.73:0/2478207744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba1c10f160 0x7fba1c111550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 --2- 172.21.15.73:0/2478207744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c074aa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 --2- 172.21.15.73:0/2478207744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba1c074fe0 0x7fba1c10ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 -- 172.21.15.73:0/2478207744 >> 172.21.15.73:0/2478207744 conn(0x7fba1c07a420 msgr2=0x7fba1c07a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 -- 172.21.15.73:0/2478207744 shutdown_connections 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.339+0000 7fba21526700 1 -- 172.21.15.73:0/2478207744 wait complete. 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 Processor -- start 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 -- start start 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba1c074fe0 0x7fba1c19b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba1c10f160 0x7fba1c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114990 con 0x7fba1c0746c0 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114810 con 0x7fba1c074fe0 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba21526700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fba1c114b10 con 0x7fba1c10f160 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:27.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60764/0 (socket says 172.21.15.73:60764) 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 -- 172.21.15.73:0/2824570640 learned_addr learned my addr 172.21.15.73:0/2824570640 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 -- 172.21.15.73:0/2824570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba1c10f160 msgr2=0x7fba1c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba1c10f160 0x7fba1c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 -- 172.21.15.73:0/2824570640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba1c074fe0 msgr2=0x7fba1c19b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba1c074fe0 0x7fba1c19b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.340+0000 7fba1affd700 1 -- 172.21.15.73:0/2824570640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba1c19fdf0 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba1affd700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00bff0 tx=0x7fba0c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c027070 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba0c022470 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba1c1a0080 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c017650 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.341+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba1c1a7960 con 0x7fba1c0746c0 2024-06-06T06:57:27.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.342+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba1c075bd0 con 0x7fba1c0746c0 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.343+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fba0c02a070 con 0x7fba1c0746c0 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.343+0000 7fba18ff9700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 0x7fba080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.344+0000 7fba127fc700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 0x7fba080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.344+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fba0c0d0500 con 0x7fba1c0746c0 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.344+0000 7fba127fc700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 0x7fba080a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:27.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.347+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba0c013070 con 0x7fba1c0746c0 2024-06-06T06:57:27.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.589+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba1c19c140 con 0x7fba1c0746c0 2024-06-06T06:57:27.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.590+0000 7fba18ff9700 1 -- 172.21.15.73:0/2824570640 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7fba0c09d2b0 con 0x7fba1c0746c0 2024-06-06T06:57:27.591 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:27.591 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24404,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:57:24.198512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24433,"qdb_cluster":[24433]},"id":1}]} 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 msgr2=0x7fba080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 0x7fba080a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fba04005fd0 tx=0x7fba04005d00 comp rx=0 tx=0).stop 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 msgr2=0x7fba1c19ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fba0c00bff0 tx=0x7fba0c009f90 comp rx=0 tx=0).stop 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 shutdown_connections 2024-06-06T06:57:27.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fba080a2c20 0x7fba080a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba1c10f160 0x7fba1c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba1c0746c0 0x7fba1c19ade0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 --2- 172.21.15.73:0/2824570640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba1c074fe0 0x7fba1c19b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 >> 172.21.15.73:0/2824570640 conn(0x7fba1c07a420 msgr2=0x7fba1c0ff930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 shutdown_connections 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:27.592+0000 7fba21526700 1 -- 172.21.15.73:0/2824570640 wait complete. 2024-06-06T06:57:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 17 2024-06-06T06:57:28.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:57:28.574 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:28.696 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:28 smithi073 ceph-mon[32303]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:28.696 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:28 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2824570640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:28.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:28 smithi136 ceph-mon[35901]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:28.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:28 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2824570640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:28 smithi170 ceph-mon[36108]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:28 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2824570640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:30.473 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:30 smithi073 ceph-mon[32303]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:30.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 -- 172.21.15.73:0/1464454794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99281073f0 msgr2=0x7f99281077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:30.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 --2- 172.21.15.73:0/1464454794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99281073f0 0x7f99281077d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f992000b0a0 tx=0x7f992000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 -- 172.21.15.73:0/1464454794 shutdown_connections 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 --2- 172.21.15.73:0/1464454794 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9928101d40 0x7f992810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 --2- 172.21.15.73:0/1464454794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99281073f0 0x7f99281077d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 --2- 172.21.15.73:0/1464454794 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f9928101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 -- 172.21.15.73:0/1464454794 >> 172.21.15.73:0/1464454794 conn(0x7f9928078300 msgr2=0x7f9928078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 -- 172.21.15.73:0/1464454794 shutdown_connections 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.472+0000 7f992f788700 1 -- 172.21.15.73:0/1464454794 wait complete. 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 Processor -- start 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 -- start start 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f992819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99281073f0 0x7f99281988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9928112660 con 0x7f9928101d40 2024-06-06T06:57:30.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f99281124e0 con 0x7f9928101380 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992f788700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f99281127e0 con 0x7f99281073f0 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60780/0 (socket says 172.21.15.73:60780) 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 -- 172.21.15.73:0/4185918876 learned_addr learned my addr 172.21.15.73:0/4185918876 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992dd25700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99281073f0 0x7f99281988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992d524700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f992819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 -- 172.21.15.73:0/4185918876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99281073f0 msgr2=0x7f99281988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99281073f0 0x7f99281988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 -- 172.21.15.73:0/4185918876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 msgr2=0x7f992819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f992819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992cd23700 1 -- 172.21.15.73:0/4185918876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9928199000 con 0x7f9928101d40 2024-06-06T06:57:30.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.473+0000 7f992d524700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f992819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f992cd23700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c9b0 tx=0x7f991c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c007b00 con 0x7f9928101d40 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f991c007ca0 con 0x7f9928101d40 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99281992f0 con 0x7f9928101d40 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991c018730 con 0x7f9928101d40 2024-06-06T06:57:30.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.474+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99281a55b0 con 0x7f9928101d40 2024-06-06T06:57:30.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.475+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9928069180 con 0x7f9928101d40 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.476+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f991c00f450 con 0x7f9928101d40 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.477+0000 7f99267fc700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 0x7f99100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.477+0000 7f992d524700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 0x7f99100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.477+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f991c0cf340 con 0x7f9928101d40 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.477+0000 7f992d524700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 0x7f99100a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9920009e60 tx=0x7f992000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:30.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.481+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f991c014030 con 0x7f9928101d40 2024-06-06T06:57:30.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.679+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f992802dca0 con 0x7f9928101d40 2024-06-06T06:57:30.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.681+0000 7f99267fc700 1 -- 172.21.15.73:0/4185918876 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24101 (secure 0 0 0) 0x7f991c09c060 con 0x7f9928101d40 2024-06-06T06:57:30.682 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:30.682 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":96,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:24.198381+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 msgr2=0x7f99100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 0x7f99100a50e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9920009e60 tx=0x7f992000b7b0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 msgr2=0x7f992819ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f991c00c9b0 tx=0x7f991c00cd70 comp rx=0 tx=0).stop 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 shutdown_connections 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f99100a2c20 0x7f99100a50e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f99281073f0 0x7f99281988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9928101380 0x7f992819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 --2- 172.21.15.73:0/4185918876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9928101d40 0x7f992819ec20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:30.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 >> 172.21.15.73:0/4185918876 conn(0x7f9928078300 msgr2=0x7f9928100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:30.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 shutdown_connections 2024-06-06T06:57:30.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:30.683+0000 7f992f788700 1 -- 172.21.15.73:0/4185918876 wait complete. 2024-06-06T06:57:30.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:30 smithi136 ceph-mon[35901]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:30 smithi170 ceph-mon[36108]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:31.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:57:31.706 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:31 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4185918876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:31 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4185918876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:31 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4185918876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:32.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:32 smithi170 ceph-mon[36108]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:32.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:32 smithi073 ceph-mon[32303]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:32 smithi136 ceph-mon[35901]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:33.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:33 smithi073 ceph-mon[32303]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:33.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.481+0000 7fe50842f700 1 -- 172.21.15.73:0/3831663366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe500108560 msgr2=0x7fe500108940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.481+0000 7fe50842f700 1 --2- 172.21.15.73:0/3831663366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe500108560 0x7fe500108940 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc0099c0 tx=0x7fe4fc009cd0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.481+0000 7fe50842f700 1 -- 172.21.15.73:0/3831663366 shutdown_connections 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.481+0000 7fe50842f700 1 --2- 172.21.15.73:0/3831663366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5001013a0 0x7fe50010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 --2- 172.21.15.73:0/3831663366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe500108560 0x7fe500108940 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 --2- 172.21.15.73:0/3831663366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001009e0 0x7fe500100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- 172.21.15.73:0/3831663366 >> 172.21.15.73:0/3831663366 conn(0x7fe500077990 msgr2=0x7fe5000fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- 172.21.15.73:0/3831663366 shutdown_connections 2024-06-06T06:57:33.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- 172.21.15.73:0/3831663366 wait complete. 2024-06-06T06:57:33.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 Processor -- start 2024-06-06T06:57:33.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- start start 2024-06-06T06:57:33.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:33.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001013a0 0x7fe50019edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 0x7fe500198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe500112740 con 0x7fe5001009e0 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001125c0 con 0x7fe500108560 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe50842f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5001128c0 con 0x7fe5001013a0 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.482+0000 7fe5061cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60796/0 (socket says 172.21.15.73:60796) 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 -- 172.21.15.73:0/2404430863 learned_addr learned my addr 172.21.15.73:0/2404430863 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5059ca700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001013a0 0x7fe50019edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5069cc700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 0x7fe500198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 -- 172.21.15.73:0/2404430863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001013a0 msgr2=0x7fe50019edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001013a0 0x7fe50019edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 -- 172.21.15.73:0/2404430863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 msgr2=0x7fe500198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 0x7fe500198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 -- 172.21.15.73:0/2404430863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5001991a0 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5069cc700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 0x7fe500198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe5061cb700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc006010 tx=0x7fe4fc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc027070 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4fc004050 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe500199430 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4fc022e10 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.483+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5001a5750 con 0x7fe5001009e0 2024-06-06T06:57:33.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.484+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50010a9f0 con 0x7fe5001009e0 2024-06-06T06:57:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.485+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe4fc02a070 con 0x7fe5001009e0 2024-06-06T06:57:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.486+0000 7fe4f77fe700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 0x7fe4ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.486+0000 7fe5059ca700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 0x7fe4ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.486+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7fe4fc0d0870 con 0x7fe5001009e0 2024-06-06T06:57:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.486+0000 7fe5059ca700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 0x7fe4ec0a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:33.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.490+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe4fc031020 con 0x7fe5001009e0 2024-06-06T06:57:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.685+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe500199f70 con 0x7fe5001009e0 2024-06-06T06:57:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.687+0000 7fe4f77fe700 1 -- 172.21.15.73:0/2404430863 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24101 (secure 0 0 0) 0x7fe4fc013070 con 0x7fe5001009e0 2024-06-06T06:57:33.688 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:33.688 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":96,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:24.198381+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:57:33.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 msgr2=0x7fe4ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:33.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 0x7fe4ec0a5160 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).stop 2024-06-06T06:57:33.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 msgr2=0x7fe50019e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:33.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc006010 tx=0x7fe4fc003680 comp rx=0 tx=0).stop 2024-06-06T06:57:33.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 shutdown_connections 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe500108560 0x7fe500198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5001009e0 0x7fe50019e880 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe4ec0a2ca0 0x7fe4ec0a5160 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 --2- 172.21.15.73:0/2404430863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5001013a0 0x7fe50019edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 >> 172.21.15.73:0/2404430863 conn(0x7fe500077990 msgr2=0x7fe5000ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 shutdown_connections 2024-06-06T06:57:33.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:33.689+0000 7fe50842f700 1 -- 172.21.15.73:0/2404430863 wait complete. 2024-06-06T06:57:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:33 smithi136 ceph-mon[35901]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:33 smithi170 ceph-mon[36108]: pgmap v317: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:34.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:57:34.703 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:34.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:34 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2404430863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:34.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:34 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2404430863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:34.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:34 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2404430863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:35 smithi073 ceph-mon[32303]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:35 smithi136 ceph-mon[35901]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:35 smithi170 ceph-mon[36108]: pgmap v318: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/3280215617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806ac00 msgr2=0x7f6a3806b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/3280215617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806ac00 0x7f6a3806b080 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009990 tx=0x7f6a2c009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/3280215617 shutdown_connections 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/3280215617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806b5c0 0x7f6a3810f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/3280215617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/3280215617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806ac00 0x7f6a3806b080 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/3280215617 >> 172.21.15.73:0/3280215617 conn(0x7f6a38077400 msgr2=0x7f6a38077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/3280215617 shutdown_connections 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/3280215617 wait complete. 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 Processor -- start 2024-06-06T06:57:36.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.439+0000 7f6a3e3c4700 1 -- start start 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 0x7f6a3819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38112610 con 0x7f6a3806b5c0 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38112490 con 0x7f6a38108530 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3e3c4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a38112790 con 0x7f6a3806ac00 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46132/0 (socket says 172.21.15.73:46132) 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 -- 172.21.15.73:0/1328933286 learned_addr learned my addr 172.21.15.73:0/1328933286 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a37fff700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 0x7f6a3819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3c961700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 -- 172.21.15.73:0/1328933286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 msgr2=0x7f6a3819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 0x7f6a3819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 -- 172.21.15.73:0/1328933286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 msgr2=0x7f6a38198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 -- 172.21.15.73:0/1328933286 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a381990b0 con 0x7f6a3806b5c0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a37fff700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 0x7f6a3819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a3c961700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a377fe700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c005f50 tx=0x7f6a2c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.440+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c027070 con 0x7f6a3806b5c0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.441+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a38199340 con 0x7f6a3806b5c0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.441+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a2c004010 con 0x7f6a3806b5c0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.441+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c022d30 con 0x7f6a3806b5c0 2024-06-06T06:57:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.441+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a381a5660 con 0x7f6a3806b5c0 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.441+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a38103bf0 con 0x7f6a3806b5c0 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.443+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f6a2c02a070 con 0x7f6a3806b5c0 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.443+0000 7f6a357fa700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 0x7f6a200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.443+0000 7f6a37fff700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 0x7f6a200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.443+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f6a2c0d0980 con 0x7f6a3806b5c0 2024-06-06T06:57:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.444+0000 7f6a37fff700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 0x7f6a200a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6a38199a90 tx=0x7f6a28005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:36.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.447+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a2c031020 con 0x7f6a3806b5c0 2024-06-06T06:57:36.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.684+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a38199e80 con 0x7f6a3806b5c0 2024-06-06T06:57:36.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.685+0000 7f6a357fa700 1 -- 172.21.15.73:0/1328933286 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11797 (secure 0 0 0) 0x7f6a2c013070 con 0x7f6a3806b5c0 2024-06-06T06:57:36.686 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:36.687 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14571,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14583,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24365,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24380,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24404,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:57:24.198512+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24433},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24433,"qdb_cluster":[24433]},"id":1}]} 2024-06-06T06:57:36.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 msgr2=0x7f6a200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:36.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 0x7f6a200a50e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6a38199a90 tx=0x7f6a28005d80 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 msgr2=0x7f6a3819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c005f50 tx=0x7f6a2c0057c0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 shutdown_connections 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a200a2c20 0x7f6a200a50e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a3806ac00 0x7f6a3819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a38108530 0x7f6a38198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 --2- 172.21.15.73:0/1328933286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a3806b5c0 0x7f6a3819ecd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 >> 172.21.15.73:0/1328933286 conn(0x7f6a38077400 msgr2=0x7f6a38100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 shutdown_connections 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:36.687+0000 7f6a3e3c4700 1 -- 172.21.15.73:0/1328933286 wait complete. 2024-06-06T06:57:36.689 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 17 2024-06-06T06:57:37.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:36 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1328933286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:37.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:36 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1328933286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:37.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:36 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1328933286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:57:37.487 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:57:37.736 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:38.006 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:37 smithi073 ceph-mon[32303]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:38.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:37 smithi136 ceph-mon[35901]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:38.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:37 smithi170 ceph-mon[36108]: pgmap v319: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 -- 172.21.15.73:0/2423759803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 msgr2=0x7f0504107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 --2- 172.21.15.73:0/2423759803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f0504107110 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f04f80099c0 tx=0x7f04f8009cd0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 -- 172.21.15.73:0/2423759803 shutdown_connections 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 --2- 172.21.15.73:0/2423759803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504107840 0x7f050410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 --2- 172.21.15.73:0/2423759803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504102740 0x7f0504102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 --2- 172.21.15.73:0/2423759803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f0504107110 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 -- 172.21.15.73:0/2423759803 >> 172.21.15.73:0/2423759803 conn(0x7f05040795f0 msgr2=0x7f0504079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 -- 172.21.15.73:0/2423759803 shutdown_connections 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.591+0000 7f050a99f700 1 -- 172.21.15.73:0/2423759803 wait complete. 2024-06-06T06:57:39.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 Processor -- start 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 -- start start 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504102740 0x7f050419e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 0x7f05041988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0504112620 con 0x7f05041030f0 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05041124a0 con 0x7f0504107840 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f050a99f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05041127a0 con 0x7f0504102740 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f05037fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f05037fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46146/0 (socket says 172.21.15.73:46146) 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f05037fe700 1 -- 172.21.15.73:0/1293578226 learned_addr learned my addr 172.21.15.73:0/1293578226 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f0503fff700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504102740 0x7f050419e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:39.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.592+0000 7f0508f3c700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 0x7f05041988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 -- 172.21.15.73:0/1293578226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504102740 msgr2=0x7f050419e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504102740 0x7f050419e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 -- 172.21.15.73:0/1293578226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 msgr2=0x7f05041988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 0x7f05041988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 -- 172.21.15.73:0/1293578226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0504199000 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f0508f3c700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 0x7f05041988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05037fe700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04f8006010 tx=0x7f04f8009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8027070 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04f8003f50 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8022cd0 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0504199290 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.593+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05041a55b0 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.594+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0504103bd0 con 0x7f05041030f0 2024-06-06T06:57:39.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.595+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f04f802a070 con 0x7f05041030f0 2024-06-06T06:57:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.596+0000 7f05017fa700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 0x7f04ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.596+0000 7f0503fff700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 0x7f04ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.596+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f04f80d0360 con 0x7f05041030f0 2024-06-06T06:57:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.596+0000 7f0503fff700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 0x7f04ec0a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05041999e0 tx=0x7f04f400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:39.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.599+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f04f8031080 con 0x7f05041030f0 2024-06-06T06:57:39.799 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.799+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f050402dca0 con 0x7f05041030f0 2024-06-06T06:57:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.800+0000 7f05017fa700 1 -- 172.21.15.73:0/1293578226 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+24101 (secure 0 0 0) 0x7f04f8013070 con 0x7f05041030f0 2024-06-06T06:57:39.801 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:39.801 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":96,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:24.198381+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:57:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.802+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 msgr2=0x7f04ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.802+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 0x7f04ec0a5160 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f05041999e0 tx=0x7f04f400a380 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.802+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 msgr2=0x7f050419ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.802+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f04f8006010 tx=0x7f04f8009f80 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 shutdown_connections 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f04ec0a2ca0 0x7f04ec0a5160 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0504102740 0x7f050419e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0504107840 0x7f05041988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 --2- 172.21.15.73:0/1293578226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05041030f0 0x7f050419ec20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 >> 172.21.15.73:0/1293578226 conn(0x7f05040795f0 msgr2=0x7f0504100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 shutdown_connections 2024-06-06T06:57:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:39.803+0000 7f050a99f700 1 -- 172.21.15.73:0/1293578226 wait complete. 2024-06-06T06:57:40.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-06-06T06:57:09.432568+0000', 'modified': '2024-06-06T06:57:24.198512+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24433}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.170:6839/3230829204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3230829204}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3230829204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24433, 'qdb_cluster': [24433]} 2024-06-06T06:57:40.584 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-06-06T06:57:40.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T06:57:40.609 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:40 smithi073 ceph-mon[32303]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:40.609 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:40 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1293578226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:40.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:40 smithi136 ceph-mon[35901]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:40.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:40 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1293578226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:40.828 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:40 smithi170 ceph-mon[36108]: pgmap v320: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:40 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1293578226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 -- 172.21.15.73:0/1119336876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044100a80 msgr2=0x7f4044100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 --2- 172.21.15.73:0/1119336876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044100a80 0x7f4044100f00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4038009990 tx=0x7f4038009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 -- 172.21.15.73:0/1119336876 shutdown_connections 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 --2- 172.21.15.73:0/1119336876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 0x7f40441088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 --2- 172.21.15.73:0/1119336876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044101440 0x7f404410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 --2- 172.21.15.73:0/1119336876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044100a80 0x7f4044100f00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 -- 172.21.15.73:0/1119336876 >> 172.21.15.73:0/1119336876 conn(0x7f4044077a00 msgr2=0x7f40440fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 -- 172.21.15.73:0/1119336876 shutdown_connections 2024-06-06T06:57:42.550 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.548+0000 7f4048d7e700 1 -- 172.21.15.73:0/1119336876 wait complete. 2024-06-06T06:57:42.551 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:42 smithi073 ceph-mon[32303]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:42.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 Processor -- start 2024-06-06T06:57:42.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 -- start start 2024-06-06T06:57:42.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 0x7f404419e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 0x7f4044198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f40441126a0 con 0x7f4044101440 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4044112520 con 0x7f4044100a80 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.549+0000 7f4048d7e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4044112820 con 0x7f4044108510 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46168/0 (socket says 172.21.15.73:46168) 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 -- 172.21.15.73:0/3935082220 learned_addr learned my addr 172.21.15.73:0/3935082220 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f404259c700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 0x7f404419e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4042d9d700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 0x7f4044198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 -- 172.21.15.73:0/3935082220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 msgr2=0x7f4044198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 0x7f4044198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 -- 172.21.15.73:0/3935082220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 msgr2=0x7f404419e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 0x7f404419e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 -- 172.21.15.73:0/3935082220 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40441991c0 con 0x7f4044101440 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f404259c700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 0x7f404419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4041d9b700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f4038003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f40337fe700 1 -- 172.21.15.73:0/3935082220 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038027070 con 0x7f4044101440 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4044199450 con 0x7f4044101440 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.550+0000 7f40337fe700 1 -- 172.21.15.73:0/3935082220 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403800bdc0 con 0x7f4044101440 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.551+0000 7f40337fe700 1 -- 172.21.15.73:0/3935082220 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403800f6c0 con 0x7f4044101440 2024-06-06T06:57:42.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.551+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40441a5770 con 0x7f4044101440 2024-06-06T06:57:42.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.553+0000 7f40337fe700 1 -- 172.21.15.73:0/3935082220 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f403802a070 con 0x7f4044101440 2024-06-06T06:57:42.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.553+0000 7f40337fe700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 0x7f402c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.553+0000 7f404259c700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 0x7f402c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.553+0000 7f40337fe700 1 -- 172.21.15.73:0/3935082220 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f40380d0440 con 0x7f4044101440 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.554+0000 7f404259c700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 0x7f402c0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4044199ba0 tx=0x7f4034008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.554+0000 7f4053601b80 1 Processor -- start 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 -- start start 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 0x55cdfa2cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa1c1f10 con 0x55cdfa2cf750 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa220800 con 0x55cdfa2d5830 2024-06-06T06:57:42.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4053601b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa188410 con 0x55cdfa2d4f10 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46172/0 (socket says 172.21.15.73:46172) 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 -- 172.21.15.73:0/2705380617 learned_addr learned my addr 172.21.15.73:0/2705380617 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f404259c700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4041d9b700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 0x55cdfa2cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 -- 172.21.15.73:0/2705380617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d4f10 msgr2=0x55cdfa2d52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.560 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 -- 172.21.15.73:0/2705380617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 msgr2=0x55cdfa2cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 0x55cdfa2cf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 -- 172.21.15.73:0/2705380617 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdfa16f880 con 0x55cdfa2cf750 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4041d9b700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 0x55cdfa2cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.555+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f403c008550 tx=0x7f403c008860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b79abbc940e6e01 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4030ff9700 1 -- 172.21.15.73:0/2705380617 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c003c00 con 0x55cdfa2cf750 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4030ff9700 1 -- 172.21.15.73:0/2705380617 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c003da0 con 0x55cdfa2cf750 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4030ff9700 1 -- 172.21.15.73:0/2705380617 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c0128d0 con 0x55cdfa2cf750 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 -- 172.21.15.73:0/2705380617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 msgr2=0x55cdfa2cfb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f403c008550 tx=0x7f403c008860 comp rx=0 tx=0).stop 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 -- 172.21.15.73:0/2705380617 shutdown_connections 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cfb70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 --2- 172.21.15.73:0/2705380617 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d5830 0x55cdfa2cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.561 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.556+0000 7f4053601b80 1 -- 172.21.15.73:0/2705380617 >> 172.21.15.73:0/2705380617 conn(0x55cdfa1fc030 msgr2=0x55cdfa2ca1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- 172.21.15.73:0/2705380617 shutdown_connections 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- 172.21.15.73:0/2705380617 wait complete. 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- start start 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 Processor -- start 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2cf750 0x55cdfa2db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d5830 0x55cdfa2d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa1c0f70 con 0x55cdfa2d4f10 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa21c7c0 con 0x55cdfa2cf750 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2dc210 con 0x55cdfa2d5830 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4041d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4053601b80 10 client.0 _ll_register_callbacks cb 0x55cdfa1b6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T06:57:42.562 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4041d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46180/0 (socket says 172.21.15.73:46180) 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4041d9b700 1 -- 172.21.15.73:0/1602154880 learned_addr learned my addr 172.21.15.73:0/1602154880 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f404259c700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2cf750 0x55cdfa2db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.557+0000 7f4042d9d700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d5830 0x55cdfa2d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 -- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d5830 msgr2=0x55cdfa2d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d5830 0x55cdfa2d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 -- 172.21.15.73:0/1602154880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2cf750 msgr2=0x55cdfa2db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2cf750 0x55cdfa2db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 -- 172.21.15.73:0/1602154880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdfa1e6700 con 0x55cdfa2d4f10 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4041d9b700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f40380098e0 tx=0x7f4038004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a5edcefeab7aff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380a0ea0 con 0x55cdfa2d4f10 2024-06-06T06:57:42.563 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f40380d0240 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c003da0 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f40380a56d0 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4053601b80 10 client.14682 fetch_fsmap learned FSMap version 17 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cdfa2e6960 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c003da0 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.558+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f40380a52d0 con 0x55cdfa2d4f10 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 10 client.14682 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 10 client.14682 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 20 client.14682 populate_metadata read hostname 'smithi073' 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa2e9010 0x55cdfa2ea480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 4 client.14682 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdfa22dd00 con 0x55cdfa2e9010 2024-06-06T06:57:42.564 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f404259c700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa2e9010 0x55cdfa2ea480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f404259c700 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa2e9010 0x55cdfa2ea480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3cb756a55ffacc server_cookie=11ecb48e4ed170bb in_seq=0 out_seq=0 2024-06-06T06:57:42.565 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.559+0000 7f4029ffb700 10 client.14682 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T06:57:42.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4029ffb700 1 -- 172.21.15.73:0/1602154880 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cdfa22dd00 con 0x55cdfa2e9010 2024-06-06T06:57:42.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4029ffb700 10 client.14682 handle_command_reply: tid=0 2024-06-06T06:57:42.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 client.14682 shutdown 2024-06-06T06:57:42.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 msgr2=0x55cdfa2d88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.566 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f40380098e0 tx=0x7f4038004500 comp rx=0 tx=0).stop 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 shutdown_connections 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa2e9010 0x55cdfa2ea480 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.560+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2d5830 0x55cdfa2d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4029ffb700 0 client.14682 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2cf750 0x55cdfa2db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 1 --2- 172.21.15.73:0/1602154880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2d88d0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 >> 172.21.15.73:0/1602154880 conn(0x55cdfa1fc030 msgr2=0x55cdfa2c96b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 shutdown_connections 2024-06-06T06:57:42.567 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 1 -- 172.21.15.73:0/1602154880 wait complete. 2024-06-06T06:57:42.568 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.561+0000 7f4053601b80 20 client.14682 trim_cache size 0 max 16384 2024-06-06T06:57:42.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 Processor -- start 2024-06-06T06:57:42.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 -- start start 2024-06-06T06:57:42.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2e87c0 con 0x55cdfa2d5830 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2e8e90 con 0x55cdfa2d4f10 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4053601b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2e9560 con 0x55cdfa2cf750 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46192/0 (socket says 172.21.15.73:46192) 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 -- 172.21.15.73:0/4005667244 learned_addr learned my addr 172.21.15.73:0/4005667244 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 -- 172.21.15.73:0/4005667244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 msgr2=0x55cdfa2cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 -- 172.21.15.73:0/4005667244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 msgr2=0x55cdfa2c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 -- 172.21.15.73:0/4005667244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdfa2dbc80 con 0x55cdfa2d5830 2024-06-06T06:57:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.582+0000 7f4042d9d700 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f403c008550 tx=0x7f403c0077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188974bdee7953be server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f402bfff700 1 -- 172.21.15.73:0/4005667244 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c023020 con 0x55cdfa2d5830 2024-06-06T06:57:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f402bfff700 1 -- 172.21.15.73:0/4005667244 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c005040 con 0x55cdfa2d5830 2024-06-06T06:57:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f402bfff700 1 -- 172.21.15.73:0/4005667244 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c012740 con 0x55cdfa2d5830 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 -- 172.21.15.73:0/4005667244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 msgr2=0x55cdfa2ca370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f403c008550 tx=0x7f403c0077a0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 -- 172.21.15.73:0/4005667244 shutdown_connections 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2cf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca370 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 --2- 172.21.15.73:0/4005667244 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 -- 172.21.15.73:0/4005667244 >> 172.21.15.73:0/4005667244 conn(0x55cdfa1fc030 msgr2=0x55cdfa2d71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 -- 172.21.15.73:0/4005667244 shutdown_connections 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.583+0000 7f4053601b80 1 -- 172.21.15.73:0/4005667244 wait complete. 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 -- start start 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 Processor -- start 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2ea300 con 0x55cdfa2d5830 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2e9c30 con 0x55cdfa2d4f10 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cdfa2ea9d0 con 0x55cdfa2cf750 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4053601b80 10 client.0 _ll_register_callbacks cb 0x55cdfa1b6560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46208/0 (socket says 172.21.15.73:46208) 2024-06-06T06:57:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 -- 172.21.15.73:0/2123302747 learned_addr learned my addr 172.21.15.73:0/2123302747 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f404259c700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4041d9b700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 -- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 msgr2=0x55cdfa2c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 -- 172.21.15.73:0/2123302747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 msgr2=0x55cdfa2c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4042d9d700 1 -- 172.21.15.73:0/2123302747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdfa2f2200 con 0x55cdfa2d5830 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.584+0000 7f4041d9b700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4042d9d700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f403c007400 tx=0x7f403c005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb8d49c0e398d88b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c012500 con 0x55cdfa2d5830 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f403c0126a0 con 0x55cdfa2d5830 2024-06-06T06:57:42.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403c005040 con 0x55cdfa2d5830 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f403c0128a0 con 0x55cdfa2d5830 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 10 client.14688 fetch_fsmap learned FSMap version 17 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55cdfa354cb0 con 0x55cdfa2d5830 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f403c005040 con 0x55cdfa2d5830 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f403c013070 con 0x55cdfa2d5830 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 10 client.14688 fetch_fsmap finished waiting for FSMap version 17 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 10 client.14688 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 20 client.14688 populate_metadata read hostname 'smithi073' 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa359130 0x55cdfa357e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:42.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 4 client.14688 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "status"}) -- 0x55cdfa228280 con 0x55cdfa359130 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.585+0000 7f404259c700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa359130 0x55cdfa357e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4029ffb700 10 client.14688 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f404259c700 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa359130 0x55cdfa357e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37754da40ea41997 server_cookie=bab9613b565a5a6c in_seq=0 out_seq=0 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4029ffb700 1 -- 172.21.15.73:0/2123302747 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55cdfa228280 con 0x55cdfa359130 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4029ffb700 10 client.14688 handle_command_reply: tid=0 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 client.14688 shutdown 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 msgr2=0x55cdfa2ca280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f403c007400 tx=0x7f403c005420 comp rx=0 tx=0).stop 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 shutdown_connections 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cdfa359130 0x55cdfa357e40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4029ffb700 0 client.14688 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T06:57:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cdfa2cf750 0x55cdfa2c9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cdfa2d5830 0x55cdfa2ca280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 --2- 172.21.15.73:0/2123302747 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cdfa2d4f10 0x55cdfa2c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 >> 172.21.15.73:0/2123302747 conn(0x55cdfa1fc030 msgr2=0x55cdfa2d5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.586+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 shutdown_connections 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4053601b80 1 -- 172.21.15.73:0/2123302747 wait complete. 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4053601b80 20 client.14688 trim_cache size 0 max 16384 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T06:57:42.594 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24433, 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 33.043672919000002, 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 17, 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 96, 2024-06-06T06:57:42.595 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 96, 2024-06-06T06:57:42.596 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 58.855781624000002 2024-06-06T06:57:42.596 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 msgr2=0x7f402c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 0x7f402c0a50e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4044199ba0 tx=0x7f4034008040 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 msgr2=0x7f404419ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4038000c00 tx=0x7f4038003b30 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 shutdown_connections 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f402c0a2c20 0x7f402c0a50e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4044108510 0x7f4044198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4044100a80 0x7f404419e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 --2- 172.21.15.73:0/3935082220 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4044101440 0x7f404419ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:42.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 >> 172.21.15.73:0/3935082220 conn(0x7f4044077a00 msgr2=0x7f40440ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:42.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 shutdown_connections 2024-06-06T06:57:42.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:42.587+0000 7f4048d7e700 1 -- 172.21.15.73:0/3935082220 wait complete. 2024-06-06T06:57:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:42 smithi136 ceph-mon[35901]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:42.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:42 smithi170 ceph-mon[36108]: pgmap v321: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:43.390 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T06:57:43.391 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:43.391 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T06:57:43.670 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:44 smithi073 ceph-mon[32303]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:44 smithi136 ceph-mon[35901]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:44 smithi170 ceph-mon[36108]: pgmap v322: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:45.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.405+0000 7f612454d700 1 -- 172.21.15.73:0/3638167878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c1073c0 msgr2=0x7f611c06a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:45.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.405+0000 7f612454d700 1 --2- 172.21.15.73:0/3638167878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c1073c0 0x7f611c06a1a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f6118009990 tx=0x7f6118009ca0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 -- 172.21.15.73:0/3638167878 shutdown_connections 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 --2- 172.21.15.73:0/3638167878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c104ed0 0x7f611c105360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 --2- 172.21.15.73:0/3638167878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c1073c0 0x7f611c06a1a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 --2- 172.21.15.73:0/3638167878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c06a6e0 0x7f611c06ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 -- 172.21.15.73:0/3638167878 >> 172.21.15.73:0/3638167878 conn(0x7f611c078340 msgr2=0x7f611c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 -- 172.21.15.73:0/3638167878 shutdown_connections 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 -- 172.21.15.73:0/3638167878 wait complete. 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 Processor -- start 2024-06-06T06:57:45.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.406+0000 7f612454d700 1 -- start start 2024-06-06T06:57:45.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c06a6e0 0x7f611c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:45.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:45.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c1073c0 0x7f611c1a3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f611c118c20 con 0x7f611c104ed0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f611c118aa0 con 0x7f611c06a6e0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f611c118da0 con 0x7f611c1073c0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49782/0 (socket says 172.21.15.73:49782) 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 -- 172.21.15.73:0/360829765 learned_addr learned my addr 172.21.15.73:0/360829765 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f61222e9700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c06a6e0 0x7f611c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6122aea700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c1073c0 0x7f611c1a3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 -- 172.21.15.73:0/360829765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c1073c0 msgr2=0x7f611c1a3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c1073c0 0x7f611c1a3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 -- 172.21.15.73:0/360829765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c06a6e0 msgr2=0x7f611c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c06a6e0 0x7f611c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 -- 172.21.15.73:0/360829765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611c1a4140 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f6121ae8700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d980 tx=0x7f610c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c017070 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611c1a4430 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f610c004830 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c005440 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.407+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611c1abcb0 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.408+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f611c069180 con 0x7f611c104ed0 2024-06-06T06:57:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.409+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f610c0055e0 con 0x7f611c104ed0 2024-06-06T06:57:45.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.410+0000 7f61137fe700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 0x7f61080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:45.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.410+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8802+0+0 (secure 0 0 0) 0x7f610c0cf600 con 0x7f611c104ed0 2024-06-06T06:57:45.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.410+0000 7f61222e9700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 0x7f61080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:45.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.411+0000 7f61222e9700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 0x7f61080a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6118006010 tx=0x7f611801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:45.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.414+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f610c020020 con 0x7f611c104ed0 2024-06-06T06:57:45.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.654+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f611c1abe80 con 0x7f611c104ed0 2024-06-06T06:57:45.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.655+0000 7f61137fe700 1 -- 172.21.15.73:0/360829765 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f610c09c320 con 0x7f611c104ed0 2024-06-06T06:57:45.655 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 msgr2=0x7f61080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 0x7f61080a50e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6118006010 tx=0x7f611801a040 comp rx=0 tx=0).stop 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 msgr2=0x7f611c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f610c00d980 tx=0x7f610c00dd40 comp rx=0 tx=0).stop 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 shutdown_connections 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f61080a2c20 0x7f61080a50e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f611c1073c0 0x7f611c1a3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f611c06a6e0 0x7f611c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 --2- 172.21.15.73:0/360829765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f611c104ed0 0x7f611c19f670 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:45.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 >> 172.21.15.73:0/360829765 conn(0x7f611c078340 msgr2=0x7f611c100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:45.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 shutdown_connections 2024-06-06T06:57:45.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:45.657+0000 7f612454d700 1 -- 172.21.15.73:0/360829765 wait complete. 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T06:57:46.366 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T06:57:46.367 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T06:57:46.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:46.367 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-06-06T06:57:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42217sec preferred_lft 42217sec 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-06-06T06:57:46.387 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T06:57:46.388 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T06:57:46.388 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T06:57:46.388 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T06:57:46.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-06-06T06:57:46.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-06-06T06:57:46.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-06-06T06:57:46.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T06:57:46.389 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T06:57:46.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T06:57:46.591 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:46 smithi073 ceph-mon[32303]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:46.591 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/360829765' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T06:57:46.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:46.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:46.623 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T06:57:46.648 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-06-06T06:57:46.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:46.650 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-06-06T06:57:46.716 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-06-06T06:57:46.716 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T06:57:46.716 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-06T06:57:46.716 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-06T06:57:46.716 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-06T06:57:46.717 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T06:57:46.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:46 smithi136 ceph-mon[35901]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:46.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:46 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/360829765' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T06:57:46.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:46 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T06:57:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:46 smithi170 ceph-mon[36108]: pgmap v323: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:46 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/360829765' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T06:57:47.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:47.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:47.027 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-06-06T06:57:47.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:47.043 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-06-06T06:57:47.098 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T06:57:47.098 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T06:57:47.098 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:47.098 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-06-06T06:57:47.099 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T06:57:47.187 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T06:57:47.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:47.217 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-06-06T06:57:47.217 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T06:57:47.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T06:57:47.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:47.357 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T06:57:47.357 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T06:57:47.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-06-06T06:57:47.357 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T06:57:47.357 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T06:57:47.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T06:57:47.429 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:47.435 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-06-06T06:57:47.435 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T06:57:47.435 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:47.450 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-06-06T06:57:47.450 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T06:57:47.450 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:47.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-06-06T06:57:47.572 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T06:57:47.629 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-06-06T06:57:47.629 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-06-06T06:57:47.629 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-06-06T06:57:47.629 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-06-06T06:57:47.630 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/10bdc6c9845f3c7e810a1f43fac4fbdc3c7fc347cceee132a6dd3766b6f20e35/merged 2024-06-06T06:57:47.631 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/872647e4eceff223a218675b1b1a57cacdbe7476433ffbbed2a09dde2dc380c0/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/0 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/5f122e402711f91bdb948a37ab4427a2cb160c4613281add4823ce6c3f234454/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/f8301984a014f1ee316b3ef130c4695a127a82eaa5b4327a54a4855b36c128a3/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9dfdb2a592c6db3b58ddb437ebca3591698c75d0f78a4b70557fb56576cd263b/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c0d0882dec388bae6e209ba50dc3afda1427df6cc7348813220120d496780c3e/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ca0b6a041d93f68e6a6e98e8435cab60a48e6a12c5c2be13727794df338edf8/merged 2024-06-06T06:57:47.632 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9d8a3b5d1f0f53e754c3a2710ff933b5c10bda06579b354b33699c61590da80a/merged 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ed2ab3935eea85dcbf9bbf5450eae68da9f481f751b2747095bd1cff9de0e97/merged 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c9c41226ab9529da15d5d56ce6671e5de43eecc0d2b8db8db6fec85947d1bb95/merged 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:47.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:47.633 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-06-06T06:57:47.685 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T06:57:47.685 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-06-06T06:57:47.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-06-06T06:57:47.753 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T06:57:47.809 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-06-06T06:57:47.810 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-06-06T06:57:47.811 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/10bdc6c9845f3c7e810a1f43fac4fbdc3c7fc347cceee132a6dd3766b6f20e35/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/872647e4eceff223a218675b1b1a57cacdbe7476433ffbbed2a09dde2dc380c0/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/0 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/5f122e402711f91bdb948a37ab4427a2cb160c4613281add4823ce6c3f234454/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/f8301984a014f1ee316b3ef130c4695a127a82eaa5b4327a54a4855b36c128a3/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9dfdb2a592c6db3b58ddb437ebca3591698c75d0f78a4b70557fb56576cd263b/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c0d0882dec388bae6e209ba50dc3afda1427df6cc7348813220120d496780c3e/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ca0b6a041d93f68e6a6e98e8435cab60a48e6a12c5c2be13727794df338edf8/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9d8a3b5d1f0f53e754c3a2710ff933b5c10bda06579b354b33699c61590da80a/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ed2ab3935eea85dcbf9bbf5450eae68da9f481f751b2747095bd1cff9de0e97/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c9c41226ab9529da15d5d56ce6671e5de43eecc0d2b8db8db6fec85947d1bb95/merged 2024-06-06T06:57:47.812 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-06-06T06:57:47.813 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:47.813 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:47.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:47.813 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-06-06T06:57:47.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:2024-06-06T06:57:47.861+0000 7fe085a81480 -1 init, newargv = 0x55a09bb5d8b0 newargc=15 2024-06-06T06:57:47.861 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[78264]: starting ceph client 2024-06-06T06:57:47.868 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[78264]: starting fuse 2024-06-06T06:57:48.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:48 smithi073 ceph-mon[32303]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:48 smithi136 ceph-mon[35901]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:48 smithi170 ceph-mon[36108]: pgmap v324: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:48.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-06-06T06:57:48.999 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-06-06T06:57:49.061 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T06:57:49.062 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/10bdc6c9845f3c7e810a1f43fac4fbdc3c7fc347cceee132a6dd3766b6f20e35/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/872647e4eceff223a218675b1b1a57cacdbe7476433ffbbed2a09dde2dc380c0/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/0 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/5f122e402711f91bdb948a37ab4427a2cb160c4613281add4823ce6c3f234454/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/f8301984a014f1ee316b3ef130c4695a127a82eaa5b4327a54a4855b36c128a3/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9dfdb2a592c6db3b58ddb437ebca3591698c75d0f78a4b70557fb56576cd263b/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c0d0882dec388bae6e209ba50dc3afda1427df6cc7348813220120d496780c3e/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ca0b6a041d93f68e6a6e98e8435cab60a48e6a12c5c2be13727794df338edf8/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9d8a3b5d1f0f53e754c3a2710ff933b5c10bda06579b354b33699c61590da80a/merged 2024-06-06T06:57:49.063 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ed2ab3935eea85dcbf9bbf5450eae68da9f481f751b2747095bd1cff9de0e97/merged 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c9c41226ab9529da15d5d56ce6671e5de43eecc0d2b8db8db6fec85947d1bb95/merged 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:/home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:49.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:49.064 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-06-06T06:57:49.119 INFO:teuthology.orchestra.run.smithi073.stdout:119 2024-06-06T06:57:49.119 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-06T06:57:49.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo stdin-killer -- python3 -c ' 2024-06-06T06:57:49.119 DEBUG:teuthology.orchestra.run.smithi073:> import glob 2024-06-06T06:57:49.119 DEBUG:teuthology.orchestra.run.smithi073:> import re 2024-06-06T06:57:49.119 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-06-06T06:57:49.119 DEBUG:teuthology.orchestra.run.smithi073:> import subprocess 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> def _find_admin_socket(client_name): 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> files = glob.glob(asok_path) 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> # Given a non-glob path, it better be there 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> if "*" not in asok_path: 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> assert(len(files) == 1) 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> return files[0] 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> for f in files: 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T06:57:49.120 DEBUG:teuthology.orchestra.run.smithi073:> contents = proc_f.read() 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> if mountpoint in contents: 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> return f 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> print(_find_admin_socket("client.admin")) 2024-06-06T06:57:49.121 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-06-06T06:57:49.246 INFO:teuthology.orchestra.run.smithi073.stdout:/var/run/ceph/ceph-client.admin.78264.asok 2024-06-06T06:57:49.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:49.257 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78264.asok 2024-06-06T06:57:49.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:49.257 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78264.asok status 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "metadata": { 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "entity_id": "admin", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": "smithi073", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "pid": "78264", 2024-06-06T06:57:49.394 INFO:teuthology.orchestra.run.smithi073.stdout: "root": "/" 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_count": 0, 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_pinned_count": 0, 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 14700, 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "inst": { 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "name": { 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "client", 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num": 14700 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1672698247 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T06:57:49.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1672698247 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "inst_str": "client.14700 192.168.0.1:0/1672698247", 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "addr_str": "192.168.0.1:0/1672698247", 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "inode_count": 1, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_epoch": 17, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch": 96, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch_barrier": 0, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklisted": false, 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs" 2024-06-06T06:57:49.396 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T06:57:49.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:49.402 DEBUG:teuthology.orchestra.run.smithi073:> stat --file-system '--printf=%T 2024-06-06T06:57:49.402 DEBUG:teuthology.orchestra.run.smithi073:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:49.418 INFO:teuthology.orchestra.run.smithi073.stdout:fuseblk 2024-06-06T06:57:49.419 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:49.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:49.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:49.484 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-06-06T06:57:49.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:49 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-06-06T06:57:49.597 INFO:teuthology.orchestra.run.smithi073.stdout:# file: . 2024-06-06T06:57:49.597 INFO:teuthology.orchestra.run.smithi073.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-06-06T06:57:49.597 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:57:49.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T06:57:49.603 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:57:49.603 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T06:57:49.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T06:57:49.899 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:50.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:50 smithi073 ceph-mon[32303]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:50 smithi136 ceph-mon[35901]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:50 smithi170 ceph-mon[36108]: pgmap v325: 129 pgs: 129 active+clean; 579 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:57:51.651 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-06T06:57:51.651 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.652 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:51 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-06T06:57:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:51 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]: dispatch 2024-06-06T06:57:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-06-06T06:57:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-06T06:57:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-06-06T06:57:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-06-06T06:57:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]: dispatch 2024-06-06T06:57:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-06-06T06:57:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]: dispatch 2024-06-06T06:57:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:51 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]: dispatch 2024-06-06T06:57:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 -- 172.21.15.73:0/3835643978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 msgr2=0x7f7660107620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 --2- 172.21.15.73:0/3835643978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660107620 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f76540099c0 tx=0x7f7654009cd0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 -- 172.21.15.73:0/3835643978 shutdown_connections 2024-06-06T06:57:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 --2- 172.21.15.73:0/3835643978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660105330 0x7f7660105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.878 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 --2- 172.21.15.73:0/3835643978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660107b60 0x7f766010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 --2- 172.21.15.73:0/3835643978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660107620 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 -- 172.21.15.73:0/3835643978 >> 172.21.15.73:0/3835643978 conn(0x7f76600766b0 msgr2=0x7f7660076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:51.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 -- 172.21.15.73:0/3835643978 shutdown_connections 2024-06-06T06:57:51.879 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.877+0000 7f7666630700 1 -- 172.21.15.73:0/3835643978 wait complete. 2024-06-06T06:57:51.880 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 Processor -- start 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 -- start start 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660105330 0x7f7660199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660107b60 0x7f76601a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7660111660 con 0x7f76601071a0 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76601114e0 con 0x7f7660105330 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7666630700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76601117e0 con 0x7f7660107b60 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f765f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49800/0 (socket says 172.21.15.73:49800) 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f765f7fe700 1 -- 172.21.15.73:0/1572911060 learned_addr learned my addr 172.21.15.73:0/1572911060 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f7664bcd700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660107b60 0x7f76601a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:51.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.878+0000 7f765ffff700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660105330 0x7f7660199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:51.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 -- 172.21.15.73:0/1572911060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660107b60 msgr2=0x7f76601a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:51.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660107b60 0x7f76601a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 -- 172.21.15.73:0/1572911060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660105330 msgr2=0x7f7660199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660105330 0x7f7660199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 -- 172.21.15.73:0/1572911060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76601a4fc0 con 0x7f76601071a0 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765f7fe700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654026070 con 0x7f76601071a0 2024-06-06T06:57:51.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7654004ca0 con 0x7f76601071a0 2024-06-06T06:57:51.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76601a5190 con 0x7f76601071a0 2024-06-06T06:57:51.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7654003d20 con 0x7f76601071a0 2024-06-06T06:57:51.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.879+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76601a5640 con 0x7f76601071a0 2024-06-06T06:57:51.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.880+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7660069180 con 0x7f76601071a0 2024-06-06T06:57:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.881+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7654029070 con 0x7f76601071a0 2024-06-06T06:57:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.882+0000 7f765d7fa700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 0x7f76480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.882+0000 7f765ffff700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 0x7f76480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.882+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9092+0+0 (secure 0 0 0) 0x7f76540cfce0 con 0x7f76601071a0 2024-06-06T06:57:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.882+0000 7f765ffff700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 0x7f76480a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7650009a10 tx=0x7f7650009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:51.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:51.885+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7654013070 con 0x7f76601071a0 2024-06-06T06:57:52.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.099+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f766002dca0 con 0x7f76601071a0 2024-06-06T06:57:52.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.099+0000 7f765d7fa700 1 -- 172.21.15.73:0/1572911060 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v97) ==== 81+0+587 (secure 0 0 0) 0x7f76540bae20 con 0x7f76601071a0 2024-06-06T06:57:52.101 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:52.101 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1159052463 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:52.101 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:52.101 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2836597527 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:52.101 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/905820354 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:52.102 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/27064626 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:52.102 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1626604755 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:52.102 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2368928774 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:52.102 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:52.102 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:52.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 msgr2=0x7f76480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:52.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 0x7f76480a5160 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7650009a10 tx=0x7f7650009190 comp rx=0 tx=0).stop 2024-06-06T06:57:52.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 msgr2=0x7f7660199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:52.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7654006010 tx=0x7f7654009f80 comp rx=0 tx=0).stop 2024-06-06T06:57:52.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 shutdown_connections 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f76480a2ca0 0x7f76480a5160 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7660107b60 0x7f76601a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7660105330 0x7f7660199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 --2- 172.21.15.73:0/1572911060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76601071a0 0x7f7660199f40 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 >> 172.21.15.73:0/1572911060 conn(0x7f76600766b0 msgr2=0x7f76600fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 shutdown_connections 2024-06-06T06:57:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:52.101+0000 7f7666630700 1 -- 172.21.15.73:0/1572911060 wait complete. 2024-06-06T06:57:52.106 INFO:teuthology.orchestra.run.smithi073.stderr:listed 10 entries 2024-06-06T06:57:52.712 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-06T06:57:52.712 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-06T06:57:52.712 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-06T06:57:52.712 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T06:57:52.713 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:52 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1572911060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:52.738 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:57:52.738 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T06:57:52.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T06:57:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-06T06:57:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T06:57:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:52 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1572911060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.0", "id": [8, 4]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3a", "id": [4, 8]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [9, 10]}]': finished 2024-06-06T06:57:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [9, 8]}]': finished 2024-06-06T06:57:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1c", "id": [9, 8]}]': finished 2024-06-06T06:57:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: osdmap e97: 12 total, 12 up, 12 in 2024-06-06T06:57:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:52 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1572911060' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:53.036 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:57:53.415 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:53 smithi073 ceph-mon[32303]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T06:57:53.415 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:53 smithi073 ceph-mon[32303]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T06:57:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:53 smithi136 ceph-mon[35901]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T06:57:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:53 smithi136 ceph-mon[35901]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T06:57:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:53 smithi170 ceph-mon[36108]: osdmap e98: 12 total, 12 up, 12 in 2024-06-06T06:57:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:53 smithi170 ceph-mon[36108]: pgmap v329: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s 2024-06-06T06:57:54.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.800+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/2183118237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806b5a0 msgr2=0x7f8a6810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.800+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/2183118237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806b5a0 0x7f8a6810f320 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000b0a0 tx=0x7f8a6000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/2183118237 shutdown_connections 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/2183118237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806b5a0 0x7f8a6810f320 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/2183118237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 0x7f8a681088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/2183118237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806abe0 0x7f8a6806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/2183118237 >> 172.21.15.73:0/2183118237 conn(0x7f8a68078360 msgr2=0x7f8a68078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/2183118237 shutdown_connections 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/2183118237 wait complete. 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 Processor -- start 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- start start 2024-06-06T06:57:54.804 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806b5a0 0x7f8a6819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 0x7f8a6819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112710 con 0x7f8a6806abe0 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112590 con 0x7f8a68108500 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.801+0000 7f8a6f4c5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a68112890 con 0x7f8a6806b5a0 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49812/0 (socket says 172.21.15.73:49812) 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 -- 172.21.15.73:0/3461473039 learned_addr learned my addr 172.21.15.73:0/3461473039 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6da62700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 0x7f8a6819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6ca60700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806b5a0 0x7f8a6819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 -- 172.21.15.73:0/3461473039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806b5a0 msgr2=0x7f8a6819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806b5a0 0x7f8a6819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.805 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 -- 172.21.15.73:0/3461473039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 msgr2=0x7f8a6819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 0x7f8a6819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 -- 172.21.15.73:0/3461473039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a6819fe90 con 0x7f8a6806abe0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6d261700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400b530 tx=0x7f8a6400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64015020 con 0x7f8a6806abe0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a64011ce0 con 0x7f8a6806abe0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a681a0180 con 0x7f8a6806abe0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a64010e40 con 0x7f8a6806abe0 2024-06-06T06:57:54.806 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.802+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a681a7a00 con 0x7f8a6806abe0 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.803+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a68069180 con 0x7f8a6806abe0 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.805+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8a64018020 con 0x7f8a6806abe0 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.806+0000 7f8a5e7fc700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 0x7f8a540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.806+0000 7f8a6ca60700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 0x7f8a540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.806+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f8a640cf180 con 0x7f8a6806abe0 2024-06-06T06:57:54.807 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.806+0000 7f8a6ca60700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 0x7f8a540a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:57:54.810 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:54.809+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a6401f020 con 0x7f8a6806abe0 2024-06-06T06:57:55.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.020+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8a6819c1e0 con 0x7f8a6806abe0 2024-06-06T06:57:55.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.021+0000 7f8a5e7fc700 1 -- 172.21.15.73:0/3461473039 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v98) ==== 81+0+587 (secure 0 0 0) 0x7f8a6819c1e0 con 0x7f8a6806abe0 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1159052463 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2836597527 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/905820354 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/27064626 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1626604755 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2368928774 2024-06-07T06:49:51.216738+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:55.049 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 msgr2=0x7f8a540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 0x7f8a540a50e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8a58006fd0 tx=0x7f8a58008040 comp rx=0 tx=0).stop 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 msgr2=0x7f8a6819ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8a6400b530 tx=0x7f8a6400b840 comp rx=0 tx=0).stop 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 shutdown_connections 2024-06-06T06:57:55.050 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a68108500 0x7f8a6819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a6806abe0 0x7f8a6819ae80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8a540a2c20 0x7f8a540a50e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 --2- 172.21.15.73:0/3461473039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a6806b5a0 0x7f8a6819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 >> 172.21.15.73:0/3461473039 conn(0x7f8a68078360 msgr2=0x7f8a6810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 shutdown_connections 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:57:55.023+0000 7f8a6f4c5700 1 -- 172.21.15.73:0/3461473039 wait complete. 2024-06-06T06:57:55.051 INFO:teuthology.orchestra.run.smithi073.stderr:listed 10 entries 2024-06-06T06:57:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:55 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3461473039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:55 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3461473039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:55 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3461473039' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T06:57:55.672 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi073.front.sepia.ceph.com... 2024-06-06T06:57:55.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:57:55.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-06-06T06:57:55.698 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi073.stderr:ceph-fuse[78264]: fuse finished with error 0 and tester_r 0 2024-06-06T06:57:55.700 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T06:57:56.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:56 smithi170 ceph-mon[36108]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:57:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:56 smithi073 ceph-mon[32303]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:57:56.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:56 smithi136 ceph-mon[35901]: pgmap v330: 129 pgs: 2 peering, 127 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:57:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:57:58 smithi073 ceph-mon[32303]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:57:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:57:58 smithi136 ceph-mon[35901]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:57:58.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:57:58 smithi170 ceph-mon[36108]: pgmap v331: 129 pgs: 1 peering, 128 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:00 smithi073 ceph-mon[32303]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:00 smithi136 ceph-mon[35901]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:00.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:00 smithi170 ceph-mon[36108]: pgmap v332: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:01.802 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T06:58:01.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T06:58:01.802 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-06-06T06:58:01.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:58:02.141 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:02 smithi073 ceph-mon[32303]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:02.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:02 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:58:02.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:02 smithi170 ceph-mon[36108]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:02.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:02 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:58:02.706 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:02 smithi136 ceph-mon[35901]: pgmap v333: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 103 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:02.707 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:02 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:58:03.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.880+0000 7fe0077ec700 1 -- 172.21.15.73:0/929756967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe000107840 msgr2=0x7fe00010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.880+0000 7fe0077ec700 1 --2- 172.21.15.73:0/929756967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe000107840 0x7fe00010f3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdff800b0d0 tx=0x7fdff800b3e0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 -- 172.21.15.73:0/929756967 shutdown_connections 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 --2- 172.21.15.73:0/929756967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe000107840 0x7fe00010f3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 --2- 172.21.15.73:0/929756967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 --2- 172.21.15.73:0/929756967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe0001030f0 0x7fe000107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 -- 172.21.15.73:0/929756967 >> 172.21.15.73:0/929756967 conn(0x7fe0000795f0 msgr2=0x7fe000079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 -- 172.21.15.73:0/929756967 shutdown_connections 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 -- 172.21.15.73:0/929756967 wait complete. 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 Processor -- start 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 -- start start 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:03.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.881+0000 7fe0077ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe000107840 0x7fe000075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112610 con 0x7fe0001030f0 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112490 con 0x7fe000102740 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe000112790 con 0x7fe000107840 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe005588700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe005588700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:41228/0 (socket says 172.21.15.73:41228) 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59774/0 (socket says 172.21.15.73:59774) 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe005588700 1 -- 172.21.15.73:0/2270016184 learned_addr learned my addr 172.21.15.73:0/2270016184 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe005d89700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe000107840 0x7fe000075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 -- 172.21.15.73:0/2270016184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe000107840 msgr2=0x7fe000075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe000107840 0x7fe000075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 -- 172.21.15.73:0/2270016184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 msgr2=0x7fe000074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:03.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:03.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 -- 172.21.15.73:0/2270016184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0001a6f70 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe004d87700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdff000d950 tx=0x7fdff000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0016070 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdff000fd40 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff000e3b0 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0001a7260 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.882+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0001a77a0 con 0x7fe0001030f0 2024-06-06T06:58:03.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.883+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe000069180 con 0x7fe0001030f0 2024-06-06T06:58:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.884+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdff0019070 con 0x7fe0001030f0 2024-06-06T06:58:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.885+0000 7fdff67fc700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 0x7fdfec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.885+0000 7fe005588700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 0x7fdfec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.885+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fdff00cf210 con 0x7fe0001030f0 2024-06-06T06:58:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.886+0000 7fe005588700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 0x7fdfec0a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007c90 tx=0x7fdffc007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:03.890 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:03.889+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdff009be10 con 0x7fe0001030f0 2024-06-06T06:58:04.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.088+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe00002dca0 con 0x7fe0001030f0 2024-06-06T06:58:04.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.090+0000 7fdff67fc700 1 -- 172.21.15.73:0/2270016184 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24628 (secure 0 0 0) 0x7fdff009b710 con 0x7fe0001030f0 2024-06-06T06:58:04.092 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:04.092 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":98,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:52.391218+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 msgr2=0x7fdfec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 0x7fdfec0a50e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fdffc007c90 tx=0x7fdffc007480 comp rx=0 tx=0).stop 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 msgr2=0x7fe000074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdff000d950 tx=0x7fdff000dc60 comp rx=0 tx=0).stop 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 shutdown_connections 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fdfec0a2c20 0x7fdfec0a50e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe000107840 0x7fe000075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe000102740 0x7fe000074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:04.094 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 --2- 172.21.15.73:0/2270016184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe0001030f0 0x7fe000074cd0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 >> 172.21.15.73:0/2270016184 conn(0x7fe0000795f0 msgr2=0x7fe0001005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 shutdown_connections 2024-06-06T06:58:04.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:04.092+0000 7fe0077ec700 1 -- 172.21.15.73:0/2270016184 wait complete. 2024-06-06T06:58:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:04 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2270016184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:04 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2270016184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: pgmap v334: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 93 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:04 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2270016184' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:05.177 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:58:05.460 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:06.598 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:06 smithi073 ceph-mon[32303]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:06 smithi136 ceph-mon[35901]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:06 smithi170 ceph-mon[36108]: pgmap v335: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.189+0000 7f44c3c9f700 1 -- 172.21.15.73:0/307630858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 msgr2=0x7f44bc10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.189+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/307630858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc10f330 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009990 tx=0x7f44b4009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 -- 172.21.15.73:0/307630858 shutdown_connections 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/307630858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc1084f0 0x7f44bc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/307630858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc10f330 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/307630858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc102550 0x7f44bc1029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 -- 172.21.15.73:0/307630858 >> 172.21.15.73:0/307630858 conn(0x7f44bc079650 msgr2=0x7f44bc079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 -- 172.21.15.73:0/307630858 shutdown_connections 2024-06-06T06:58:07.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.190+0000 7f44c3c9f700 1 -- 172.21.15.73:0/307630858 wait complete. 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 Processor -- start 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 -- start start 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 0x7f44bc19ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc1084f0 0x7f44bc19f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112680 con 0x7f44bc102f10 2024-06-06T06:58:07.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112500 con 0x7f44bc1084f0 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c3c9f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44bc112800 con 0x7f44bc102550 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45012/0 (socket says 172.21.15.73:45012) 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 -- 172.21.15.73:0/1665967610 learned_addr learned my addr 172.21.15.73:0/1665967610 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c1a3b700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 0x7f44bc19ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c223c700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc1084f0 0x7f44bc19f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 -- 172.21.15.73:0/1665967610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 msgr2=0x7f44bc19ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 0x7f44bc19ad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 -- 172.21.15.73:0/1665967610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc1084f0 msgr2=0x7f44bc19f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.191+0000 7f44c123a700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc1084f0 0x7f44bc19f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44c123a700 1 -- 172.21.15.73:0/1665967610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44bc19fd50 con 0x7f44bc102f10 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44c1a3b700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 0x7f44bc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:58:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44c123a700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00d950 tx=0x7f44ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac016070 con 0x7f44bc102f10 2024-06-06T06:58:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44ac010460 con 0x7f44bc102f10 2024-06-06T06:58:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac00f620 con 0x7f44bc102f10 2024-06-06T06:58:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44bc1a0040 con 0x7f44bc102f10 2024-06-06T06:58:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.192+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44bc1a78c0 con 0x7f44bc102f10 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.193+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44bc069180 con 0x7f44bc102f10 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.194+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f44ac019070 con 0x7f44bc102f10 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.195+0000 7f44b2ffd700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 0x7f44a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.195+0000 7f44c1a3b700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 0x7f44a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.195+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f44ac0cefa0 con 0x7f44bc102f10 2024-06-06T06:58:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.195+0000 7f44c1a3b700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 0x7f44a80a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009fa0 tx=0x7f44b8009570 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.199+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f44ac09bba0 con 0x7f44bc102f10 2024-06-06T06:58:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.408+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44bc1a7a90 con 0x7f44bc102f10 2024-06-06T06:58:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.409+0000 7f44b2ffd700 1 -- 172.21.15.73:0/1665967610 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24628 (secure 0 0 0) 0x7f44ac09b4a0 con 0x7f44bc102f10 2024-06-06T06:58:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:07.411 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":98,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:52.391218+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 msgr2=0x7f44a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 0x7f44a80a5160 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009fa0 tx=0x7f44b8009570 comp rx=0 tx=0).stop 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 msgr2=0x7f44bc19b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00d950 tx=0x7f44ac00dd10 comp rx=0 tx=0).stop 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 shutdown_connections 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44a80a2ca0 0x7f44a80a5160 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f44bc102550 0x7f44bc19ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f44bc1084f0 0x7f44bc19f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 --2- 172.21.15.73:0/1665967610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44bc102f10 0x7f44bc19b280 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 >> 172.21.15.73:0/1665967610 conn(0x7f44bc079650 msgr2=0x7f44bc10d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 shutdown_connections 2024-06-06T06:58:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:07.412+0000 7f44c3c9f700 1 -- 172.21.15.73:0/1665967610 wait complete. 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:58:07.818 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1665967610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:07.819 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:07 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1665967610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:08.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:07 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: pgmap v336: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1665967610' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:08.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:07 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:58:08.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-06T06:58:08.410 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:10.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 -- 172.21.15.73:0/1054520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101380 msgr2=0x7f0a50101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:10.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 --2- 172.21.15.73:0/1054520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101380 0x7f0a50101800 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0a38009990 tx=0x7f0a38009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 -- 172.21.15.73:0/1054520788 shutdown_connections 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 --2- 172.21.15.73:0/1054520788 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a501073f0 0x7f0a501077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 --2- 172.21.15.73:0/1054520788 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a50101d40 0x7f0a5010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 --2- 172.21.15.73:0/1054520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101380 0x7f0a50101800 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 -- 172.21.15.73:0/1054520788 >> 172.21.15.73:0/1054520788 conn(0x7f0a50078300 msgr2=0x7f0a50078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 -- 172.21.15.73:0/1054520788 shutdown_connections 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.240+0000 7f0a56635700 1 -- 172.21.15.73:0/1054520788 wait complete. 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.241+0000 7f0a56635700 1 Processor -- start 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.241+0000 7f0a56635700 1 -- start start 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.241+0000 7f0a56635700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a50101380 0x7f0a5019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.241+0000 7f0a56635700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.241+0000 7f0a56635700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a501073f0 0x7f0a50197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:10.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a56635700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50111530 con 0x7f0a50101d40 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a56635700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501113b0 con 0x7f0a501073f0 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a56635700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0a501116b0 con 0x7f0a50101380 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45020/0 (socket says 172.21.15.73:45020) 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 -- 172.21.15.73:0/3626651902 learned_addr learned my addr 172.21.15.73:0/3626651902 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a4ffff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a50101380 0x7f0a5019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a54bd2700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a501073f0 0x7f0a50197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 -- 172.21.15.73:0/3626651902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a50101380 msgr2=0x7f0a5019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a50101380 0x7f0a5019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 -- 172.21.15.73:0/3626651902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a501073f0 msgr2=0x7f0a50197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a501073f0 0x7f0a50197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 -- 172.21.15.73:0/3626651902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a50197ff0 con 0x7f0a50101d40 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a47fff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a38003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38027070 con 0x7f0a50101d40 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a3800bdc0 con 0x7f0a50101d40 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a50198280 con 0x7f0a50101d40 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a38022cd0 con 0x7f0a50101d40 2024-06-06T06:58:10.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.242+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a5006aa00 con 0x7f0a50101d40 2024-06-06T06:58:10.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.243+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a50102aa0 con 0x7f0a50101d40 2024-06-06T06:58:10.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.245+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f0a3802a070 con 0x7f0a50101d40 2024-06-06T06:58:10.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.246+0000 7f0a4dffb700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 0x7f0a3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:10.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.246+0000 7f0a4ffff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 0x7f0a3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:10.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.246+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f0a380d0420 con 0x7f0a50101d40 2024-06-06T06:58:10.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.246+0000 7f0a4ffff700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 0x7f0a3c0a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a400045b0 tx=0x7f0a4000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:10.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.249+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0a38013070 con 0x7f0a50101d40 2024-06-06T06:58:10.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:10.489+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f0a50198d40 con 0x7f0a50101d40 2024-06-06T06:58:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:10 smithi073 ceph-mon[32303]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:10.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:10 smithi136 ceph-mon[35901]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:10.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:10 smithi170 ceph-mon[36108]: pgmap v337: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:11.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.369+0000 7f0a4dffb700 1 -- 172.21.15.73:0/3626651902 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f0a3809d0b0 con 0x7f0a50101d40 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 msgr2=0x7f0a3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 0x7f0a3c0a50e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f0a400045b0 tx=0x7f0a4000b3c0 comp rx=0 tx=0).stop 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 msgr2=0x7f0a5019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f0a38000c00 tx=0x7f0a38003a80 comp rx=0 tx=0).stop 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 shutdown_connections 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0a3c0a2c20 0x7f0a3c0a50e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0a50101380 0x7f0a5019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0a501073f0 0x7f0a50197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 --2- 172.21.15.73:0/3626651902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a50101d40 0x7f0a5019dbd0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 >> 172.21.15.73:0/3626651902 conn(0x7f0a50078300 msgr2=0x7f0a500ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 shutdown_connections 2024-06-06T06:58:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:11.372+0000 7f0a56635700 1 -- 172.21.15.73:0/3626651902 wait complete. 2024-06-06T06:58:11.749 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:11 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T06:58:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:11 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T06:58:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:11 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-06T06:58:12.092 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-06-06T06:58:12.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-06T06:58:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:12 smithi073 ceph-mon[32303]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:12 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T06:58:12.625 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:12 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:12 smithi136 ceph-mon[35901]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:12 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T06:58:12.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:12 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:12 smithi170 ceph-mon[36108]: pgmap v338: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-06-06T06:58:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:12 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3626651902' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-06T06:58:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:12 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 -- 172.21.15.73:0/2692779963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006abe0 msgr2=0x7fd3b006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 --2- 172.21.15.73:0/2692779963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006abe0 0x7fd3b006b060 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009990 tx=0x7fd3a4009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 -- 172.21.15.73:0/2692779963 shutdown_connections 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 --2- 172.21.15.73:0/2692779963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b0108520 0x7fd3b0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 --2- 172.21.15.73:0/2692779963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b006b5a0 0x7fd3b010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 --2- 172.21.15.73:0/2692779963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006abe0 0x7fd3b006b060 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 -- 172.21.15.73:0/2692779963 >> 172.21.15.73:0/2692779963 conn(0x7fd3b0078330 msgr2=0x7fd3b0078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:14.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 -- 172.21.15.73:0/2692779963 shutdown_connections 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.290+0000 7fd3b4993700 1 -- 172.21.15.73:0/2692779963 wait complete. 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 Processor -- start 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 -- start start 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b006abe0 0x7fd3b0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 0x7fd3b0101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b01126c0 con 0x7fd3b006b5a0 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0112540 con 0x7fd3b0108520 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3b4993700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd3b0112840 con 0x7fd3b006abe0 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45042/0 (socket says 172.21.15.73:45042) 2024-06-06T06:58:14.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3add9b700 1 -- 172.21.15.73:0/3120851008 learned_addr learned my addr 172.21.15.73:0/3120851008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.291+0000 7fd3ae59c700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b006abe0 0x7fd3b0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3aed9d700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 0x7fd3b0101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 -- 172.21.15.73:0/3120851008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b006abe0 msgr2=0x7fd3b0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b006abe0 0x7fd3b0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 -- 172.21.15.73:0/3120851008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 msgr2=0x7fd3b0101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 0x7fd3b0101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 -- 172.21.15.73:0/3120851008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3b01a7130 con 0x7fd3b006b5a0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3aed9d700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 0x7fd3b0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3add9b700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4004830 con 0x7fd3b006b5a0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3a4005020 con 0x7fd3b006b5a0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3b01a73c0 con 0x7fd3b006b5a0 2024-06-06T06:58:14.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3a4003e50 con 0x7fd3b006b5a0 2024-06-06T06:58:14.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.292+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3b01a77b0 con 0x7fd3b006b5a0 2024-06-06T06:58:14.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.293+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3b0103bc0 con 0x7fd3b006b5a0 2024-06-06T06:58:14.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.294+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3a4005540 con 0x7fd3b006b5a0 2024-06-06T06:58:14.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.295+0000 7fd39f7fe700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 0x7fd3980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.295+0000 7fd3ae59c700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 0x7fd3980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.295+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fd3a40d0210 con 0x7fd3b006b5a0 2024-06-06T06:58:14.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.295+0000 7fd3ae59c700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 0x7fd3980a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0003f90 tx=0x7fd3a0009180 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:14.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.299+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3a4013070 con 0x7fd3b006b5a0 2024-06-06T06:58:14.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:14.546+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7fd3b002dca0 con 0x7fd3b006b5a0 2024-06-06T06:58:14.547 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:14 smithi073 ceph-mon[32303]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:14.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:14 smithi136 ceph-mon[35901]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:14.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:14 smithi170 ceph-mon[36108]: pgmap v339: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:15.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.383+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3120851008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7fd3a409cfc0 con 0x7fd3b006b5a0 2024-06-06T06:58:15.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 msgr2=0x7fd3980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:15.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 0x7fd3980a50e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd3a0003f90 tx=0x7fd3a0009180 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 msgr2=0x7fd3b01019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4000c00 tx=0x7fd3a40047b0 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 shutdown_connections 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3980a2c20 0x7fd3980a50e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd3b006abe0 0x7fd3b0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd3b0108520 0x7fd3b0101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 --2- 172.21.15.73:0/3120851008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3b006b5a0 0x7fd3b01019d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.386+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 >> 172.21.15.73:0/3120851008 conn(0x7fd3b0078330 msgr2=0x7fd3b01000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.387+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 shutdown_connections 2024-06-06T06:58:15.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:15.387+0000 7fd3b4993700 1 -- 172.21.15.73:0/3120851008 wait complete. 2024-06-06T06:58:15.723 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:15 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T06:58:15.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:15 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T06:58:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:15 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-06T06:58:16.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-06T06:58:16.329 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:16.735 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-06T06:58:16.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:16 smithi073 ceph-mon[32303]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-06T06:58:16.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:16 smithi136 ceph-mon[35901]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-06T06:58:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: pgmap v340: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T06:58:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-06T06:58:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3120851008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-06T06:58:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: daemon mds.c assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-06T06:58:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-06-06T06:58:16.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: fsmap cephfs:2 {0=i=up:active,1=c=up:creating} 10 up:standby 2024-06-06T06:58:16.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:16 smithi170 ceph-mon[36108]: daemon mds.c is now active in filesystem cephfs as rank 1 2024-06-06T06:58:17.677 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:17 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T06:58:17.677 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:17 smithi073 ceph-mon[32303]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T06:58:17.678 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:17 smithi073 ceph-mon[32303]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-06T06:58:17.678 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:17 smithi073 ceph-mon[32303]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-06T06:58:17.678 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:17 smithi073 ceph-mon[32303]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-06T06:58:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:17 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T06:58:17.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:17 smithi136 ceph-mon[35901]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T06:58:17.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:17 smithi136 ceph-mon[35901]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-06T06:58:17.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:17 smithi136 ceph-mon[35901]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-06T06:58:17.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:17 smithi136 ceph-mon[35901]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-06T06:58:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:17 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T06:58:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:17 smithi170 ceph-mon[36108]: daemon mds.b assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-06T06:58:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:17 smithi170 ceph-mon[36108]: fsmap cephfs:2 {0=i=up:active,1=c=up:active} 10 up:standby 2024-06-06T06:58:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:17 smithi170 ceph-mon[36108]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:creating} 9 up:standby 2024-06-06T06:58:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:17 smithi170 ceph-mon[36108]: daemon mds.b is now active in filesystem cephfs as rank 2 2024-06-06T06:58:18.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 -- 172.21.15.73:0/1385480754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8107400 msgr2=0x7f94e81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 --2- 172.21.15.73:0/1385480754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8107400 0x7f94e81077e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009990 tx=0x7f94d0009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 -- 172.21.15.73:0/1385480754 shutdown_connections 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 --2- 172.21.15.73:0/1385480754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101d50 0x7f94e810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 --2- 172.21.15.73:0/1385480754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8107400 0x7f94e81077e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 --2- 172.21.15.73:0/1385480754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8101390 0x7f94e8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 -- 172.21.15.73:0/1385480754 >> 172.21.15.73:0/1385480754 conn(0x7f94e8078340 msgr2=0x7f94e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 -- 172.21.15.73:0/1385480754 shutdown_connections 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.072+0000 7f94edbd7700 1 -- 172.21.15.73:0/1385480754 wait complete. 2024-06-06T06:58:18.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 Processor -- start 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 -- start start 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101390 0x7f94e819d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8107400 0x7f94e81978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f94e8111570 con 0x7f94e8101d50 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81113f0 con 0x7f94e8101390 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94edbd7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81116f0 con 0x7f94e8107400 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.073+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59806/0 (socket says 172.21.15.73:59806) 2024-06-06T06:58:18.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 -- 172.21.15.73:0/2597443808 learned_addr learned my addr 172.21.15.73:0/2597443808 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e77fe700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101390 0x7f94e819d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e7fff700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8107400 0x7f94e81978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 -- 172.21.15.73:0/2597443808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8107400 msgr2=0x7f94e81978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8107400 0x7f94e81978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 -- 172.21.15.73:0/2597443808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101390 msgr2=0x7f94e819d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101390 0x7f94e819d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 -- 172.21.15.73:0/2597443808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e8198010 con 0x7f94e8101d50 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e6ffd700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94d800d980 tx=0x7f94d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8017070 con 0x7f94e8101d50 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94d8005020 con 0x7f94e8101d50 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e8198300 con 0x7f94e8101d50 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d800fa90 con 0x7f94e8101d50 2024-06-06T06:58:18.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.074+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a45c0 con 0x7f94e8101d50 2024-06-06T06:58:18.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.075+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e8069180 con 0x7f94e8101d50 2024-06-06T06:58:18.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.077+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f94d80051d0 con 0x7f94e8101d50 2024-06-06T06:58:18.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.078+0000 7f94e4ff9700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 0x7f94d40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:18.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.078+0000 7f94e77fe700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 0x7f94d40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:18.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.078+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f94d80cf680 con 0x7f94e8101d50 2024-06-06T06:58:18.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.078+0000 7f94e77fe700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 0x7f94d40a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006010 tx=0x7f94d000b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:18.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.081+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f94d8020020 con 0x7f94e8101d50 2024-06-06T06:58:18.323 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.323+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f94e802dca0 con 0x7f94e8101d50 2024-06-06T06:58:18.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.404+0000 7f94e4ff9700 1 -- 172.21.15.73:0/2597443808 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f94d809c220 con 0x7f94e8101d50 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.406+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 msgr2=0x7f94d40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.406+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 0x7f94d40a50e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94d0006010 tx=0x7f94d000b560 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.406+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 msgr2=0x7f94e819dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.406+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f94d800d980 tx=0x7f94d800dc90 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 shutdown_connections 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f94d40a2c20 0x7f94d40a50e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f94e8107400 0x7f94e81978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f94e8101390 0x7f94e819d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 --2- 172.21.15.73:0/2597443808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94e8101d50 0x7f94e819dc30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 >> 172.21.15.73:0/2597443808 conn(0x7f94e8078340 msgr2=0x7f94e81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:18.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 shutdown_connections 2024-06-06T06:58:18.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:18.407+0000 7f94edbd7700 1 -- 172.21.15.73:0/2597443808 wait complete. 2024-06-06T06:58:18.747 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:active 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-06-06T06:58:18.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:18 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T06:58:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:active 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-06-06T06:58:18.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:18 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T06:58:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: pgmap v341: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] up:active 2024-06-06T06:58:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: daemon mds.f assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-06T06:58:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: fsmap cephfs:3 {0=i=up:active,1=c=up:active,2=b=up:active} 9 up:standby 2024-06-06T06:58:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:creating} 8 up:standby 2024-06-06T06:58:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: daemon mds.f is now active in filesystem cephfs as rank 3 2024-06-06T06:58:18.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:18 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-06T06:58:19.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-06T06:58:19.308 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:19.735 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:active 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: Cluster is now healthy 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-06T06:58:19.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:19 smithi073 ceph-mon[32303]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:active 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: Cluster is now healthy 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-06T06:58:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:19 smithi136 ceph-mon[35901]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:19.847 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T06:58:19.359+0000 7f514b1c7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-06-06T06:58:19.847 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T06:58:19.359+0000 7f514b1c7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-06T06:58:19.847 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T06:58:19.359+0000 7f514b1c7700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-06-06T06:58:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2597443808' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] up:active 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: daemon mds.h assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: Cluster is now healthy 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: fsmap cephfs:4 {0=i=up:active,1=c=up:active,2=b=up:active,3=f=up:active} 8 up:standby 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: daemon mds.h is now active in filesystem cephfs as rank 4 2024-06-06T06:58:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:19 smithi170 ceph-mon[36108]: pgmap v342: 129 pgs: 129 active+clean; 581 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s wr, 3 op/s 2024-06-06T06:58:20.515 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:20 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:active 2024-06-06T06:58:20.515 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:20 smithi073 ceph-mon[32303]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T06:58:20.516 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:20 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:20 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:active 2024-06-06T06:58:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:20 smithi136 ceph-mon[35901]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T06:58:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:20 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:20 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] up:active 2024-06-06T06:58:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:20 smithi170 ceph-mon[36108]: fsmap cephfs:5 7 up:standby 5 up:active 2024-06-06T06:58:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:20 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:21.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 -- 172.21.15.73:0/146041019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba41074e0 msgr2=0x7f2ba41078c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:21.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 --2- 172.21.15.73:0/146041019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba41074e0 0x7f2ba41078c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2b94009990 tx=0x7f2b94009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 -- 172.21.15.73:0/146041019 shutdown_connections 2024-06-06T06:58:21.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 --2- 172.21.15.73:0/146041019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba4102030 0x7f2ba410e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 --2- 172.21.15.73:0/146041019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba41074e0 0x7f2ba41078c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 --2- 172.21.15.73:0/146041019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 0x7f2ba4101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 -- 172.21.15.73:0/146041019 >> 172.21.15.73:0/146041019 conn(0x7f2ba4078280 msgr2=0x7f2ba4078690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:21.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 -- 172.21.15.73:0/146041019 shutdown_connections 2024-06-06T06:58:21.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.108+0000 7f2baa590700 1 -- 172.21.15.73:0/146041019 wait complete. 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 Processor -- start 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 -- start start 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 0x7f2ba4199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba41074e0 0x7f2ba41a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba41115e0 con 0x7f2ba4102030 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4111460 con 0x7f2ba4101670 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.109+0000 7f2baa590700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba4111760 con 0x7f2ba41074e0 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59820/0 (socket says 172.21.15.73:59820) 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 -- 172.21.15.73:0/1521966499 learned_addr learned my addr 172.21.15.73:0/1521966499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:21.112 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2ba8b2d700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba41074e0 0x7f2ba41a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2ba3fff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 0x7f2ba4199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 -- 172.21.15.73:0/1521966499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba41074e0 msgr2=0x7f2ba41a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba41074e0 0x7f2ba41a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 -- 172.21.15.73:0/1521966499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 msgr2=0x7f2ba4199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 0x7f2ba4199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 -- 172.21.15.73:0/1521966499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba41a4f20 con 0x7f2ba4102030 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2b9bfff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00d980 tx=0x7f2b8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c016070 con 0x7f2ba4102030 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba41a51b0 con 0x7f2ba4102030 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b8c004830 con 0x7f2ba4102030 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b8c005530 con 0x7f2ba4102030 2024-06-06T06:58:21.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.110+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba41a5710 con 0x7f2ba4102030 2024-06-06T06:58:21.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.111+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba41098c0 con 0x7f2ba4102030 2024-06-06T06:58:21.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.112+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f2b8c0049e0 con 0x7f2ba4102030 2024-06-06T06:58:21.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.113+0000 7f2ba1ffb700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 0x7f2b900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:21.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.113+0000 7f2ba3fff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 0x7f2b900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:21.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.113+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f2b8c0cf0b0 con 0x7f2ba4102030 2024-06-06T06:58:21.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.113+0000 7f2ba3fff700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 0x7f2b900a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400b5c0 tx=0x7f2b94005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:21.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.118+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b8c01f020 con 0x7f2ba4102030 2024-06-06T06:58:21.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.358+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f2ba4102b00 con 0x7f2ba4102030 2024-06-06T06:58:21.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.420+0000 7f2ba1ffb700 1 -- 172.21.15.73:0/1521966499 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7f2b8c09bcb0 con 0x7f2ba4102030 2024-06-06T06:58:21.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 msgr2=0x7f2b900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 0x7f2b900a50e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f2b9400b5c0 tx=0x7f2b94005960 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 msgr2=0x7f2ba4199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c00d980 tx=0x7f2b8c00dc90 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 shutdown_connections 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2b900a2c20 0x7f2b900a50e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ba41074e0 0x7f2ba41a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ba4101670 0x7f2ba4199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 --2- 172.21.15.73:0/1521966499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba4102030 0x7f2ba4199ea0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.422+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 >> 172.21.15.73:0/1521966499 conn(0x7f2ba4078280 msgr2=0x7f2ba4100300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.423+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 shutdown_connections 2024-06-06T06:58:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:21.423+0000 7f2baa590700 1 -- 172.21.15.73:0/1521966499 wait complete. 2024-06-06T06:58:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:21 smithi073 ceph-mon[32303]: pgmap v343: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-06-06T06:58:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T06:58:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:21 smithi136 ceph-mon[35901]: pgmap v343: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-06-06T06:58:21.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T06:58:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:21 smithi170 ceph-mon[36108]: pgmap v343: 129 pgs: 129 active+clean; 585 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-06-06T06:58:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-06T06:58:22.130 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-06-06T06:58:22.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-06T06:58:22.373 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:22 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T06:58:22.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:22 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:22 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T06:58:22.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:22 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:22 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1521966499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-06T06:58:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:22 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T06:58:23.722 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:23 smithi073 ceph-mon[32303]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-06T06:58:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:23 smithi136 ceph-mon[35901]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-06T06:58:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:23 smithi170 ceph-mon[36108]: pgmap v344: 129 pgs: 129 active+clean; 587 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 4.0 KiB/s wr, 17 op/s 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.134+0000 7feba505b700 1 -- 172.21.15.73:0/163981019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0103120 msgr2=0x7feba0107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 --2- 172.21.15.73:0/163981019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0103120 0x7feba0107130 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7feb940099c0 tx=0x7feb94009cd0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 -- 172.21.15.73:0/163981019 shutdown_connections 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 --2- 172.21.15.73:0/163981019 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0102770 0x7feba0102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 --2- 172.21.15.73:0/163981019 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0107860 0x7feba010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 --2- 172.21.15.73:0/163981019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0103120 0x7feba0107130 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 -- 172.21.15.73:0/163981019 >> 172.21.15.73:0/163981019 conn(0x7feba0079600 msgr2=0x7feba0079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 -- 172.21.15.73:0/163981019 shutdown_connections 2024-06-06T06:58:24.136 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 -- 172.21.15.73:0/163981019 wait complete. 2024-06-06T06:58:24.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.135+0000 7feba505b700 1 Processor -- start 2024-06-06T06:58:24.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 -- start start 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102770 0x7feba019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0103120 0x7feba019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feba01126a0 con 0x7feba0107860 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7feba0112520 con 0x7feba0102770 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feba505b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7feba0112820 con 0x7feba0103120 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59832/0 (socket says 172.21.15.73:59832) 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 -- 172.21.15.73:0/1754121009 learned_addr learned my addr 172.21.15.73:0/1754121009 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9ed9d700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102770 0x7feba019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9e59c700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0103120 0x7feba019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 -- 172.21.15.73:0/1754121009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0103120 msgr2=0x7feba019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0103120 0x7feba019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 -- 172.21.15.73:0/1754121009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102770 msgr2=0x7feba019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102770 0x7feba019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.136+0000 7feb9f59e700 1 -- 172.21.15.73:0/1754121009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba0199160 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feb9f59e700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb9800b970 tx=0x7feb9800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb9800d860 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba0199450 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb9800e430 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98017660 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.137+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba01a56c0 con 0x7feba0107860 2024-06-06T06:58:24.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.138+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba0069180 con 0x7feba0107860 2024-06-06T06:58:24.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.139+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7feb9800e5e0 con 0x7feba0107860 2024-06-06T06:58:24.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.140+0000 7feb87fff700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 0x7feb880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:24.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.140+0000 7feb9ed9d700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 0x7feb880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:24.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.140+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7feb980cf330 con 0x7feba0107860 2024-06-06T06:58:24.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.140+0000 7feb9ed9d700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 0x7feb880a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feba0107310 tx=0x7feb9000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:24.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.143+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feb98013030 con 0x7feba0107860 2024-06-06T06:58:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.373+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7feba00628d0 con 0x7feb880a2c20 2024-06-06T06:58:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.402+0000 7feb87fff700 1 -- 172.21.15.73:0/1754121009 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7feba00628d0 con 0x7feb880a2c20 2024-06-06T06:58:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.404+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 msgr2=0x7feb880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.404+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 0x7feb880a50e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7feba0107310 tx=0x7feb9000b410 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.404+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 msgr2=0x7feba0198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.404+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7feb9800b970 tx=0x7feb9800bc80 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 shutdown_connections 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feba0107860 0x7feba0198a00 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7feba0102770 0x7feba019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7feb880a2c20 0x7feb880a50e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 --2- 172.21.15.73:0/1754121009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7feba0103120 0x7feba019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 >> 172.21.15.73:0/1754121009 conn(0x7feba0079600 msgr2=0x7feba0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 shutdown_connections 2024-06-06T06:58:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:24.405+0000 7feba505b700 1 -- 172.21.15.73:0/1754121009 wait complete. 2024-06-06T06:58:24.706 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-mon[32303]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:24.707 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:58:24.378+0000 7fab8e2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T06:58:24.707 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:58:24.378+0000 7fab8e2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T06:58:24.707 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:58:24.378+0000 7fab8e2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T06:58:24.707 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:58:24.378+0000 7fab8e2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T06:58:24.707 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 06:58:24 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T06:58:24.378+0000 7fab8e2fb700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T06:58:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:24 smithi136 ceph-mon[35901]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:24.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:24 smithi170 ceph-mon[36108]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:25.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-06T06:58:25.283 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:25.677 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:25 smithi073 ceph-mon[32303]: pgmap v345: 129 pgs: 129 active+clean; 593 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-06-06T06:58:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:25 smithi136 ceph-mon[35901]: pgmap v345: 129 pgs: 129 active+clean; 593 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-06-06T06:58:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:25 smithi170 ceph-mon[36108]: pgmap v345: 129 pgs: 129 active+clean; 593 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 4.7 KiB/s wr, 20 op/s 2024-06-06T06:58:26.688 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:26 smithi073 ceph-mon[32303]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T06:58:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:26 smithi136 ceph-mon[35901]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T06:58:26.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:26 smithi170 ceph-mon[36108]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-06-06T06:58:26.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/2266347238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f41024e0 msgr2=0x7fb3f4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:26.940 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/2266347238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f41024e0 0x7fb3f4102940 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8009990 tx=0x7fb3e8009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/2266347238 shutdown_connections 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/2266347238 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 0x7fb3f41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/2266347238 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f4102e80 0x7fb3f410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/2266347238 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f41024e0 0x7fb3f4102940 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/2266347238 >> 172.21.15.73:0/2266347238 conn(0x7fb3f4079530 msgr2=0x7fb3f4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/2266347238 shutdown_connections 2024-06-06T06:58:26.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.939+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/2266347238 wait complete. 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 Processor -- start 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 -- start start 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f41024e0 0x7fb3f419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 0x7fb3f419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112600 con 0x7fb3f4102e80 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112480 con 0x7fb3f41024e0 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.940+0000 7fb3f8a1c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f4112780 con 0x7fb3f41084f0 2024-06-06T06:58:26.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57534/0 (socket says 172.21.15.73:57534) 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 -- 172.21.15.73:0/3143063528 learned_addr learned my addr 172.21.15.73:0/3143063528 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f2d9d700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 0x7fb3f419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f259c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f41024e0 0x7fb3f419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 -- 172.21.15.73:0/3143063528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 msgr2=0x7fb3f419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 0x7fb3f419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 -- 172.21.15.73:0/3143063528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f41024e0 msgr2=0x7fb3f419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f41024e0 0x7fb3f419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 -- 172.21.15.73:0/3143063528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f419fd40 con 0x7fb3f4102e80 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f1d9b700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8000c00 tx=0x7fb3e8003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8027070 con 0x7fb3f4102e80 2024-06-06T06:58:26.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f419ffd0 con 0x7fb3f4102e80 2024-06-06T06:58:26.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb3e800bdc0 con 0x7fb3f4102e80 2024-06-06T06:58:26.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.941+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f41a78b0 con 0x7fb3f4102e80 2024-06-06T06:58:26.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.942+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8022cd0 con 0x7fb3f4102e80 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.943+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d4005330 con 0x7fb3f4102e80 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.943+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb3e802a070 con 0x7fb3f4102e80 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.944+0000 7fb3e37fe700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 0x7fb3dc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.944+0000 7fb3f259c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 0x7fb3dc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.944+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fb3e80d0030 con 0x7fb3f4102e80 2024-06-06T06:58:26.946 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.945+0000 7fb3f259c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 0x7fb3dc0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40045b0 tx=0x7fb3e400b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:26.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:26.949+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb3e8013070 con 0x7fb3f4102e80 2024-06-06T06:58:27.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.175+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb3d4000bc0 con 0x7fb3dc0a2d30 2024-06-06T06:58:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.201+0000 7fb3e37fe700 1 -- 172.21.15.73:0/3143063528 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb3d4000bc0 con 0x7fb3dc0a2d30 2024-06-06T06:58:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 msgr2=0x7fb3dc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 0x7fb3dc0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb3e40045b0 tx=0x7fb3e400b3c0 comp rx=0 tx=0).stop 2024-06-06T06:58:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 msgr2=0x7fb3f419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8000c00 tx=0x7fb3e8003a80 comp rx=0 tx=0).stop 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 shutdown_connections 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb3dc0a2d30 0x7fb3dc0a51f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb3f41084f0 0x7fb3f419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3f41024e0 0x7fb3f419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 --2- 172.21.15.73:0/3143063528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb3f4102e80 0x7fb3f419b270 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 >> 172.21.15.73:0/3143063528 conn(0x7fb3f4079530 msgr2=0x7fb3f4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 shutdown_connections 2024-06-06T06:58:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:27.203+0000 7fb3f8a1c700 1 -- 172.21.15.73:0/3143063528 wait complete. 2024-06-06T06:58:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:27 smithi073 ceph-mon[32303]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:27.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:27 smithi073 ceph-mon[32303]: pgmap v346: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-06-06T06:58:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:27 smithi136 ceph-mon[35901]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:27 smithi136 ceph-mon[35901]: pgmap v346: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-06-06T06:58:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:27 smithi170 ceph-mon[36108]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:27 smithi170 ceph-mon[36108]: pgmap v346: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.9 KiB/s wr, 22 op/s 2024-06-06T06:58:27.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-06T06:58:28.141 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 -- 172.21.15.73:0/3829475050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 msgr2=0x7f0b2010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 --2- 172.21.15.73:0/3829475050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2010e240 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800b0a0 tx=0x7f0b1800b3b0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 -- 172.21.15.73:0/3829475050 shutdown_connections 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 --2- 172.21.15.73:0/3829475050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2010e240 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 --2- 172.21.15.73:0/3829475050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 --2- 172.21.15.73:0/3829475050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 0x7f0b20101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 -- 172.21.15.73:0/3829475050 >> 172.21.15.73:0/3829475050 conn(0x7f0b20078340 msgr2=0x7f0b20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 -- 172.21.15.73:0/3829475050 shutdown_connections 2024-06-06T06:58:30.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.998+0000 7f0b2495b700 1 -- 172.21.15.73:0/3829475050 wait complete. 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 Processor -- start 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 -- start start 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 0x7f0b2019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b2019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b20111620 con 0x7f0b20101d50 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201114a0 con 0x7f0b20107400 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b2495b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0b201117a0 con 0x7f0b20101390 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57554/0 (socket says 172.21.15.73:57554) 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 -- 172.21.15.73:0/1988198583 learned_addr learned my addr 172.21.15.73:0/1988198583 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:30.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1e59c700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 0x7f0b2019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1ed9d700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b2019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 -- 172.21.15.73:0/1988198583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 msgr2=0x7f0b2019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 0x7f0b2019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 -- 172.21.15.73:0/1988198583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 msgr2=0x7f0b2019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b2019f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 -- 172.21.15.73:0/1988198583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b2019fe90 con 0x7f0b20101d50 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1ed9d700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b2019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:29.999+0000 7f0b1dd9b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b530 tx=0x7f0b1400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.000+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14015020 con 0x7f0b20101d50 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.000+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b14011ce0 con 0x7f0b20101d50 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.000+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b201a0180 con 0x7f0b20101d50 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.000+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14010d60 con 0x7f0b20101d50 2024-06-06T06:58:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.000+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b201a7a00 con 0x7f0b20101d50 2024-06-06T06:58:30.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.001+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b20069180 con 0x7f0b20101d50 2024-06-06T06:58:30.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.002+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0b14018020 con 0x7f0b20101d50 2024-06-06T06:58:30.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.003+0000 7f0b0f7fe700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 0x7f0b080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:30.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.003+0000 7f0b1e59c700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 0x7f0b080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:30.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.003+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f0b140cec40 con 0x7f0b20101d50 2024-06-06T06:58:30.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.003+0000 7f0b1e59c700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 0x7f0b080a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b10006fd0 tx=0x7f0b10008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.007+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b140d5080 con 0x7f0b20101d50 2024-06-06T06:58:30.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.239+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f0b2006a8e0 con 0x7f0b080a2d30 2024-06-06T06:58:30.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.244+0000 7f0b0f7fe700 1 -- 172.21.15.73:0/1988198583 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f0b2006a8e0 con 0x7f0b080a2d30 2024-06-06T06:58:30.245 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 msgr2=0x7f0b080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 0x7f0b080a51f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0b10006fd0 tx=0x7f0b10008040 comp rx=0 tx=0).stop 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 msgr2=0x7f0b2019b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400b530 tx=0x7f0b1400b8f0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 shutdown_connections 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f0b080a2d30 0x7f0b080a51f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0b20101390 0x7f0b2019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0b20107400 0x7f0b2019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.248 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 --2- 172.21.15.73:0/1988198583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b20101d50 0x7f0b2019b3c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:30.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 >> 172.21.15.73:0/1988198583 conn(0x7f0b20078340 msgr2=0x7f0b2006a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:30.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 shutdown_connections 2024-06-06T06:58:30.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:30.247+0000 7f0b2495b700 1 -- 172.21.15.73:0/1988198583 wait complete. 2024-06-06T06:58:30.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:30 smithi073 ceph-mon[32303]: pgmap v347: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T06:58:30.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:30 smithi136 ceph-mon[35901]: pgmap v347: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T06:58:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:30 smithi170 ceph-mon[36108]: pgmap v347: 129 pgs: 129 active+clean; 594 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-06-06T06:58:30.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-06T06:58:31.183 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:31.557 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:31 smithi073 ceph-mon[32303]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:31.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:31 smithi136 ceph-mon[35901]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:31 smithi170 ceph-mon[36108]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:32.632 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:32 smithi073 ceph-mon[32303]: pgmap v348: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-06T06:58:32.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:32 smithi170 ceph-mon[36108]: pgmap v348: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-06T06:58:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:32 smithi136 ceph-mon[35901]: pgmap v348: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 5.8 KiB/s wr, 22 op/s 2024-06-06T06:58:33.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 -- 172.21.15.73:0/417172657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 msgr2=0x7f1ae010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:33.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/417172657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae010f3c0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f1ad800b0d0 tx=0x7f1ad800b3e0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 -- 172.21.15.73:0/417172657 shutdown_connections 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/417172657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae010f3c0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/417172657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 0x7f1ae0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/417172657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 -- 172.21.15.73:0/417172657 >> 172.21.15.73:0/417172657 conn(0x7f1ae00795f0 msgr2=0x7f1ae0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 -- 172.21.15.73:0/417172657 shutdown_connections 2024-06-06T06:58:33.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.189+0000 7f1ae4f38700 1 -- 172.21.15.73:0/417172657 wait complete. 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 Processor -- start 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 -- start start 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 0x7f1ae019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae019ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0112690 con 0x7f1ae0107840 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0112510 con 0x7f1ae0102740 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ae4f38700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1ae0112810 con 0x7f1ae01030f0 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:33.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1aded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57566/0 (socket says 172.21.15.73:57566) 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1aded9d700 1 -- 172.21.15.73:0/1318996975 learned_addr learned my addr 172.21.15.73:0/1318996975 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.190+0000 7f1ade59c700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 0x7f1ae019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ad7fff700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae019ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 -- 172.21.15.73:0/1318996975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 msgr2=0x7f1ae019ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae019ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 -- 172.21.15.73:0/1318996975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 msgr2=0x7f1ae019e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 0x7f1ae019e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 -- 172.21.15.73:0/1318996975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ae0199010 con 0x7f1ae0107840 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ad7fff700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1aded9d700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:33.194 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad801f070 con 0x7f1ae0107840 2024-06-06T06:58:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ae01992a0 con 0x7f1ae0107840 2024-06-06T06:58:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ad8003c00 con 0x7f1ae0107840 2024-06-06T06:58:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ad801b770 con 0x7f1ae0107840 2024-06-06T06:58:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.191+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ae01a55c0 con 0x7f1ae0107840 2024-06-06T06:58:33.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.192+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ae0069180 con 0x7f1ae0107840 2024-06-06T06:58:33.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.193+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1ad8022070 con 0x7f1ae0107840 2024-06-06T06:58:33.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.194+0000 7f1ad77fe700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 0x7f1acc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:33.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.194+0000 7f1ade59c700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 0x7f1acc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:33.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.194+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f1ad800f020 con 0x7f1ae0107840 2024-06-06T06:58:33.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.194+0000 7f1ade59c700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 0x7f1acc0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01999f0 tx=0x7f1ad0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:33.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.198+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ad8029020 con 0x7f1ae0107840 2024-06-06T06:58:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.431+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1ae00628d0 con 0x7f1acc0a2cb0 2024-06-06T06:58:33.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.457+0000 7f1ad77fe700 1 -- 172.21.15.73:0/1318996975 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1ae00628d0 con 0x7f1acc0a2cb0 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 msgr2=0x7f1acc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 0x7f1acc0a5170 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1ae01999f0 tx=0x7f1ad0009190 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 msgr2=0x7f1ae01988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1ad8009fd0 tx=0x7f1ad8007a20 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 shutdown_connections 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ae0107840 0x7f1ae01988b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1ae0102740 0x7f1ae019e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1acc0a2cb0 0x7f1acc0a5170 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 --2- 172.21.15.73:0/1318996975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1ae01030f0 0x7f1ae019ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 >> 172.21.15.73:0/1318996975 conn(0x7f1ae00795f0 msgr2=0x7f1ae0100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 shutdown_connections 2024-06-06T06:58:33.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:33.460+0000 7f1ae4f38700 1 -- 172.21.15.73:0/1318996975 wait complete. 2024-06-06T06:58:34.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-06T06:58:34.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:34 smithi073 ceph-mon[32303]: pgmap v349: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-06T06:58:34.543 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:34 smithi136 ceph-mon[35901]: pgmap v349: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-06T06:58:34.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:34 smithi170 ceph-mon[36108]: pgmap v349: 129 pgs: 129 active+clean; 612 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.7 KiB/s wr, 13 op/s 2024-06-06T06:58:35.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:35 smithi073 ceph-mon[32303]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:35.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:35 smithi136 ceph-mon[35901]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:35 smithi170 ceph-mon[36108]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.273+0000 7fea3efe3700 1 -- 172.21.15.73:0/1910068108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 msgr2=0x7fea381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.273+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1910068108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea381029b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fea240099c0 tx=0x7fea24009cd0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- 172.21.15.73:0/1910068108 shutdown_connections 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1910068108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 0x7fea3810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1910068108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1910068108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea381029b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- 172.21.15.73:0/1910068108 >> 172.21.15.73:0/1910068108 conn(0x7fea38079640 msgr2=0x7fea38079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- 172.21.15.73:0/1910068108 shutdown_connections 2024-06-06T06:58:36.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- 172.21.15.73:0/1910068108 wait complete. 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 Processor -- start 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- start start 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 0x7fea3819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea381126c0 con 0x7fea38102530 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea38112540 con 0x7fea381085d0 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.274+0000 7fea3efe3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea38112840 con 0x7fea38102ef0 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33202/0 (socket says 172.21.15.73:33202) 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 -- 172.21.15.73:0/1821364598 learned_addr learned my addr 172.21.15.73:0/1821364598 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea37fff700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 0x7fea3819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3d580700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 -- 172.21.15.73:0/1821364598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 msgr2=0x7fea3819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 0x7fea3819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 -- 172.21.15.73:0/1821364598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 msgr2=0x7fea381988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 -- 172.21.15.73:0/1821364598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea38199000 con 0x7fea38102530 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3d580700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3cd7f700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00c9b0 tx=0x7fea2c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c007b00 con 0x7fea38102530 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea2c007ca0 con 0x7fea38102530 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea381992f0 con 0x7fea38102530 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.275+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c018660 con 0x7fea38102530 2024-06-06T06:58:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.276+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a55b0 con 0x7fea38102530 2024-06-06T06:58:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.277+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea38069180 con 0x7fea38102530 2024-06-06T06:58:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.277+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea2c00f450 con 0x7fea38102530 2024-06-06T06:58:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.278+0000 7fea35ffb700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 0x7fea280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.278+0000 7fea37fff700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 0x7fea280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.278+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fea2c0cf540 con 0x7fea38102530 2024-06-06T06:58:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.278+0000 7fea37fff700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 0x7fea280a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea24000c00 tx=0x7fea24005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:36.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.282+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fea2c014030 con 0x7fea38102530 2024-06-06T06:58:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.517+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fea38199dd0 con 0x7fea280a2cb0 2024-06-06T06:58:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.523+0000 7fea35ffb700 1 -- 172.21.15.73:0/1821364598 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fea38199dd0 con 0x7fea280a2cb0 2024-06-06T06:58:36.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:36 smithi073 ceph-mon[32303]: pgmap v350: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-06-06T06:58:36.524 INFO:teuthology.orchestra.run.smithi073.stdout:/volumes/qa/sv_1/78ad03eb-0716-4804-bfc8-463454d03413 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 msgr2=0x7fea280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 0x7fea280a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fea24000c00 tx=0x7fea24005fb0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 msgr2=0x7fea3819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00c9b0 tx=0x7fea2c00ccc0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 shutdown_connections 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea381085d0 0x7fea381988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea38102530 0x7fea3819e6e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea280a2cb0 0x7fea280a5170 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 --2- 172.21.15.73:0/1821364598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea38102ef0 0x7fea3819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 >> 172.21.15.73:0/1821364598 conn(0x7fea38079640 msgr2=0x7fea38100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 shutdown_connections 2024-06-06T06:58:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:36.526+0000 7fea3efe3700 1 -- 172.21.15.73:0/1821364598 wait complete. 2024-06-06T06:58:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:36 smithi136 ceph-mon[35901]: pgmap v350: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-06-06T06:58:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:36 smithi170 ceph-mon[36108]: pgmap v350: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-06-06T06:58:37.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T06:58:37.429 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:38.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:38 smithi073 ceph-mon[32303]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:38.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:38 smithi073 ceph-mon[32303]: pgmap v351: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-06-06T06:58:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:38 smithi136 ceph-mon[35901]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:38.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:38 smithi136 ceph-mon[35901]: pgmap v351: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-06-06T06:58:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:38 smithi170 ceph-mon[36108]: from='client.14784 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T06:58:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:38 smithi170 ceph-mon[36108]: pgmap v351: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.5 KiB/s wr, 12 op/s 2024-06-06T06:58:39.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.357+0000 7ff3d79f9700 1 -- 172.21.15.73:0/1563689751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 msgr2=0x7ff3d010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:39.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.357+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/1563689751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d010e230 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009f90 tx=0x7ff3c800b330 comp rx=0 tx=0).stop 2024-06-06T06:58:39.360 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 -- 172.21.15.73:0/1563689751 shutdown_connections 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/1563689751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d010e230 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/1563689751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/1563689751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 0x7ff3d0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 -- 172.21.15.73:0/1563689751 >> 172.21.15.73:0/1563689751 conn(0x7ff3d0078300 msgr2=0x7ff3d0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 -- 172.21.15.73:0/1563689751 shutdown_connections 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 -- 172.21.15.73:0/1563689751 wait complete. 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.358+0000 7ff3d79f9700 1 Processor -- start 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 -- start start 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 0x7ff3d019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:39.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111510 con 0x7ff3d01073f0 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111390 con 0x7ff3d0101d40 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d79f9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d0111690 con 0x7ff3d0101380 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d5f96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d5f96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33228/0 (socket says 172.21.15.73:33228) 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d5f96700 1 -- 172.21.15.73:0/3926006043 learned_addr learned my addr 172.21.15.73:0/3926006043 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d4f94700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:39.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.359+0000 7ff3d5795700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 0x7ff3d019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 -- 172.21.15.73:0/3926006043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 msgr2=0x7ff3d019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 0x7ff3d019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 -- 172.21.15.73:0/3926006043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 msgr2=0x7ff3d019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 -- 172.21.15.73:0/3926006043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d0199040 con 0x7ff3d01073f0 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d4f94700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d5f96700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009790 tx=0x7ff3c800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8021070 con 0x7ff3d01073f0 2024-06-06T06:58:39.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d01992d0 con 0x7ff3d01073f0 2024-06-06T06:58:39.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3c800ce50 con 0x7ff3d01073f0 2024-06-06T06:58:39.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c8005040 con 0x7ff3d01073f0 2024-06-06T06:58:39.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.360+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d01a55f0 con 0x7ff3d01073f0 2024-06-06T06:58:39.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.361+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d0069180 con 0x7ff3d01073f0 2024-06-06T06:58:39.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.363+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff3c8024070 con 0x7ff3d01073f0 2024-06-06T06:58:39.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.364+0000 7ff3c27fc700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 0x7ff3bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:39.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.364+0000 7ff3d5795700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 0x7ff3bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:39.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.364+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7ff3c800f020 con 0x7ff3d01073f0 2024-06-06T06:58:39.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.365+0000 7ff3d5795700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 0x7ff3bc0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009750 tx=0x7ff3cc009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:39.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.368+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff3c802b020 con 0x7ff3d01073f0 2024-06-06T06:58:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.624+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d01a57c0 con 0x7ff3d01073f0 2024-06-06T06:58:39.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.624+0000 7ff3c27fc700 1 -- 172.21.15.73:0/3926006043 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7ff3c809d940 con 0x7ff3d01073f0 2024-06-06T06:58:39.625 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:39.626 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 msgr2=0x7ff3bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 0x7ff3bc0a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff3cc009750 tx=0x7ff3cc009310 comp rx=0 tx=0).stop 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 msgr2=0x7ff3d01988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff3c8009790 tx=0x7ff3c800cc30 comp rx=0 tx=0).stop 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 shutdown_connections 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff3bc0a2cb0 0x7ff3bc0a5170 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3d0101380 0x7ff3d019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d01073f0 0x7ff3d01988e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 --2- 172.21.15.73:0/3926006043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3d0101d40 0x7ff3d019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:39.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 >> 172.21.15.73:0/3926006043 conn(0x7ff3d0078300 msgr2=0x7ff3d0100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:39.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 shutdown_connections 2024-06-06T06:58:39.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:39.627+0000 7ff3d79f9700 1 -- 172.21.15.73:0/3926006043 wait complete. 2024-06-06T06:58:39.629 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T06:58:40.415 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:58:40.670 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:40.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:40 smithi073 ceph-mon[32303]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-06-06T06:58:40.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:40 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3926006043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:58:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:40 smithi136 ceph-mon[35901]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-06-06T06:58:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:40 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3926006043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:58:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:40 smithi170 ceph-mon[36108]: pgmap v352: 129 pgs: 129 active+clean; 635 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.3 KiB/s wr, 10 op/s 2024-06-06T06:58:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:40 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3926006043' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T06:58:42.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 -- 172.21.15.73:0/1052007286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a41019d0 msgr2=0x7f98a4101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:42.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 --2- 172.21.15.73:0/1052007286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a41019d0 0x7f98a4101db0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f98940099c0 tx=0x7f9894009cd0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 -- 172.21.15.73:0/1052007286 shutdown_connections 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 --2- 172.21.15.73:0/1052007286 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 0x7f98a410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 --2- 172.21.15.73:0/1052007286 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a41019d0 0x7f98a4101db0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 --2- 172.21.15.73:0/1052007286 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a4102380 0x7f98a4105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 -- 172.21.15.73:0/1052007286 >> 172.21.15.73:0/1052007286 conn(0x7f98a4078340 msgr2=0x7f98a4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 -- 172.21.15.73:0/1052007286 shutdown_connections 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.371+0000 7f98a9b92700 1 -- 172.21.15.73:0/1052007286 wait complete. 2024-06-06T06:58:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 Processor -- start 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 -- start start 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a41019d0 0x7f98a4199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 0x7f98a41a4a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98a41115b0 con 0x7f98a4102380 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4111430 con 0x7f98a41019d0 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a9b92700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98a4111730 con 0x7f98a4106720 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33242/0 (socket says 172.21.15.73:33242) 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a2ffd700 1 -- 172.21.15.73:0/3722661298 learned_addr learned my addr 172.21.15.73:0/3722661298 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a37fe700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a41019d0 0x7f98a4199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.372+0000 7f98a3fff700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 0x7f98a41a4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 -- 172.21.15.73:0/3722661298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 msgr2=0x7f98a41a4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 0x7f98a41a4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 -- 172.21.15.73:0/3722661298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a41019d0 msgr2=0x7f98a4199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a41019d0 0x7f98a4199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 -- 172.21.15.73:0/3722661298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98a41a4fd0 con 0x7f98a4102380 2024-06-06T06:58:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a2ffd700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9898017070 con 0x7f98a4102380 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98a41a51a0 con 0x7f98a4102380 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f989800fd40 con 0x7f98a4102380 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989800e3b0 con 0x7f98a4102380 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.373+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98a41a5730 con 0x7f98a4102380 2024-06-06T06:58:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.374+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98a4102ad0 con 0x7f98a4102380 2024-06-06T06:58:42.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.375+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f989801a070 con 0x7f98a4102380 2024-06-06T06:58:42.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.376+0000 7f98a0ff9700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 0x7f988c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:42.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.376+0000 7f98a37fe700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 0x7f988c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:42.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.376+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f98980ced10 con 0x7f98a4102380 2024-06-06T06:58:42.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.376+0000 7f98a37fe700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 0x7f988c0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9894003760 tx=0x7f9894005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:42.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.380+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f989809b7b0 con 0x7f98a4102380 2024-06-06T06:58:42.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.581+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98a402dca0 con 0x7f98a4102380 2024-06-06T06:58:42.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.583+0000 7f98a0ff9700 1 -- 172.21.15.73:0/3722661298 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) ==== 74+0+24628 (secure 0 0 0) 0x7f989809b0b0 con 0x7f98a4102380 2024-06-06T06:58:42.584 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:42.584 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":98,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:57:52.391218+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:58:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 msgr2=0x7f988c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 0x7f988c0a5170 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9894003760 tx=0x7f9894005fb0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 msgr2=0x7f98a4199f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f989800d950 tx=0x7f989800dc60 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 shutdown_connections 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f988c0a2cb0 0x7f988c0a5170 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98a4106720 0x7f98a41a4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98a41019d0 0x7f98a4199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 --2- 172.21.15.73:0/3722661298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a4102380 0x7f98a4199f50 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 >> 172.21.15.73:0/3722661298 conn(0x7f98a4078340 msgr2=0x7f98a40ff9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:42.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 shutdown_connections 2024-06-06T06:58:42.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:42.585+0000 7f98a9b92700 1 -- 172.21.15.73:0/3722661298 wait complete. 2024-06-06T06:58:42.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:42 smithi073 ceph-mon[32303]: pgmap v353: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-06T06:58:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:42 smithi136 ceph-mon[35901]: pgmap v353: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-06T06:58:42.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:42 smithi170 ceph-mon[36108]: pgmap v353: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-06T06:58:43.358 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T06:58:43.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T06:58:43.607 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:43.729 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:43 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3722661298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:43 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3722661298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:43 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3722661298' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:44.770 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:44 smithi073 ceph-mon[32303]: pgmap v354: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-06T06:58:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:44 smithi136 ceph-mon[35901]: pgmap v354: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-06T06:58:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:44 smithi170 ceph-mon[36108]: pgmap v354: 129 pgs: 129 active+clean; 639 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-06-06T06:58:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 -- 172.21.15.73:0/4229497832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a01073b0 msgr2=0x7fb5a006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/4229497832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a01073b0 0x7fb5a006abe0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb590009990 tx=0x7fb590009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 -- 172.21.15.73:0/4229497832 shutdown_connections 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/4229497832 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a0104ec0 0x7fb5a0105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/4229497832 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a01073b0 0x7fb5a006abe0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/4229497832 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a006b120 0x7fb5a006b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 -- 172.21.15.73:0/4229497832 >> 172.21.15.73:0/4229497832 conn(0x7fb5a0078360 msgr2=0x7fb5a0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 -- 172.21.15.73:0/4229497832 shutdown_connections 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.474+0000 7fb5a65e9700 1 -- 172.21.15.73:0/4229497832 wait complete. 2024-06-06T06:58:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 Processor -- start 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 -- start start 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0104ec0 0x7fb5a019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a01073b0 0x7fb5a01a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0118c20 con 0x7fb5a006b120 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0118aa0 con 0x7fb5a01073b0 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb5a65e9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a0118da0 con 0x7fb5a0104ec0 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:45.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38140/0 (socket says 172.21.15.73:38140) 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 -- 172.21.15.73:0/105512815 learned_addr learned my addr 172.21.15.73:0/105512815 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb597fff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0104ec0 0x7fb5a019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 -- 172.21.15.73:0/105512815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0104ec0 msgr2=0x7fb5a019f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0104ec0 0x7fb5a019f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 -- 172.21.15.73:0/105512815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a01073b0 msgr2=0x7fb5a01a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a01073b0 0x7fb5a01a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.475+0000 7fb59ffff700 1 -- 172.21.15.73:0/105512815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a01a4200 con 0x7fb5a006b120 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb59ffff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb590000c00 tx=0x7fb590003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb590027070 con 0x7fb5a006b120 2024-06-06T06:58:45.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb59000bdc0 con 0x7fb5a006b120 2024-06-06T06:58:45.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59000f460 con 0x7fb5a006b120 2024-06-06T06:58:45.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a01a4490 con 0x7fb5a006b120 2024-06-06T06:58:45.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.476+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a01abd70 con 0x7fb5a006b120 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.477+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5a0069180 con 0x7fb5a006b120 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.478+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb59002a070 con 0x7fb5a006b120 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.479+0000 7fb59dffb700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 0x7fb58c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.479+0000 7fb597fff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 0x7fb58c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.479+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7fb5900d0420 con 0x7fb5a006b120 2024-06-06T06:58:45.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.479+0000 7fb597fff700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 0x7fb58c0a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb588005950 tx=0x7fb5880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:45.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.483+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb590013070 con 0x7fb5a006b120 2024-06-06T06:58:45.678 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.676+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb5a0100ac0 con 0x7fb58c0a2cb0 2024-06-06T06:58:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.688+0000 7fb59dffb700 1 -- 172.21.15.73:0/105512815 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340680 (secure 0 0 0) 0x7fb5a0100ac0 con 0x7fb58c0a2cb0 2024-06-06T06:58:45.695 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:45.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 msgr2=0x7fb58c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:45.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 0x7fb58c0a5170 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb588005950 tx=0x7fb5880058e0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 msgr2=0x7fb5a019f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb590000c00 tx=0x7fb590003b30 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 shutdown_connections 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5a01073b0 0x7fb5a01a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5a006b120 0x7fb5a019f1f0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb58c0a2cb0 0x7fb58c0a5170 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 --2- 172.21.15.73:0/105512815 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb5a0104ec0 0x7fb5a019f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.691+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 >> 172.21.15.73:0/105512815 conn(0x7fb5a0078360 msgr2=0x7fb5a01003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.692+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 shutdown_connections 2024-06-06T06:58:45.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:45.692+0000 7fb5a65e9700 1 -- 172.21.15.73:0/105512815 wait complete. 2024-06-06T06:58:45.700 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T06:58:46.471 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-06-06T06:58:45.320609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660729,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":421,"num_read_kb":342,"num_write":433,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383204,"kb_used_data":13252,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558620,"statfs":{"total":1151940427776,"available":1151548026880,"internally_reserved":0,"allocated":13570048,"data_stored":8594307,"data_compressed":424821,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33837,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":134,"num_read_kb":105,"num_write":27,"num_write_kb":43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003487"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399432+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T06:57:52.399432+0000","last_peered":"2024-06-06T06:57:52.399432+0000","last_clean":"2024-06-06T06:57:52.399432+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T06:57:52.399432+0000","last_undegraded":"2024-06-06T06:57:52.399432+0000","last_fullsized":"2024-06-06T06:57:52.399432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430831+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T06:57:52.430831+0000","last_peered":"2024-06-06T06:57:52.430831+0000","last_clean":"2024-06-06T06:57:52.430831+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T06:57:52.430831+0000","last_undegraded":"2024-06-06T06:57:52.430831+0000","last_fullsized":"2024-06-06T06:57:52.430831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430720+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T06:57:52.430720+0000","last_peered":"2024-06-06T06:57:52.430720+0000","last_clean":"2024-06-06T06:57:52.430720+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T06:57:52.430720+0000","last_undegraded":"2024-06-06T06:57:52.430720+0000","last_fullsized":"2024-06-06T06:57:52.430720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399516+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T06:57:52.399516+0000","last_peered":"2024-06-06T06:57:52.399516+0000","last_clean":"2024-06-06T06:57:52.399516+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T06:57:52.399516+0000","last_undegraded":"2024-06-06T06:57:52.399516+0000","last_fullsized":"2024-06-06T06:57:52.399516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404121+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T06:57:52.404121+0000","last_peered":"2024-06-06T06:57:52.404121+0000","last_clean":"2024-06-06T06:57:52.404121+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T06:57:52.404121+0000","last_undegraded":"2024-06-06T06:57:52.404121+0000","last_fullsized":"2024-06-06T06:57:52.404121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.405561+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T06:57:52.405561+0000","last_peered":"2024-06-06T06:57:52.405561+0000","last_clean":"2024-06-06T06:57:52.405561+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T06:57:52.405561+0000","last_undegraded":"2024-06-06T06:57:52.405561+0000","last_fullsized":"2024-06-06T06:57:52.405561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395075+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T06:57:52.395075+0000","last_peered":"2024-06-06T06:57:52.395075+0000","last_clean":"2024-06-06T06:57:52.395075+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T06:57:52.395075+0000","last_undegraded":"2024-06-06T06:57:52.395075+0000","last_fullsized":"2024-06-06T06:57:52.395075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394689+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T06:57:52.394689+0000","last_peered":"2024-06-06T06:57:52.394689+0000","last_clean":"2024-06-06T06:57:52.394689+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T06:57:52.394689+0000","last_undegraded":"2024-06-06T06:57:52.394689+0000","last_fullsized":"2024-06-06T06:57:52.394689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398135+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T06:57:52.398135+0000","last_peered":"2024-06-06T06:57:52.398135+0000","last_clean":"2024-06-06T06:57:52.398135+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T06:57:52.398135+0000","last_undegraded":"2024-06-06T06:57:52.398135+0000","last_fullsized":"2024-06-06T06:57:52.398135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.401359+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T06:57:52.401359+0000","last_peered":"2024-06-06T06:57:52.401359+0000","last_clean":"2024-06-06T06:57:52.401359+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T06:57:52.401359+0000","last_undegraded":"2024-06-06T06:57:52.401359+0000","last_fullsized":"2024-06-06T06:57:52.401359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395761+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T06:57:52.395761+0000","last_peered":"2024-06-06T06:57:52.395761+0000","last_clean":"2024-06-06T06:57:52.395761+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T06:57:52.395761+0000","last_undegraded":"2024-06-06T06:57:52.395761+0000","last_fullsized":"2024-06-06T06:57:52.395761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399025+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T06:57:52.399025+0000","last_peered":"2024-06-06T06:57:52.399025+0000","last_clean":"2024-06-06T06:57:52.399025+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T06:57:52.399025+0000","last_undegraded":"2024-06-06T06:57:52.399025+0000","last_fullsized":"2024-06-06T06:57:52.399025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431263+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T06:57:52.431263+0000","last_peered":"2024-06-06T06:57:52.431263+0000","last_clean":"2024-06-06T06:57:52.431263+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T06:57:52.431263+0000","last_undegraded":"2024-06-06T06:57:52.431263+0000","last_fullsized":"2024-06-06T06:57:52.431263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430192+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T06:57:52.430192+0000","last_peered":"2024-06-06T06:57:52.430192+0000","last_clean":"2024-06-06T06:57:52.430192+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T06:57:52.430192+0000","last_undegraded":"2024-06-06T06:57:52.430192+0000","last_fullsized":"2024-06-06T06:57:52.430192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430327+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T06:57:52.430327+0000","last_peered":"2024-06-06T06:57:52.430327+0000","last_clean":"2024-06-06T06:57:52.430327+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T06:57:52.430327+0000","last_undegraded":"2024-06-06T06:57:52.430327+0000","last_fullsized":"2024-06-06T06:57:52.430327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397739+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T06:57:52.397739+0000","last_peered":"2024-06-06T06:57:52.397739+0000","last_clean":"2024-06-06T06:57:52.397739+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T06:57:52.397739+0000","last_undegraded":"2024-06-06T06:57:52.397739+0000","last_fullsized":"2024-06-06T06:57:52.397739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399182+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T06:57:52.399182+0000","last_peered":"2024-06-06T06:57:52.399182+0000","last_clean":"2024-06-06T06:57:52.399182+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T06:57:52.399182+0000","last_undegraded":"2024-06-06T06:57:52.399182+0000","last_fullsized":"2024-06-06T06:57:52.399182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397247+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T06:57:52.397247+0000","last_peered":"2024-06-06T06:57:52.397247+0000","last_clean":"2024-06-06T06:57:52.397247+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T06:57:52.397247+0000","last_undegraded":"2024-06-06T06:57:52.397247+0000","last_fullsized":"2024-06-06T06:57:52.397247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396069+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T06:57:52.396069+0000","last_peered":"2024-06-06T06:57:52.396069+0000","last_clean":"2024-06-06T06:57:52.396069+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T06:57:52.396069+0000","last_undegraded":"2024-06-06T06:57:52.396069+0000","last_fullsized":"2024-06-06T06:57:52.396069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394994+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T06:57:52.394994+0000","last_peered":"2024-06-06T06:57:52.394994+0000","last_clean":"2024-06-06T06:57:52.394994+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T06:57:52.394994+0000","last_undegraded":"2024-06-06T06:57:52.394994+0000","last_fullsized":"2024-06-06T06:57:52.394994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":79,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:42.456521+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T06:58:42.456521+0000","last_peered":"2024-06-06T06:58:42.456521+0000","last_clean":"2024-06-06T06:58:42.456521+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T06:58:42.456521+0000","last_undegraded":"2024-06-06T06:58:42.456521+0000","last_fullsized":"2024-06-06T06:58:42.456521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395032+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T06:57:52.395032+0000","last_peered":"2024-06-06T06:57:52.395032+0000","last_clean":"2024-06-06T06:57:52.395032+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T06:57:52.395032+0000","last_undegraded":"2024-06-06T06:57:52.395032+0000","last_fullsized":"2024-06-06T06:57:52.395032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397919+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T06:57:52.397919+0000","last_peered":"2024-06-06T06:57:52.397919+0000","last_clean":"2024-06-06T06:57:52.397919+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T06:57:52.397919+0000","last_undegraded":"2024-06-06T06:57:52.397919+0000","last_fullsized":"2024-06-06T06:57:52.397919+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395414+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T06:57:52.395414+0000","last_peered":"2024-06-06T06:57:52.395414+0000","last_clean":"2024-06-06T06:57:52.395414+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T06:57:52.395414+0000","last_undegraded":"2024-06-06T06:57:52.395414+0000","last_fullsized":"2024-06-06T06:57:52.395414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430711+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T06:57:52.430711+0000","last_peered":"2024-06-06T06:57:52.430711+0000","last_clean":"2024-06-06T06:57:52.430711+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T06:57:52.430711+0000","last_undegraded":"2024-06-06T06:57:52.430711+0000","last_fullsized":"2024-06-06T06:57:52.430711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.406751+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T06:57:52.406751+0000","last_peered":"2024-06-06T06:57:52.406751+0000","last_clean":"2024-06-06T06:57:52.406751+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T06:57:52.406751+0000","last_undegraded":"2024-06-06T06:57:52.406751+0000","last_fullsized":"2024-06-06T06:57:52.406751+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.422789+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T06:58:19.422789+0000","last_peered":"2024-06-06T06:58:19.422789+0000","last_clean":"2024-06-06T06:58:19.422789+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T06:58:19.422789+0000","last_undegraded":"2024-06-06T06:58:19.422789+0000","last_fullsized":"2024-06-06T06:58:19.422789+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399174+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T06:57:52.399174+0000","last_peered":"2024-06-06T06:57:52.399174+0000","last_clean":"2024-06-06T06:57:52.399174+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T06:57:52.399174+0000","last_undegraded":"2024-06-06T06:57:52.399174+0000","last_fullsized":"2024-06-06T06:57:52.399174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397583+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T06:57:52.397583+0000","last_peered":"2024-06-06T06:57:52.397583+0000","last_clean":"2024-06-06T06:57:52.397583+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T06:57:52.397583+0000","last_undegraded":"2024-06-06T06:57:52.397583+0000","last_fullsized":"2024-06-06T06:57:52.397583+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420196+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T06:58:18.420196+0000","last_peered":"2024-06-06T06:58:18.420196+0000","last_clean":"2024-06-06T06:58:18.420196+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T06:58:18.420196+0000","last_undegraded":"2024-06-06T06:58:18.420196+0000","last_fullsized":"2024-06-06T06:58:18.420196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399333+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T06:57:52.399333+0000","last_peered":"2024-06-06T06:57:52.399333+0000","last_clean":"2024-06-06T06:57:52.399333+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T06:57:52.399333+0000","last_undegraded":"2024-06-06T06:57:52.399333+0000","last_fullsized":"2024-06-06T06:57:52.399333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395563+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T06:57:52.395563+0000","last_peered":"2024-06-06T06:57:52.395563+0000","last_clean":"2024-06-06T06:57:52.395563+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T06:57:52.395563+0000","last_undegraded":"2024-06-06T06:57:52.395563+0000","last_fullsized":"2024-06-06T06:57:52.395563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":78,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:43.459556+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T06:58:43.459556+0000","last_peered":"2024-06-06T06:58:43.459556+0000","last_clean":"2024-06-06T06:58:43.459556+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T06:58:43.459556+0000","last_undegraded":"2024-06-06T06:58:43.459556+0000","last_fullsized":"2024-06-06T06:58:43.459556+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404268+0000","last_change":"2024-06-06T06:57:07.549415+0000","last_active":"2024-06-06T06:57:52.404268+0000","last_peered":"2024-06-06T06:57:52.404268+0000","last_clean":"2024-06-06T06:57:52.404268+0000","last_became_active":"2024-06-06T06:57:07.549069+0000","last_became_peered":"2024-06-06T06:57:07.549069+0000","last_unstale":"2024-06-06T06:57:52.404268+0000","last_undegraded":"2024-06-06T06:57:52.404268+0000","last_fullsized":"2024-06-06T06:57:52.404268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:44.047896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.432492+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T06:57:52.432492+0000","last_peered":"2024-06-06T06:57:52.432492+0000","last_clean":"2024-06-06T06:57:52.432492+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T06:57:52.432492+0000","last_undegraded":"2024-06-06T06:57:52.432492+0000","last_fullsized":"2024-06-06T06:57:52.432492+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.415153+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T06:58:17.415153+0000","last_peered":"2024-06-06T06:58:17.415153+0000","last_clean":"2024-06-06T06:58:17.415153+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T06:58:17.415153+0000","last_undegraded":"2024-06-06T06:58:17.415153+0000","last_fullsized":"2024-06-06T06:58:17.415153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398951+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T06:57:52.398951+0000","last_peered":"2024-06-06T06:57:52.398951+0000","last_clean":"2024-06-06T06:57:52.398951+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T06:57:52.398951+0000","last_undegraded":"2024-06-06T06:57:52.398951+0000","last_fullsized":"2024-06-06T06:57:52.398951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399074+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T06:57:52.399074+0000","last_peered":"2024-06-06T06:57:52.399074+0000","last_clean":"2024-06-06T06:57:52.399074+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T06:57:52.399074+0000","last_undegraded":"2024-06-06T06:57:52.399074+0000","last_fullsized":"2024-06-06T06:57:52.399074+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:16.408627+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T06:58:16.408627+0000","last_peered":"2024-06-06T06:58:16.408627+0000","last_clean":"2024-06-06T06:58:16.408627+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T06:58:16.408627+0000","last_undegraded":"2024-06-06T06:58:16.408627+0000","last_fullsized":"2024-06-06T06:58:16.408627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394818+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T06:57:52.394818+0000","last_peered":"2024-06-06T06:57:52.394818+0000","last_clean":"2024-06-06T06:57:52.394818+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T06:57:52.394818+0000","last_undegraded":"2024-06-06T06:57:52.394818+0000","last_fullsized":"2024-06-06T06:57:52.394818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430739+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T06:57:52.430739+0000","last_peered":"2024-06-06T06:57:52.430739+0000","last_clean":"2024-06-06T06:57:52.430739+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T06:57:52.430739+0000","last_undegraded":"2024-06-06T06:57:52.430739+0000","last_fullsized":"2024-06-06T06:57:52.430739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420058+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T06:58:18.420058+0000","last_peered":"2024-06-06T06:58:18.420058+0000","last_clean":"2024-06-06T06:58:18.420058+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T06:58:18.420058+0000","last_undegraded":"2024-06-06T06:58:18.420058+0000","last_fullsized":"2024-06-06T06:58:18.420058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396538+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T06:57:52.396538+0000","last_peered":"2024-06-06T06:57:52.396538+0000","last_clean":"2024-06-06T06:57:52.396538+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T06:57:52.396538+0000","last_undegraded":"2024-06-06T06:57:52.396538+0000","last_fullsized":"2024-06-06T06:57:52.396538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394734+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T06:57:52.394734+0000","last_peered":"2024-06-06T06:57:52.394734+0000","last_clean":"2024-06-06T06:57:52.394734+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T06:57:52.394734+0000","last_undegraded":"2024-06-06T06:57:52.394734+0000","last_fullsized":"2024-06-06T06:57:52.394734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":96,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.460290+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T06:58:44.460290+0000","last_peered":"2024-06-06T06:58:44.460290+0000","last_clean":"2024-06-06T06:58:44.460290+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T06:58:44.460290+0000","last_undegraded":"2024-06-06T06:58:44.460290+0000","last_fullsized":"2024-06-06T06:58:44.460290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398571+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T06:57:52.398571+0000","last_peered":"2024-06-06T06:57:52.398571+0000","last_clean":"2024-06-06T06:57:52.398571+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:57:52.398571+0000","last_undegraded":"2024-06-06T06:57:52.398571+0000","last_fullsized":"2024-06-06T06:57:52.398571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396489+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T06:57:52.396489+0000","last_peered":"2024-06-06T06:57:52.396489+0000","last_clean":"2024-06-06T06:57:52.396489+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T06:57:52.396489+0000","last_undegraded":"2024-06-06T06:57:52.396489+0000","last_fullsized":"2024-06-06T06:57:52.396489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420037+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T06:58:18.420037+0000","last_peered":"2024-06-06T06:58:18.420037+0000","last_clean":"2024-06-06T06:58:18.420037+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T06:58:18.420037+0000","last_undegraded":"2024-06-06T06:58:18.420037+0000","last_fullsized":"2024-06-06T06:58:18.420037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430942+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T06:57:52.430942+0000","last_peered":"2024-06-06T06:57:52.430942+0000","last_clean":"2024-06-06T06:57:52.430942+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T06:57:52.430942+0000","last_undegraded":"2024-06-06T06:57:52.430942+0000","last_fullsized":"2024-06-06T06:57:52.430942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404541+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T06:57:52.404541+0000","last_peered":"2024-06-06T06:57:52.404541+0000","last_clean":"2024-06-06T06:57:52.404541+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T06:57:52.404541+0000","last_undegraded":"2024-06-06T06:57:52.404541+0000","last_fullsized":"2024-06-06T06:57:52.404541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.414532+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T06:58:17.414532+0000","last_peered":"2024-06-06T06:58:17.414532+0000","last_clean":"2024-06-06T06:58:17.414532+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T06:58:17.414532+0000","last_undegraded":"2024-06-06T06:58:17.414532+0000","last_fullsized":"2024-06-06T06:58:17.414532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398894+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T06:57:52.398894+0000","last_peered":"2024-06-06T06:57:52.398894+0000","last_clean":"2024-06-06T06:57:52.398894+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T06:57:52.398894+0000","last_undegraded":"2024-06-06T06:57:52.398894+0000","last_fullsized":"2024-06-06T06:57:52.398894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396704+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T06:57:52.396704+0000","last_peered":"2024-06-06T06:57:52.396704+0000","last_clean":"2024-06-06T06:57:52.396704+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T06:57:52.396704+0000","last_undegraded":"2024-06-06T06:57:52.396704+0000","last_fullsized":"2024-06-06T06:57:52.396704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'35","reported_seq":136,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.464729+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T06:58:44.464729+0000","last_peered":"2024-06-06T06:58:44.464729+0000","last_clean":"2024-06-06T06:58:44.464729+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T06:58:44.464729+0000","last_undegraded":"2024-06-06T06:58:44.464729+0000","last_fullsized":"2024-06-06T06:58:44.464729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62925,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":92,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404658+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T06:57:52.404658+0000","last_peered":"2024-06-06T06:57:52.404658+0000","last_clean":"2024-06-06T06:57:52.404658+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T06:57:52.404658+0000","last_undegraded":"2024-06-06T06:57:52.404658+0000","last_fullsized":"2024-06-06T06:57:52.404658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399635+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T06:57:52.399635+0000","last_peered":"2024-06-06T06:57:52.399635+0000","last_clean":"2024-06-06T06:57:52.399635+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T06:57:52.399635+0000","last_undegraded":"2024-06-06T06:57:52.399635+0000","last_fullsized":"2024-06-06T06:57:52.399635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.423045+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T06:58:19.423045+0000","last_peered":"2024-06-06T06:58:19.423045+0000","last_clean":"2024-06-06T06:58:19.423045+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T06:58:19.423045+0000","last_undegraded":"2024-06-06T06:58:19.423045+0000","last_fullsized":"2024-06-06T06:58:19.423045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431059+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T06:57:52.431059+0000","last_peered":"2024-06-06T06:57:52.431059+0000","last_clean":"2024-06-06T06:57:52.431059+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T06:57:52.431059+0000","last_undegraded":"2024-06-06T06:57:52.431059+0000","last_fullsized":"2024-06-06T06:57:52.431059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398283+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T06:57:52.398283+0000","last_peered":"2024-06-06T06:57:52.398283+0000","last_clean":"2024-06-06T06:57:52.398283+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T06:57:52.398283+0000","last_undegraded":"2024-06-06T06:57:52.398283+0000","last_fullsized":"2024-06-06T06:57:52.398283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419019+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T06:58:18.419019+0000","last_peered":"2024-06-06T06:58:18.419019+0000","last_clean":"2024-06-06T06:58:18.419019+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T06:58:18.419019+0000","last_undegraded":"2024-06-06T06:58:18.419019+0000","last_fullsized":"2024-06-06T06:58:18.419019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395770+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T06:57:52.395770+0000","last_peered":"2024-06-06T06:57:52.395770+0000","last_clean":"2024-06-06T06:57:52.395770+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T06:57:52.395770+0000","last_undegraded":"2024-06-06T06:57:52.395770+0000","last_fullsized":"2024-06-06T06:57:52.395770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397189+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T06:57:52.397189+0000","last_peered":"2024-06-06T06:57:52.397189+0000","last_clean":"2024-06-06T06:57:52.397189+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T06:57:52.397189+0000","last_undegraded":"2024-06-06T06:57:52.397189+0000","last_fullsized":"2024-06-06T06:57:52.397189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:39.453205+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T06:58:39.453205+0000","last_peered":"2024-06-06T06:58:39.453205+0000","last_clean":"2024-06-06T06:58:39.453205+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T06:58:39.453205+0000","last_undegraded":"2024-06-06T06:58:39.453205+0000","last_fullsized":"2024-06-06T06:58:39.453205+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395086+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T06:57:52.395086+0000","last_peered":"2024-06-06T06:57:52.395086+0000","last_clean":"2024-06-06T06:57:52.395086+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T06:57:52.395086+0000","last_undegraded":"2024-06-06T06:57:52.395086+0000","last_fullsized":"2024-06-06T06:57:52.395086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431540+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T06:57:52.431540+0000","last_peered":"2024-06-06T06:57:52.431540+0000","last_clean":"2024-06-06T06:57:52.431540+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T06:57:52.431540+0000","last_undegraded":"2024-06-06T06:57:52.431540+0000","last_fullsized":"2024-06-06T06:57:52.431540+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419689+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T06:58:18.419689+0000","last_peered":"2024-06-06T06:58:18.419689+0000","last_clean":"2024-06-06T06:58:18.419689+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T06:58:18.419689+0000","last_undegraded":"2024-06-06T06:58:18.419689+0000","last_fullsized":"2024-06-06T06:58:18.419689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398796+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T06:57:52.398796+0000","last_peered":"2024-06-06T06:57:52.398796+0000","last_clean":"2024-06-06T06:57:52.398796+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T06:57:52.398796+0000","last_undegraded":"2024-06-06T06:57:52.398796+0000","last_fullsized":"2024-06-06T06:57:52.398796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398409+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T06:57:52.398409+0000","last_peered":"2024-06-06T06:57:52.398409+0000","last_clean":"2024-06-06T06:57:52.398409+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T06:57:52.398409+0000","last_undegraded":"2024-06-06T06:57:52.398409+0000","last_fullsized":"2024-06-06T06:57:52.398409+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.403648+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T06:58:17.403648+0000","last_peered":"2024-06-06T06:58:17.403648+0000","last_clean":"2024-06-06T06:58:17.403648+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T06:58:17.403648+0000","last_undegraded":"2024-06-06T06:58:17.403648+0000","last_fullsized":"2024-06-06T06:58:17.403648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.432469+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T06:57:52.432469+0000","last_peered":"2024-06-06T06:57:52.432469+0000","last_clean":"2024-06-06T06:57:52.432469+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T06:57:52.432469+0000","last_undegraded":"2024-06-06T06:57:52.432469+0000","last_fullsized":"2024-06-06T06:57:52.432469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.403985+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T06:57:52.403985+0000","last_peered":"2024-06-06T06:57:52.403985+0000","last_clean":"2024-06-06T06:57:52.403985+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T06:57:52.403985+0000","last_undegraded":"2024-06-06T06:57:52.403985+0000","last_fullsized":"2024-06-06T06:57:52.403985+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396937+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T06:57:52.396937+0000","last_peered":"2024-06-06T06:57:52.396937+0000","last_clean":"2024-06-06T06:57:52.396937+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T06:57:52.396937+0000","last_undegraded":"2024-06-06T06:57:52.396937+0000","last_fullsized":"2024-06-06T06:57:52.396937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400758+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:57:52.400758+0000","last_peered":"2024-06-06T06:57:52.400758+0000","last_clean":"2024-06-06T06:57:52.400758+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T06:57:52.400758+0000","last_undegraded":"2024-06-06T06:57:52.400758+0000","last_fullsized":"2024-06-06T06:57:52.400758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431248+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T06:57:52.431248+0000","last_peered":"2024-06-06T06:57:52.431248+0000","last_clean":"2024-06-06T06:57:52.431248+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T06:57:52.431248+0000","last_undegraded":"2024-06-06T06:57:52.431248+0000","last_fullsized":"2024-06-06T06:57:52.431248+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'5","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:40.454564+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T06:58:40.454564+0000","last_peered":"2024-06-06T06:58:40.454564+0000","last_clean":"2024-06-06T06:58:40.454564+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T06:58:40.454564+0000","last_undegraded":"2024-06-06T06:58:40.454564+0000","last_fullsized":"2024-06-06T06:58:40.454564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":24,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400210+0000","last_change":"2024-06-06T06:57:07.551478+0000","last_active":"2024-06-06T06:57:52.400210+0000","last_peered":"2024-06-06T06:57:52.400210+0000","last_clean":"2024-06-06T06:57:52.400210+0000","last_became_active":"2024-06-06T06:57:07.550554+0000","last_became_peered":"2024-06-06T06:57:07.550554+0000","last_unstale":"2024-06-06T06:57:52.400210+0000","last_undegraded":"2024-06-06T06:57:52.400210+0000","last_fullsized":"2024-06-06T06:57:52.400210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:30:05.500978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431369+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T06:57:52.431369+0000","last_peered":"2024-06-06T06:57:52.431369+0000","last_clean":"2024-06-06T06:57:52.431369+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T06:57:52.431369+0000","last_undegraded":"2024-06-06T06:57:52.431369+0000","last_fullsized":"2024-06-06T06:57:52.431369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398322+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T06:57:52.398322+0000","last_peered":"2024-06-06T06:57:52.398322+0000","last_clean":"2024-06-06T06:57:52.398322+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T06:57:52.398322+0000","last_undegraded":"2024-06-06T06:57:52.398322+0000","last_fullsized":"2024-06-06T06:57:52.398322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":68,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:39.452456+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T06:58:39.452456+0000","last_peered":"2024-06-06T06:58:39.452456+0000","last_clean":"2024-06-06T06:58:39.452456+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T06:58:39.452456+0000","last_undegraded":"2024-06-06T06:58:39.452456+0000","last_fullsized":"2024-06-06T06:58:39.452456+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395646+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T06:57:52.395646+0000","last_peered":"2024-06-06T06:57:52.395646+0000","last_clean":"2024-06-06T06:57:52.395646+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T06:57:52.395646+0000","last_undegraded":"2024-06-06T06:57:52.395646+0000","last_fullsized":"2024-06-06T06:57:52.395646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394759+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T06:57:52.394759+0000","last_peered":"2024-06-06T06:57:52.394759+0000","last_clean":"2024-06-06T06:57:52.394759+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T06:57:52.394759+0000","last_undegraded":"2024-06-06T06:57:52.394759+0000","last_fullsized":"2024-06-06T06:57:52.394759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.423168+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T06:58:19.423168+0000","last_peered":"2024-06-06T06:58:19.423168+0000","last_clean":"2024-06-06T06:58:19.423168+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T06:58:19.423168+0000","last_undegraded":"2024-06-06T06:58:19.423168+0000","last_fullsized":"2024-06-06T06:58:19.423168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":372,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395007+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T06:57:52.395007+0000","last_peered":"2024-06-06T06:57:52.395007+0000","last_clean":"2024-06-06T06:57:52.395007+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T06:57:52.395007+0000","last_undegraded":"2024-06-06T06:57:52.395007+0000","last_fullsized":"2024-06-06T06:57:52.395007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398618+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T06:57:52.398618+0000","last_peered":"2024-06-06T06:57:52.398618+0000","last_clean":"2024-06-06T06:57:52.398618+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T06:57:52.398618+0000","last_undegraded":"2024-06-06T06:57:52.398618+0000","last_fullsized":"2024-06-06T06:57:52.398618+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430833+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T06:57:52.430833+0000","last_peered":"2024-06-06T06:57:52.430833+0000","last_clean":"2024-06-06T06:57:52.430833+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T06:57:52.430833+0000","last_undegraded":"2024-06-06T06:57:52.430833+0000","last_fullsized":"2024-06-06T06:57:52.430833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:16.408314+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T06:58:16.408314+0000","last_peered":"2024-06-06T06:58:16.408314+0000","last_clean":"2024-06-06T06:58:16.408314+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T06:58:16.408314+0000","last_undegraded":"2024-06-06T06:58:16.408314+0000","last_fullsized":"2024-06-06T06:58:16.408314+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398943+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T06:57:52.398943+0000","last_peered":"2024-06-06T06:57:52.398943+0000","last_clean":"2024-06-06T06:57:52.398943+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T06:57:52.398943+0000","last_undegraded":"2024-06-06T06:57:52.398943+0000","last_fullsized":"2024-06-06T06:57:52.398943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419295+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T06:58:18.419295+0000","last_peered":"2024-06-06T06:58:18.419295+0000","last_clean":"2024-06-06T06:58:18.419295+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T06:58:18.419295+0000","last_undegraded":"2024-06-06T06:58:18.419295+0000","last_fullsized":"2024-06-06T06:58:18.419295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431395+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T06:57:52.431395+0000","last_peered":"2024-06-06T06:57:52.431395+0000","last_clean":"2024-06-06T06:57:52.431395+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T06:57:52.431395+0000","last_undegraded":"2024-06-06T06:57:52.431395+0000","last_fullsized":"2024-06-06T06:57:52.431395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394696+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T06:57:52.394696+0000","last_peered":"2024-06-06T06:57:52.394696+0000","last_clean":"2024-06-06T06:57:52.394696+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T06:57:52.394696+0000","last_undegraded":"2024-06-06T06:57:52.394696+0000","last_fullsized":"2024-06-06T06:57:52.394696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'16","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:39.562182+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T06:58:39.562182+0000","last_peered":"2024-06-06T06:58:39.562182+0000","last_clean":"2024-06-06T06:58:39.562182+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T06:58:39.562182+0000","last_undegraded":"2024-06-06T06:58:39.562182+0000","last_fullsized":"2024-06-06T06:58:39.562182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395426+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T06:57:52.395426+0000","last_peered":"2024-06-06T06:57:52.395426+0000","last_clean":"2024-06-06T06:57:52.395426+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T06:57:52.395426+0000","last_undegraded":"2024-06-06T06:57:52.395426+0000","last_fullsized":"2024-06-06T06:57:52.395426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394888+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T06:57:52.394888+0000","last_peered":"2024-06-06T06:57:52.394888+0000","last_clean":"2024-06-06T06:57:52.394888+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T06:57:52.394888+0000","last_undegraded":"2024-06-06T06:57:52.394888+0000","last_fullsized":"2024-06-06T06:57:52.394888+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420133+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T06:58:18.420133+0000","last_peered":"2024-06-06T06:58:18.420133+0000","last_clean":"2024-06-06T06:58:18.420133+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T06:58:18.420133+0000","last_undegraded":"2024-06-06T06:58:18.420133+0000","last_fullsized":"2024-06-06T06:58:18.420133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430922+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T06:57:52.430922+0000","last_peered":"2024-06-06T06:57:52.430922+0000","last_clean":"2024-06-06T06:57:52.430922+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T06:57:52.430922+0000","last_undegraded":"2024-06-06T06:57:52.430922+0000","last_fullsized":"2024-06-06T06:57:52.430922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398340+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T06:57:52.398340+0000","last_peered":"2024-06-06T06:57:52.398340+0000","last_clean":"2024-06-06T06:57:52.398340+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T06:57:52.398340+0000","last_undegraded":"2024-06-06T06:57:52.398340+0000","last_fullsized":"2024-06-06T06:57:52.398340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:15.400917+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T06:58:15.400917+0000","last_peered":"2024-06-06T06:58:15.400917+0000","last_clean":"2024-06-06T06:58:15.400917+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T06:58:15.400917+0000","last_undegraded":"2024-06-06T06:58:15.400917+0000","last_fullsized":"2024-06-06T06:58:15.400917+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.403669+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T06:57:52.403669+0000","last_peered":"2024-06-06T06:57:52.403669+0000","last_clean":"2024-06-06T06:57:52.403669+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T06:57:52.403669+0000","last_undegraded":"2024-06-06T06:57:52.403669+0000","last_fullsized":"2024-06-06T06:57:52.403669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.405259+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T06:57:52.405259+0000","last_peered":"2024-06-06T06:57:52.405259+0000","last_clean":"2024-06-06T06:57:52.405259+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T06:57:52.405259+0000","last_undegraded":"2024-06-06T06:57:52.405259+0000","last_fullsized":"2024-06-06T06:57:52.405259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.418676+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T06:58:18.418676+0000","last_peered":"2024-06-06T06:58:18.418676+0000","last_clean":"2024-06-06T06:58:18.418676+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T06:58:18.418676+0000","last_undegraded":"2024-06-06T06:58:18.418676+0000","last_fullsized":"2024-06-06T06:58:18.418676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398694+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T06:57:52.398694+0000","last_peered":"2024-06-06T06:57:52.398694+0000","last_clean":"2024-06-06T06:57:52.398694+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T06:57:52.398694+0000","last_undegraded":"2024-06-06T06:57:52.398694+0000","last_fullsized":"2024-06-06T06:57:52.398694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398402+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T06:57:52.398402+0000","last_peered":"2024-06-06T06:57:52.398402+0000","last_clean":"2024-06-06T06:57:52.398402+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T06:57:52.398402+0000","last_undegraded":"2024-06-06T06:57:52.398402+0000","last_fullsized":"2024-06-06T06:57:52.398402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.414765+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T06:58:17.414765+0000","last_peered":"2024-06-06T06:58:17.414765+0000","last_clean":"2024-06-06T06:58:17.414765+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T06:58:17.414765+0000","last_undegraded":"2024-06-06T06:58:17.414765+0000","last_fullsized":"2024-06-06T06:58:17.414765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394965+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T06:57:52.394965+0000","last_peered":"2024-06-06T06:57:52.394965+0000","last_clean":"2024-06-06T06:57:52.394965+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T06:57:52.394965+0000","last_undegraded":"2024-06-06T06:57:52.394965+0000","last_fullsized":"2024-06-06T06:57:52.394965+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398265+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T06:57:52.398265+0000","last_peered":"2024-06-06T06:57:52.398265+0000","last_clean":"2024-06-06T06:57:52.398265+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T06:57:52.398265+0000","last_undegraded":"2024-06-06T06:57:52.398265+0000","last_fullsized":"2024-06-06T06:57:52.398265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:15.402494+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T06:58:15.402494+0000","last_peered":"2024-06-06T06:58:15.402494+0000","last_clean":"2024-06-06T06:58:15.402494+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T06:58:15.402494+0000","last_undegraded":"2024-06-06T06:58:15.402494+0000","last_fullsized":"2024-06-06T06:58:15.402494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395189+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T06:57:52.395189+0000","last_peered":"2024-06-06T06:57:52.395189+0000","last_clean":"2024-06-06T06:57:52.395189+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T06:57:52.395189+0000","last_undegraded":"2024-06-06T06:57:52.395189+0000","last_fullsized":"2024-06-06T06:57:52.395189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397308+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T06:57:52.397308+0000","last_peered":"2024-06-06T06:57:52.397308+0000","last_clean":"2024-06-06T06:57:52.397308+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T06:57:52.397308+0000","last_undegraded":"2024-06-06T06:57:52.397308+0000","last_fullsized":"2024-06-06T06:57:52.397308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":70,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:39.453850+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T06:58:39.453850+0000","last_peered":"2024-06-06T06:58:39.453850+0000","last_clean":"2024-06-06T06:58:39.453850+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T06:58:39.453850+0000","last_undegraded":"2024-06-06T06:58:39.453850+0000","last_fullsized":"2024-06-06T06:58:39.453850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395970+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T06:57:52.395970+0000","last_peered":"2024-06-06T06:57:52.395970+0000","last_clean":"2024-06-06T06:57:52.395970+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T06:57:52.395970+0000","last_undegraded":"2024-06-06T06:57:52.395970+0000","last_fullsized":"2024-06-06T06:57:52.395970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430186+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T06:57:52.430186+0000","last_peered":"2024-06-06T06:57:52.430186+0000","last_clean":"2024-06-06T06:57:52.430186+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T06:57:52.430186+0000","last_undegraded":"2024-06-06T06:57:52.430186+0000","last_fullsized":"2024-06-06T06:57:52.430186+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.418841+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T06:58:18.418841+0000","last_peered":"2024-06-06T06:58:18.418841+0000","last_clean":"2024-06-06T06:58:18.418841+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T06:58:18.418841+0000","last_undegraded":"2024-06-06T06:58:18.418841+0000","last_fullsized":"2024-06-06T06:58:18.418841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395590+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:57:52.395590+0000","last_peered":"2024-06-06T06:57:52.395590+0000","last_clean":"2024-06-06T06:57:52.395590+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T06:57:52.395590+0000","last_undegraded":"2024-06-06T06:57:52.395590+0000","last_fullsized":"2024-06-06T06:57:52.395590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397434+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T06:57:52.397434+0000","last_peered":"2024-06-06T06:57:52.397434+0000","last_clean":"2024-06-06T06:57:52.397434+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T06:57:52.397434+0000","last_undegraded":"2024-06-06T06:57:52.397434+0000","last_fullsized":"2024-06-06T06:57:52.397434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419506+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T06:58:18.419506+0000","last_peered":"2024-06-06T06:58:18.419506+0000","last_clean":"2024-06-06T06:58:18.419506+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T06:58:18.419506+0000","last_undegraded":"2024-06-06T06:58:18.419506+0000","last_fullsized":"2024-06-06T06:58:18.419506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396138+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T06:57:52.396138+0000","last_peered":"2024-06-06T06:57:52.396138+0000","last_clean":"2024-06-06T06:57:52.396138+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T06:57:52.396138+0000","last_undegraded":"2024-06-06T06:57:52.396138+0000","last_fullsized":"2024-06-06T06:57:52.396138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398805+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T06:57:52.398805+0000","last_peered":"2024-06-06T06:57:52.398805+0000","last_clean":"2024-06-06T06:57:52.398805+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T06:57:52.398805+0000","last_undegraded":"2024-06-06T06:57:52.398805+0000","last_fullsized":"2024-06-06T06:57:52.398805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395230+0000","last_change":"2024-06-06T06:57:07.558536+0000","last_active":"2024-06-06T06:57:52.395230+0000","last_peered":"2024-06-06T06:57:52.395230+0000","last_clean":"2024-06-06T06:57:52.395230+0000","last_became_active":"2024-06-06T06:57:07.558201+0000","last_became_peered":"2024-06-06T06:57:07.558201+0000","last_unstale":"2024-06-06T06:57:52.395230+0000","last_undegraded":"2024-06-06T06:57:52.395230+0000","last_fullsized":"2024-06-06T06:57:52.395230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:58:01.546923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394926+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T06:57:52.394926+0000","last_peered":"2024-06-06T06:57:52.394926+0000","last_clean":"2024-06-06T06:57:52.394926+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T06:57:52.394926+0000","last_undegraded":"2024-06-06T06:57:52.394926+0000","last_fullsized":"2024-06-06T06:57:52.394926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430470+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T06:57:52.430470+0000","last_peered":"2024-06-06T06:57:52.430470+0000","last_clean":"2024-06-06T06:57:52.430470+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T06:57:52.430470+0000","last_undegraded":"2024-06-06T06:57:52.430470+0000","last_fullsized":"2024-06-06T06:57:52.430470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431528+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T06:57:52.431528+0000","last_peered":"2024-06-06T06:57:52.431528+0000","last_clean":"2024-06-06T06:57:52.431528+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T06:57:52.431528+0000","last_undegraded":"2024-06-06T06:57:52.431528+0000","last_fullsized":"2024-06-06T06:57:52.431528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400274+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T06:57:52.400274+0000","last_peered":"2024-06-06T06:57:52.400274+0000","last_clean":"2024-06-06T06:57:52.400274+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T06:57:52.400274+0000","last_undegraded":"2024-06-06T06:57:52.400274+0000","last_fullsized":"2024-06-06T06:57:52.400274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395966+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T06:57:52.395966+0000","last_peered":"2024-06-06T06:57:52.395966+0000","last_clean":"2024-06-06T06:57:52.395966+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T06:57:52.395966+0000","last_undegraded":"2024-06-06T06:57:52.395966+0000","last_fullsized":"2024-06-06T06:57:52.395966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395220+0000","last_change":"2024-06-06T06:57:07.553546+0000","last_active":"2024-06-06T06:57:52.395220+0000","last_peered":"2024-06-06T06:57:52.395220+0000","last_clean":"2024-06-06T06:57:52.395220+0000","last_became_active":"2024-06-06T06:57:07.553173+0000","last_became_peered":"2024-06-06T06:57:07.553173+0000","last_unstale":"2024-06-06T06:57:52.395220+0000","last_undegraded":"2024-06-06T06:57:52.395220+0000","last_fullsized":"2024-06-06T06:57:52.395220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:50:43.222641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398182+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T06:57:52.398182+0000","last_peered":"2024-06-06T06:57:52.398182+0000","last_clean":"2024-06-06T06:57:52.398182+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T06:57:52.398182+0000","last_undegraded":"2024-06-06T06:57:52.398182+0000","last_fullsized":"2024-06-06T06:57:52.398182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394863+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T06:57:52.394863+0000","last_peered":"2024-06-06T06:57:52.394863+0000","last_clean":"2024-06-06T06:57:52.394863+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T06:57:52.394863+0000","last_undegraded":"2024-06-06T06:57:52.394863+0000","last_fullsized":"2024-06-06T06:57:52.394863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397623+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T06:57:52.397623+0000","last_peered":"2024-06-06T06:57:52.397623+0000","last_clean":"2024-06-06T06:57:52.397623+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:57:52.397623+0000","last_undegraded":"2024-06-06T06:57:52.397623+0000","last_fullsized":"2024-06-06T06:57:52.397623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431182+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T06:57:52.431182+0000","last_peered":"2024-06-06T06:57:52.431182+0000","last_clean":"2024-06-06T06:57:52.431182+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T06:57:52.431182+0000","last_undegraded":"2024-06-06T06:57:52.431182+0000","last_fullsized":"2024-06-06T06:57:52.431182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398480+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T06:57:52.398480+0000","last_peered":"2024-06-06T06:57:52.398480+0000","last_clean":"2024-06-06T06:57:52.398480+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T06:57:52.398480+0000","last_undegraded":"2024-06-06T06:57:52.398480+0000","last_fullsized":"2024-06-06T06:57:52.398480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285685,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70127,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":285,"num_read_kb":223,"num_write":250,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":197875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":555058,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1236992,"data_stored":1204799,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":558694,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037810,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":554976,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299447,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1151472,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462972,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":550102,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":607804,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":551948,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36468,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708684,"statfs":{"total":95995035648,"available":95957692416,"internally_reserved":0,"allocated":1159168,"data_stored":607792,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149770,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":549810,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":555188,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1146664,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":58412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T06:58:46.477 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T06:58:46.721 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:46.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:46 smithi073 ceph-mon[32303]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-06-06T06:58:46.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:46 smithi136 ceph-mon[35901]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-06-06T06:58:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:46 smithi170 ceph-mon[36108]: pgmap v355: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.6 KiB/s wr, 13 op/s 2024-06-06T06:58:47.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:47 smithi073 ceph-mon[32303]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:47 smithi136 ceph-mon[35901]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:47 smithi170 ceph-mon[36108]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:48.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 -- 172.21.15.73:0/3670906596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 msgr2=0x7f16581088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:48.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 --2- 172.21.15.73:0/3670906596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f16581088d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1648009990 tx=0x7f1648009ca0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 -- 172.21.15.73:0/3670906596 shutdown_connections 2024-06-06T06:58:48.539 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 --2- 172.21.15.73:0/3670906596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 0x7f165810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 --2- 172.21.15.73:0/3670906596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f16581088d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 --2- 172.21.15.73:0/3670906596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f16581029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 -- 172.21.15.73:0/3670906596 >> 172.21.15.73:0/3670906596 conn(0x7f1658079650 msgr2=0x7f1658079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 -- 172.21.15.73:0/3670906596 shutdown_connections 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.535+0000 7f165e9a9700 1 -- 172.21.15.73:0/3670906596 wait complete. 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 Processor -- start 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 -- start start 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f1658074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 0x7f1658074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1658112680 con 0x7f16581084f0 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1658112500 con 0x7f1658102f10 2024-06-06T06:58:48.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165e9a9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1658112800 con 0x7f1658102550 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165cf46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165cf46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38144/0 (socket says 172.21.15.73:38144) 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f165cf46700 1 -- 172.21.15.73:0/2619198410 learned_addr learned my addr 172.21.15.73:0/2619198410 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.536+0000 7f164ffff700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 0x7f1658074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f1657fff700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f1658074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 -- 172.21.15.73:0/2619198410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 msgr2=0x7f1658074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f1658074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 -- 172.21.15.73:0/2619198410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 msgr2=0x7f1658074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 0x7f1658074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 -- 172.21.15.73:0/2619198410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16581a70e0 con 0x7f16581084f0 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f1657fff700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f1658074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:48.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165cf46700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1650007ee0 tx=0x7f165000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:48.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165000dbd0 con 0x7f16581084f0 2024-06-06T06:58:48.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f165000dd70 con 0x7f16581084f0 2024-06-06T06:58:48.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16581a73d0 con 0x7f16581084f0 2024-06-06T06:58:48.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1650005a20 con 0x7f16581084f0 2024-06-06T06:58:48.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.537+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16581a78c0 con 0x7f16581084f0 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.538+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1658103ba0 con 0x7f16581084f0 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.539+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f165001a020 con 0x7f16581084f0 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.540+0000 7f1655ffb700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 0x7f16440a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.540+0000 7f1657fff700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 0x7f16440a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.540+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f16500cf390 con 0x7f16581084f0 2024-06-06T06:58:48.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.540+0000 7f1657fff700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 0x7f16440a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164800b5c0 tx=0x7f1648005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:48.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.543+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1650010070 con 0x7f16581084f0 2024-06-06T06:58:48.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.735+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f16580628d0 con 0x7f16440a2cb0 2024-06-06T06:58:48.737 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:48 smithi073 ceph-mon[32303]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-06T06:58:48.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.745+0000 7f1655ffb700 1 -- 172.21.15.73:0/2619198410 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340679 (secure 0 0 0) 0x7f16580628d0 con 0x7f16440a2cb0 2024-06-06T06:58:48.751 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 msgr2=0x7f16440a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 0x7f16440a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f164800b5c0 tx=0x7f1648005fb0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 msgr2=0x7f1658075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f1650007ee0 tx=0x7f165000d3b0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 shutdown_connections 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f16440a2cb0 0x7f16440a5170 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1658102550 0x7f1658074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16581084f0 0x7f1658075210 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 --2- 172.21.15.73:0/2619198410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1658102f10 0x7f1658074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 >> 172.21.15.73:0/2619198410 conn(0x7f1658079650 msgr2=0x7f1658100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 shutdown_connections 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:48.748+0000 7f165e9a9700 1 -- 172.21.15.73:0/2619198410 wait complete. 2024-06-06T06:58:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T06:58:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:48 smithi136 ceph-mon[35901]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-06T06:58:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:48 smithi170 ceph-mon[36108]: pgmap v356: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-06T06:58:49.378 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":356,"stamp":"2024-06-06T06:58:47.321148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660729,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":441,"num_read_kb":347,"num_write":433,"num_write_kb":3082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":354,"ondisk_log_size":354,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":383212,"kb_used_data":13260,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124558612,"statfs":{"total":1151940427776,"available":1151548018688,"internally_reserved":0,"allocated":13578240,"data_stored":8600560,"data_compressed":424821,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19142,"internal_metadata":378713402},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":10635,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":122,"num_read_kb":67,"num_write":13,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003508"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399432+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T06:57:52.399432+0000","last_peered":"2024-06-06T06:57:52.399432+0000","last_clean":"2024-06-06T06:57:52.399432+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T06:57:52.399432+0000","last_undegraded":"2024-06-06T06:57:52.399432+0000","last_fullsized":"2024-06-06T06:57:52.399432+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430831+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T06:57:52.430831+0000","last_peered":"2024-06-06T06:57:52.430831+0000","last_clean":"2024-06-06T06:57:52.430831+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T06:57:52.430831+0000","last_undegraded":"2024-06-06T06:57:52.430831+0000","last_fullsized":"2024-06-06T06:57:52.430831+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430720+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T06:57:52.430720+0000","last_peered":"2024-06-06T06:57:52.430720+0000","last_clean":"2024-06-06T06:57:52.430720+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T06:57:52.430720+0000","last_undegraded":"2024-06-06T06:57:52.430720+0000","last_fullsized":"2024-06-06T06:57:52.430720+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399516+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T06:57:52.399516+0000","last_peered":"2024-06-06T06:57:52.399516+0000","last_clean":"2024-06-06T06:57:52.399516+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T06:57:52.399516+0000","last_undegraded":"2024-06-06T06:57:52.399516+0000","last_fullsized":"2024-06-06T06:57:52.399516+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404121+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T06:57:52.404121+0000","last_peered":"2024-06-06T06:57:52.404121+0000","last_clean":"2024-06-06T06:57:52.404121+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T06:57:52.404121+0000","last_undegraded":"2024-06-06T06:57:52.404121+0000","last_fullsized":"2024-06-06T06:57:52.404121+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.405561+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T06:57:52.405561+0000","last_peered":"2024-06-06T06:57:52.405561+0000","last_clean":"2024-06-06T06:57:52.405561+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T06:57:52.405561+0000","last_undegraded":"2024-06-06T06:57:52.405561+0000","last_fullsized":"2024-06-06T06:57:52.405561+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395075+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T06:57:52.395075+0000","last_peered":"2024-06-06T06:57:52.395075+0000","last_clean":"2024-06-06T06:57:52.395075+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T06:57:52.395075+0000","last_undegraded":"2024-06-06T06:57:52.395075+0000","last_fullsized":"2024-06-06T06:57:52.395075+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394689+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T06:57:52.394689+0000","last_peered":"2024-06-06T06:57:52.394689+0000","last_clean":"2024-06-06T06:57:52.394689+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T06:57:52.394689+0000","last_undegraded":"2024-06-06T06:57:52.394689+0000","last_fullsized":"2024-06-06T06:57:52.394689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398135+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T06:57:52.398135+0000","last_peered":"2024-06-06T06:57:52.398135+0000","last_clean":"2024-06-06T06:57:52.398135+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T06:57:52.398135+0000","last_undegraded":"2024-06-06T06:57:52.398135+0000","last_fullsized":"2024-06-06T06:57:52.398135+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.401359+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T06:57:52.401359+0000","last_peered":"2024-06-06T06:57:52.401359+0000","last_clean":"2024-06-06T06:57:52.401359+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T06:57:52.401359+0000","last_undegraded":"2024-06-06T06:57:52.401359+0000","last_fullsized":"2024-06-06T06:57:52.401359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395761+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T06:57:52.395761+0000","last_peered":"2024-06-06T06:57:52.395761+0000","last_clean":"2024-06-06T06:57:52.395761+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T06:57:52.395761+0000","last_undegraded":"2024-06-06T06:57:52.395761+0000","last_fullsized":"2024-06-06T06:57:52.395761+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399025+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T06:57:52.399025+0000","last_peered":"2024-06-06T06:57:52.399025+0000","last_clean":"2024-06-06T06:57:52.399025+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T06:57:52.399025+0000","last_undegraded":"2024-06-06T06:57:52.399025+0000","last_fullsized":"2024-06-06T06:57:52.399025+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431263+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T06:57:52.431263+0000","last_peered":"2024-06-06T06:57:52.431263+0000","last_clean":"2024-06-06T06:57:52.431263+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T06:57:52.431263+0000","last_undegraded":"2024-06-06T06:57:52.431263+0000","last_fullsized":"2024-06-06T06:57:52.431263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430192+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T06:57:52.430192+0000","last_peered":"2024-06-06T06:57:52.430192+0000","last_clean":"2024-06-06T06:57:52.430192+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T06:57:52.430192+0000","last_undegraded":"2024-06-06T06:57:52.430192+0000","last_fullsized":"2024-06-06T06:57:52.430192+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430327+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T06:57:52.430327+0000","last_peered":"2024-06-06T06:57:52.430327+0000","last_clean":"2024-06-06T06:57:52.430327+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T06:57:52.430327+0000","last_undegraded":"2024-06-06T06:57:52.430327+0000","last_fullsized":"2024-06-06T06:57:52.430327+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397739+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T06:57:52.397739+0000","last_peered":"2024-06-06T06:57:52.397739+0000","last_clean":"2024-06-06T06:57:52.397739+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T06:57:52.397739+0000","last_undegraded":"2024-06-06T06:57:52.397739+0000","last_fullsized":"2024-06-06T06:57:52.397739+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399182+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T06:57:52.399182+0000","last_peered":"2024-06-06T06:57:52.399182+0000","last_clean":"2024-06-06T06:57:52.399182+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T06:57:52.399182+0000","last_undegraded":"2024-06-06T06:57:52.399182+0000","last_fullsized":"2024-06-06T06:57:52.399182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397247+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T06:57:52.397247+0000","last_peered":"2024-06-06T06:57:52.397247+0000","last_clean":"2024-06-06T06:57:52.397247+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T06:57:52.397247+0000","last_undegraded":"2024-06-06T06:57:52.397247+0000","last_fullsized":"2024-06-06T06:57:52.397247+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396069+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T06:57:52.396069+0000","last_peered":"2024-06-06T06:57:52.396069+0000","last_clean":"2024-06-06T06:57:52.396069+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T06:57:52.396069+0000","last_undegraded":"2024-06-06T06:57:52.396069+0000","last_fullsized":"2024-06-06T06:57:52.396069+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394994+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T06:57:52.394994+0000","last_peered":"2024-06-06T06:57:52.394994+0000","last_clean":"2024-06-06T06:57:52.394994+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T06:57:52.394994+0000","last_undegraded":"2024-06-06T06:57:52.394994+0000","last_fullsized":"2024-06-06T06:57:52.394994+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":79,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:42.456521+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T06:58:42.456521+0000","last_peered":"2024-06-06T06:58:42.456521+0000","last_clean":"2024-06-06T06:58:42.456521+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T06:58:42.456521+0000","last_undegraded":"2024-06-06T06:58:42.456521+0000","last_fullsized":"2024-06-06T06:58:42.456521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395032+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T06:57:52.395032+0000","last_peered":"2024-06-06T06:57:52.395032+0000","last_clean":"2024-06-06T06:57:52.395032+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T06:57:52.395032+0000","last_undegraded":"2024-06-06T06:57:52.395032+0000","last_fullsized":"2024-06-06T06:57:52.395032+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397919+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T06:57:52.397919+0000","last_peered":"2024-06-06T06:57:52.397919+0000","last_clean":"2024-06-06T06:57:52.397919+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T06:57:52.397919+0000","last_undegraded":"2024-06-06T06:57:52.397919+0000","last_fullsized":"2024-06-06T06:57:52.397919+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395414+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T06:57:52.395414+0000","last_peered":"2024-06-06T06:57:52.395414+0000","last_clean":"2024-06-06T06:57:52.395414+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T06:57:52.395414+0000","last_undegraded":"2024-06-06T06:57:52.395414+0000","last_fullsized":"2024-06-06T06:57:52.395414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430711+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T06:57:52.430711+0000","last_peered":"2024-06-06T06:57:52.430711+0000","last_clean":"2024-06-06T06:57:52.430711+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T06:57:52.430711+0000","last_undegraded":"2024-06-06T06:57:52.430711+0000","last_fullsized":"2024-06-06T06:57:52.430711+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.406751+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T06:57:52.406751+0000","last_peered":"2024-06-06T06:57:52.406751+0000","last_clean":"2024-06-06T06:57:52.406751+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T06:57:52.406751+0000","last_undegraded":"2024-06-06T06:57:52.406751+0000","last_fullsized":"2024-06-06T06:57:52.406751+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":26,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.422789+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T06:58:19.422789+0000","last_peered":"2024-06-06T06:58:19.422789+0000","last_clean":"2024-06-06T06:58:19.422789+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T06:58:19.422789+0000","last_undegraded":"2024-06-06T06:58:19.422789+0000","last_fullsized":"2024-06-06T06:58:19.422789+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399174+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T06:57:52.399174+0000","last_peered":"2024-06-06T06:57:52.399174+0000","last_clean":"2024-06-06T06:57:52.399174+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T06:57:52.399174+0000","last_undegraded":"2024-06-06T06:57:52.399174+0000","last_fullsized":"2024-06-06T06:57:52.399174+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397583+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T06:57:52.397583+0000","last_peered":"2024-06-06T06:57:52.397583+0000","last_clean":"2024-06-06T06:57:52.397583+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T06:57:52.397583+0000","last_undegraded":"2024-06-06T06:57:52.397583+0000","last_fullsized":"2024-06-06T06:57:52.397583+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":46,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420196+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T06:58:18.420196+0000","last_peered":"2024-06-06T06:58:18.420196+0000","last_clean":"2024-06-06T06:58:18.420196+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T06:58:18.420196+0000","last_undegraded":"2024-06-06T06:58:18.420196+0000","last_fullsized":"2024-06-06T06:58:18.420196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399333+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T06:57:52.399333+0000","last_peered":"2024-06-06T06:57:52.399333+0000","last_clean":"2024-06-06T06:57:52.399333+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T06:57:52.399333+0000","last_undegraded":"2024-06-06T06:57:52.399333+0000","last_fullsized":"2024-06-06T06:57:52.399333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395563+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T06:57:52.395563+0000","last_peered":"2024-06-06T06:57:52.395563+0000","last_clean":"2024-06-06T06:57:52.395563+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T06:57:52.395563+0000","last_undegraded":"2024-06-06T06:57:52.395563+0000","last_fullsized":"2024-06-06T06:57:52.395563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":78,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:43.459556+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T06:58:43.459556+0000","last_peered":"2024-06-06T06:58:43.459556+0000","last_clean":"2024-06-06T06:58:43.459556+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T06:58:43.459556+0000","last_undegraded":"2024-06-06T06:58:43.459556+0000","last_fullsized":"2024-06-06T06:58:43.459556+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404268+0000","last_change":"2024-06-06T06:57:07.549415+0000","last_active":"2024-06-06T06:57:52.404268+0000","last_peered":"2024-06-06T06:57:52.404268+0000","last_clean":"2024-06-06T06:57:52.404268+0000","last_became_active":"2024-06-06T06:57:07.549069+0000","last_became_peered":"2024-06-06T06:57:07.549069+0000","last_unstale":"2024-06-06T06:57:52.404268+0000","last_undegraded":"2024-06-06T06:57:52.404268+0000","last_fullsized":"2024-06-06T06:57:52.404268+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:50:44.047896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.432492+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T06:57:52.432492+0000","last_peered":"2024-06-06T06:57:52.432492+0000","last_clean":"2024-06-06T06:57:52.432492+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T06:57:52.432492+0000","last_undegraded":"2024-06-06T06:57:52.432492+0000","last_fullsized":"2024-06-06T06:57:52.432492+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.415153+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T06:58:17.415153+0000","last_peered":"2024-06-06T06:58:17.415153+0000","last_clean":"2024-06-06T06:58:17.415153+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T06:58:17.415153+0000","last_undegraded":"2024-06-06T06:58:17.415153+0000","last_fullsized":"2024-06-06T06:58:17.415153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398951+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T06:57:52.398951+0000","last_peered":"2024-06-06T06:57:52.398951+0000","last_clean":"2024-06-06T06:57:52.398951+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T06:57:52.398951+0000","last_undegraded":"2024-06-06T06:57:52.398951+0000","last_fullsized":"2024-06-06T06:57:52.398951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399074+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T06:57:52.399074+0000","last_peered":"2024-06-06T06:57:52.399074+0000","last_clean":"2024-06-06T06:57:52.399074+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T06:57:52.399074+0000","last_undegraded":"2024-06-06T06:57:52.399074+0000","last_fullsized":"2024-06-06T06:57:52.399074+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:16.408627+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T06:58:16.408627+0000","last_peered":"2024-06-06T06:58:16.408627+0000","last_clean":"2024-06-06T06:58:16.408627+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T06:58:16.408627+0000","last_undegraded":"2024-06-06T06:58:16.408627+0000","last_fullsized":"2024-06-06T06:58:16.408627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394818+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T06:57:52.394818+0000","last_peered":"2024-06-06T06:57:52.394818+0000","last_clean":"2024-06-06T06:57:52.394818+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T06:57:52.394818+0000","last_undegraded":"2024-06-06T06:57:52.394818+0000","last_fullsized":"2024-06-06T06:57:52.394818+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430739+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T06:57:52.430739+0000","last_peered":"2024-06-06T06:57:52.430739+0000","last_clean":"2024-06-06T06:57:52.430739+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T06:57:52.430739+0000","last_undegraded":"2024-06-06T06:57:52.430739+0000","last_fullsized":"2024-06-06T06:57:52.430739+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420058+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T06:58:18.420058+0000","last_peered":"2024-06-06T06:58:18.420058+0000","last_clean":"2024-06-06T06:58:18.420058+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T06:58:18.420058+0000","last_undegraded":"2024-06-06T06:58:18.420058+0000","last_fullsized":"2024-06-06T06:58:18.420058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396538+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T06:57:52.396538+0000","last_peered":"2024-06-06T06:57:52.396538+0000","last_clean":"2024-06-06T06:57:52.396538+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T06:57:52.396538+0000","last_undegraded":"2024-06-06T06:57:52.396538+0000","last_fullsized":"2024-06-06T06:57:52.396538+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394734+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T06:57:52.394734+0000","last_peered":"2024-06-06T06:57:52.394734+0000","last_clean":"2024-06-06T06:57:52.394734+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T06:57:52.394734+0000","last_undegraded":"2024-06-06T06:57:52.394734+0000","last_fullsized":"2024-06-06T06:57:52.394734+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"98'4","reported_seq":96,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.460290+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T06:58:44.460290+0000","last_peered":"2024-06-06T06:58:44.460290+0000","last_clean":"2024-06-06T06:58:44.460290+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T06:58:44.460290+0000","last_undegraded":"2024-06-06T06:58:44.460290+0000","last_fullsized":"2024-06-06T06:58:44.460290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398571+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T06:57:52.398571+0000","last_peered":"2024-06-06T06:57:52.398571+0000","last_clean":"2024-06-06T06:57:52.398571+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:57:52.398571+0000","last_undegraded":"2024-06-06T06:57:52.398571+0000","last_fullsized":"2024-06-06T06:57:52.398571+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396489+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T06:57:52.396489+0000","last_peered":"2024-06-06T06:57:52.396489+0000","last_clean":"2024-06-06T06:57:52.396489+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T06:57:52.396489+0000","last_undegraded":"2024-06-06T06:57:52.396489+0000","last_fullsized":"2024-06-06T06:57:52.396489+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420037+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T06:58:18.420037+0000","last_peered":"2024-06-06T06:58:18.420037+0000","last_clean":"2024-06-06T06:58:18.420037+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T06:58:18.420037+0000","last_undegraded":"2024-06-06T06:58:18.420037+0000","last_fullsized":"2024-06-06T06:58:18.420037+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430942+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T06:57:52.430942+0000","last_peered":"2024-06-06T06:57:52.430942+0000","last_clean":"2024-06-06T06:57:52.430942+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T06:57:52.430942+0000","last_undegraded":"2024-06-06T06:57:52.430942+0000","last_fullsized":"2024-06-06T06:57:52.430942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404541+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T06:57:52.404541+0000","last_peered":"2024-06-06T06:57:52.404541+0000","last_clean":"2024-06-06T06:57:52.404541+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T06:57:52.404541+0000","last_undegraded":"2024-06-06T06:57:52.404541+0000","last_fullsized":"2024-06-06T06:57:52.404541+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.414532+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T06:58:17.414532+0000","last_peered":"2024-06-06T06:58:17.414532+0000","last_clean":"2024-06-06T06:58:17.414532+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T06:58:17.414532+0000","last_undegraded":"2024-06-06T06:58:17.414532+0000","last_fullsized":"2024-06-06T06:58:17.414532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398894+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T06:57:52.398894+0000","last_peered":"2024-06-06T06:57:52.398894+0000","last_clean":"2024-06-06T06:57:52.398894+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T06:57:52.398894+0000","last_undegraded":"2024-06-06T06:57:52.398894+0000","last_fullsized":"2024-06-06T06:57:52.398894+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396704+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T06:57:52.396704+0000","last_peered":"2024-06-06T06:57:52.396704+0000","last_clean":"2024-06-06T06:57:52.396704+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T06:57:52.396704+0000","last_undegraded":"2024-06-06T06:57:52.396704+0000","last_fullsized":"2024-06-06T06:57:52.396704+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"98'35","reported_seq":136,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.464729+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T06:58:44.464729+0000","last_peered":"2024-06-06T06:58:44.464729+0000","last_clean":"2024-06-06T06:58:44.464729+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T06:58:44.464729+0000","last_undegraded":"2024-06-06T06:58:44.464729+0000","last_fullsized":"2024-06-06T06:58:44.464729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62925,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":92,"num_write":37,"num_write_kb":76,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.404658+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T06:57:52.404658+0000","last_peered":"2024-06-06T06:57:52.404658+0000","last_clean":"2024-06-06T06:57:52.404658+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T06:57:52.404658+0000","last_undegraded":"2024-06-06T06:57:52.404658+0000","last_fullsized":"2024-06-06T06:57:52.404658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.399635+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T06:57:52.399635+0000","last_peered":"2024-06-06T06:57:52.399635+0000","last_clean":"2024-06-06T06:57:52.399635+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T06:57:52.399635+0000","last_undegraded":"2024-06-06T06:57:52.399635+0000","last_fullsized":"2024-06-06T06:57:52.399635+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":50,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.423045+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T06:58:19.423045+0000","last_peered":"2024-06-06T06:58:19.423045+0000","last_clean":"2024-06-06T06:58:19.423045+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T06:58:19.423045+0000","last_undegraded":"2024-06-06T06:58:19.423045+0000","last_fullsized":"2024-06-06T06:58:19.423045+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431059+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T06:57:52.431059+0000","last_peered":"2024-06-06T06:57:52.431059+0000","last_clean":"2024-06-06T06:57:52.431059+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T06:57:52.431059+0000","last_undegraded":"2024-06-06T06:57:52.431059+0000","last_fullsized":"2024-06-06T06:57:52.431059+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398283+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T06:57:52.398283+0000","last_peered":"2024-06-06T06:57:52.398283+0000","last_clean":"2024-06-06T06:57:52.398283+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T06:57:52.398283+0000","last_undegraded":"2024-06-06T06:57:52.398283+0000","last_fullsized":"2024-06-06T06:57:52.398283+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419019+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T06:58:18.419019+0000","last_peered":"2024-06-06T06:58:18.419019+0000","last_clean":"2024-06-06T06:58:18.419019+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T06:58:18.419019+0000","last_undegraded":"2024-06-06T06:58:18.419019+0000","last_fullsized":"2024-06-06T06:58:18.419019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395770+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T06:57:52.395770+0000","last_peered":"2024-06-06T06:57:52.395770+0000","last_clean":"2024-06-06T06:57:52.395770+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T06:57:52.395770+0000","last_undegraded":"2024-06-06T06:57:52.395770+0000","last_fullsized":"2024-06-06T06:57:52.395770+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397189+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T06:57:52.397189+0000","last_peered":"2024-06-06T06:57:52.397189+0000","last_clean":"2024-06-06T06:57:52.397189+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T06:57:52.397189+0000","last_undegraded":"2024-06-06T06:57:52.397189+0000","last_fullsized":"2024-06-06T06:57:52.397189+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"98'6","reported_seq":75,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.460265+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T06:58:44.460265+0000","last_peered":"2024-06-06T06:58:44.460265+0000","last_clean":"2024-06-06T06:58:44.460265+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T06:58:44.460265+0000","last_undegraded":"2024-06-06T06:58:44.460265+0000","last_fullsized":"2024-06-06T06:58:44.460265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":978,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395086+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T06:57:52.395086+0000","last_peered":"2024-06-06T06:57:52.395086+0000","last_clean":"2024-06-06T06:57:52.395086+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T06:57:52.395086+0000","last_undegraded":"2024-06-06T06:57:52.395086+0000","last_fullsized":"2024-06-06T06:57:52.395086+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431540+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T06:57:52.431540+0000","last_peered":"2024-06-06T06:57:52.431540+0000","last_clean":"2024-06-06T06:57:52.431540+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T06:57:52.431540+0000","last_undegraded":"2024-06-06T06:57:52.431540+0000","last_fullsized":"2024-06-06T06:57:52.431540+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":58,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419689+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T06:58:18.419689+0000","last_peered":"2024-06-06T06:58:18.419689+0000","last_clean":"2024-06-06T06:58:18.419689+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T06:58:18.419689+0000","last_undegraded":"2024-06-06T06:58:18.419689+0000","last_fullsized":"2024-06-06T06:58:18.419689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398796+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T06:57:52.398796+0000","last_peered":"2024-06-06T06:57:52.398796+0000","last_clean":"2024-06-06T06:57:52.398796+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T06:57:52.398796+0000","last_undegraded":"2024-06-06T06:57:52.398796+0000","last_fullsized":"2024-06-06T06:57:52.398796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398409+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T06:57:52.398409+0000","last_peered":"2024-06-06T06:57:52.398409+0000","last_clean":"2024-06-06T06:57:52.398409+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T06:57:52.398409+0000","last_undegraded":"2024-06-06T06:57:52.398409+0000","last_fullsized":"2024-06-06T06:57:52.398409+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.403648+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T06:58:17.403648+0000","last_peered":"2024-06-06T06:58:17.403648+0000","last_clean":"2024-06-06T06:58:17.403648+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T06:58:17.403648+0000","last_undegraded":"2024-06-06T06:58:17.403648+0000","last_fullsized":"2024-06-06T06:58:17.403648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":40,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.432469+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T06:57:52.432469+0000","last_peered":"2024-06-06T06:57:52.432469+0000","last_clean":"2024-06-06T06:57:52.432469+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T06:57:52.432469+0000","last_undegraded":"2024-06-06T06:57:52.432469+0000","last_fullsized":"2024-06-06T06:57:52.432469+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.403985+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T06:57:52.403985+0000","last_peered":"2024-06-06T06:57:52.403985+0000","last_clean":"2024-06-06T06:57:52.403985+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T06:57:52.403985+0000","last_undegraded":"2024-06-06T06:57:52.403985+0000","last_fullsized":"2024-06-06T06:57:52.403985+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":41,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396937+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T06:57:52.396937+0000","last_peered":"2024-06-06T06:57:52.396937+0000","last_clean":"2024-06-06T06:57:52.396937+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T06:57:52.396937+0000","last_undegraded":"2024-06-06T06:57:52.396937+0000","last_fullsized":"2024-06-06T06:57:52.396937+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400758+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:57:52.400758+0000","last_peered":"2024-06-06T06:57:52.400758+0000","last_clean":"2024-06-06T06:57:52.400758+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T06:57:52.400758+0000","last_undegraded":"2024-06-06T06:57:52.400758+0000","last_fullsized":"2024-06-06T06:57:52.400758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431248+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T06:57:52.431248+0000","last_peered":"2024-06-06T06:57:52.431248+0000","last_clean":"2024-06-06T06:57:52.431248+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T06:57:52.431248+0000","last_undegraded":"2024-06-06T06:57:52.431248+0000","last_fullsized":"2024-06-06T06:57:52.431248+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'5","reported_seq":73,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:45.461547+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T06:58:45.461547+0000","last_peered":"2024-06-06T06:58:45.461547+0000","last_clean":"2024-06-06T06:58:45.461547+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T06:58:45.461547+0000","last_undegraded":"2024-06-06T06:58:45.461547+0000","last_fullsized":"2024-06-06T06:58:45.461547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400210+0000","last_change":"2024-06-06T06:57:07.551478+0000","last_active":"2024-06-06T06:57:52.400210+0000","last_peered":"2024-06-06T06:57:52.400210+0000","last_clean":"2024-06-06T06:57:52.400210+0000","last_became_active":"2024-06-06T06:57:07.550554+0000","last_became_peered":"2024-06-06T06:57:07.550554+0000","last_unstale":"2024-06-06T06:57:52.400210+0000","last_undegraded":"2024-06-06T06:57:52.400210+0000","last_fullsized":"2024-06-06T06:57:52.400210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:30:05.500978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431369+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T06:57:52.431369+0000","last_peered":"2024-06-06T06:57:52.431369+0000","last_clean":"2024-06-06T06:57:52.431369+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T06:57:52.431369+0000","last_undegraded":"2024-06-06T06:57:52.431369+0000","last_fullsized":"2024-06-06T06:57:52.431369+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398322+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T06:57:52.398322+0000","last_peered":"2024-06-06T06:57:52.398322+0000","last_clean":"2024-06-06T06:57:52.398322+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T06:57:52.398322+0000","last_undegraded":"2024-06-06T06:57:52.398322+0000","last_fullsized":"2024-06-06T06:57:52.398322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"98'5","reported_seq":73,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.459589+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T06:58:44.459589+0000","last_peered":"2024-06-06T06:58:44.459589+0000","last_clean":"2024-06-06T06:58:44.459589+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T06:58:44.459589+0000","last_undegraded":"2024-06-06T06:58:44.459589+0000","last_fullsized":"2024-06-06T06:58:44.459589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":28,"num_read_kb":28,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395646+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T06:57:52.395646+0000","last_peered":"2024-06-06T06:57:52.395646+0000","last_clean":"2024-06-06T06:57:52.395646+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T06:57:52.395646+0000","last_undegraded":"2024-06-06T06:57:52.395646+0000","last_fullsized":"2024-06-06T06:57:52.395646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394759+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T06:57:52.394759+0000","last_peered":"2024-06-06T06:57:52.394759+0000","last_clean":"2024-06-06T06:57:52.394759+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T06:57:52.394759+0000","last_undegraded":"2024-06-06T06:57:52.394759+0000","last_fullsized":"2024-06-06T06:57:52.394759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:19.423168+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T06:58:19.423168+0000","last_peered":"2024-06-06T06:58:19.423168+0000","last_clean":"2024-06-06T06:58:19.423168+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T06:58:19.423168+0000","last_undegraded":"2024-06-06T06:58:19.423168+0000","last_fullsized":"2024-06-06T06:58:19.423168+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":372,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395007+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T06:57:52.395007+0000","last_peered":"2024-06-06T06:57:52.395007+0000","last_clean":"2024-06-06T06:57:52.395007+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T06:57:52.395007+0000","last_undegraded":"2024-06-06T06:57:52.395007+0000","last_fullsized":"2024-06-06T06:57:52.395007+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398618+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T06:57:52.398618+0000","last_peered":"2024-06-06T06:57:52.398618+0000","last_clean":"2024-06-06T06:57:52.398618+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T06:57:52.398618+0000","last_undegraded":"2024-06-06T06:57:52.398618+0000","last_fullsized":"2024-06-06T06:57:52.398618+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430833+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T06:57:52.430833+0000","last_peered":"2024-06-06T06:57:52.430833+0000","last_clean":"2024-06-06T06:57:52.430833+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T06:57:52.430833+0000","last_undegraded":"2024-06-06T06:57:52.430833+0000","last_fullsized":"2024-06-06T06:57:52.430833+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":49,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:16.408314+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T06:58:16.408314+0000","last_peered":"2024-06-06T06:58:16.408314+0000","last_clean":"2024-06-06T06:58:16.408314+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T06:58:16.408314+0000","last_undegraded":"2024-06-06T06:58:16.408314+0000","last_fullsized":"2024-06-06T06:58:16.408314+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398943+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T06:57:52.398943+0000","last_peered":"2024-06-06T06:57:52.398943+0000","last_clean":"2024-06-06T06:57:52.398943+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T06:57:52.398943+0000","last_undegraded":"2024-06-06T06:57:52.398943+0000","last_fullsized":"2024-06-06T06:57:52.398943+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419295+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T06:58:18.419295+0000","last_peered":"2024-06-06T06:58:18.419295+0000","last_clean":"2024-06-06T06:58:18.419295+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T06:58:18.419295+0000","last_undegraded":"2024-06-06T06:58:18.419295+0000","last_fullsized":"2024-06-06T06:58:18.419295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431395+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T06:57:52.431395+0000","last_peered":"2024-06-06T06:57:52.431395+0000","last_clean":"2024-06-06T06:57:52.431395+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T06:57:52.431395+0000","last_undegraded":"2024-06-06T06:57:52.431395+0000","last_fullsized":"2024-06-06T06:57:52.431395+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394696+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T06:57:52.394696+0000","last_peered":"2024-06-06T06:57:52.394696+0000","last_clean":"2024-06-06T06:57:52.394696+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T06:57:52.394696+0000","last_undegraded":"2024-06-06T06:57:52.394696+0000","last_fullsized":"2024-06-06T06:57:52.394696+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"98'16","reported_seq":56,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:39.562182+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T06:58:39.562182+0000","last_peered":"2024-06-06T06:58:39.562182+0000","last_clean":"2024-06-06T06:58:39.562182+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T06:58:39.562182+0000","last_undegraded":"2024-06-06T06:58:39.562182+0000","last_fullsized":"2024-06-06T06:58:39.562182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395426+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T06:57:52.395426+0000","last_peered":"2024-06-06T06:57:52.395426+0000","last_clean":"2024-06-06T06:57:52.395426+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T06:57:52.395426+0000","last_undegraded":"2024-06-06T06:57:52.395426+0000","last_fullsized":"2024-06-06T06:57:52.395426+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394888+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T06:57:52.394888+0000","last_peered":"2024-06-06T06:57:52.394888+0000","last_clean":"2024-06-06T06:57:52.394888+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T06:57:52.394888+0000","last_undegraded":"2024-06-06T06:57:52.394888+0000","last_fullsized":"2024-06-06T06:57:52.394888+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.420133+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T06:58:18.420133+0000","last_peered":"2024-06-06T06:58:18.420133+0000","last_clean":"2024-06-06T06:58:18.420133+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T06:58:18.420133+0000","last_undegraded":"2024-06-06T06:58:18.420133+0000","last_fullsized":"2024-06-06T06:58:18.420133+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430922+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T06:57:52.430922+0000","last_peered":"2024-06-06T06:57:52.430922+0000","last_clean":"2024-06-06T06:57:52.430922+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T06:57:52.430922+0000","last_undegraded":"2024-06-06T06:57:52.430922+0000","last_fullsized":"2024-06-06T06:57:52.430922+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398340+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T06:57:52.398340+0000","last_peered":"2024-06-06T06:57:52.398340+0000","last_clean":"2024-06-06T06:57:52.398340+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T06:57:52.398340+0000","last_undegraded":"2024-06-06T06:57:52.398340+0000","last_fullsized":"2024-06-06T06:57:52.398340+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":48,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:15.400917+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T06:58:15.400917+0000","last_peered":"2024-06-06T06:58:15.400917+0000","last_clean":"2024-06-06T06:58:15.400917+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T06:58:15.400917+0000","last_undegraded":"2024-06-06T06:58:15.400917+0000","last_fullsized":"2024-06-06T06:58:15.400917+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.403669+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T06:57:52.403669+0000","last_peered":"2024-06-06T06:57:52.403669+0000","last_clean":"2024-06-06T06:57:52.403669+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T06:57:52.403669+0000","last_undegraded":"2024-06-06T06:57:52.403669+0000","last_fullsized":"2024-06-06T06:57:52.403669+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.405259+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T06:57:52.405259+0000","last_peered":"2024-06-06T06:57:52.405259+0000","last_clean":"2024-06-06T06:57:52.405259+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T06:57:52.405259+0000","last_undegraded":"2024-06-06T06:57:52.405259+0000","last_fullsized":"2024-06-06T06:57:52.405259+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":47,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.418676+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T06:58:18.418676+0000","last_peered":"2024-06-06T06:58:18.418676+0000","last_clean":"2024-06-06T06:58:18.418676+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T06:58:18.418676+0000","last_undegraded":"2024-06-06T06:58:18.418676+0000","last_fullsized":"2024-06-06T06:58:18.418676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398694+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T06:57:52.398694+0000","last_peered":"2024-06-06T06:57:52.398694+0000","last_clean":"2024-06-06T06:57:52.398694+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T06:57:52.398694+0000","last_undegraded":"2024-06-06T06:57:52.398694+0000","last_fullsized":"2024-06-06T06:57:52.398694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":25,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398402+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T06:57:52.398402+0000","last_peered":"2024-06-06T06:57:52.398402+0000","last_clean":"2024-06-06T06:57:52.398402+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T06:57:52.398402+0000","last_undegraded":"2024-06-06T06:57:52.398402+0000","last_fullsized":"2024-06-06T06:57:52.398402+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":42,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:17.414765+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T06:58:17.414765+0000","last_peered":"2024-06-06T06:58:17.414765+0000","last_clean":"2024-06-06T06:58:17.414765+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T06:58:17.414765+0000","last_undegraded":"2024-06-06T06:58:17.414765+0000","last_fullsized":"2024-06-06T06:58:17.414765+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394965+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T06:57:52.394965+0000","last_peered":"2024-06-06T06:57:52.394965+0000","last_clean":"2024-06-06T06:57:52.394965+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T06:57:52.394965+0000","last_undegraded":"2024-06-06T06:57:52.394965+0000","last_fullsized":"2024-06-06T06:57:52.394965+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398265+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T06:57:52.398265+0000","last_peered":"2024-06-06T06:57:52.398265+0000","last_clean":"2024-06-06T06:57:52.398265+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T06:57:52.398265+0000","last_undegraded":"2024-06-06T06:57:52.398265+0000","last_fullsized":"2024-06-06T06:57:52.398265+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:15.402494+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T06:58:15.402494+0000","last_peered":"2024-06-06T06:58:15.402494+0000","last_clean":"2024-06-06T06:58:15.402494+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T06:58:15.402494+0000","last_undegraded":"2024-06-06T06:58:15.402494+0000","last_fullsized":"2024-06-06T06:58:15.402494+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395189+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T06:57:52.395189+0000","last_peered":"2024-06-06T06:57:52.395189+0000","last_clean":"2024-06-06T06:57:52.395189+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T06:57:52.395189+0000","last_undegraded":"2024-06-06T06:57:52.395189+0000","last_fullsized":"2024-06-06T06:57:52.395189+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397308+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T06:57:52.397308+0000","last_peered":"2024-06-06T06:57:52.397308+0000","last_clean":"2024-06-06T06:57:52.397308+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T06:57:52.397308+0000","last_undegraded":"2024-06-06T06:57:52.397308+0000","last_fullsized":"2024-06-06T06:57:52.397308+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"98'6","reported_seq":75,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:44.460957+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T06:58:44.460957+0000","last_peered":"2024-06-06T06:58:44.460957+0000","last_clean":"2024-06-06T06:58:44.460957+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T06:58:44.460957+0000","last_undegraded":"2024-06-06T06:58:44.460957+0000","last_fullsized":"2024-06-06T06:58:44.460957+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395970+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T06:57:52.395970+0000","last_peered":"2024-06-06T06:57:52.395970+0000","last_clean":"2024-06-06T06:57:52.395970+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T06:57:52.395970+0000","last_undegraded":"2024-06-06T06:57:52.395970+0000","last_fullsized":"2024-06-06T06:57:52.395970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430186+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T06:57:52.430186+0000","last_peered":"2024-06-06T06:57:52.430186+0000","last_clean":"2024-06-06T06:57:52.430186+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T06:57:52.430186+0000","last_undegraded":"2024-06-06T06:57:52.430186+0000","last_fullsized":"2024-06-06T06:57:52.430186+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":43,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.418841+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T06:58:18.418841+0000","last_peered":"2024-06-06T06:58:18.418841+0000","last_clean":"2024-06-06T06:58:18.418841+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T06:58:18.418841+0000","last_undegraded":"2024-06-06T06:58:18.418841+0000","last_fullsized":"2024-06-06T06:58:18.418841+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395590+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:57:52.395590+0000","last_peered":"2024-06-06T06:57:52.395590+0000","last_clean":"2024-06-06T06:57:52.395590+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T06:57:52.395590+0000","last_undegraded":"2024-06-06T06:57:52.395590+0000","last_fullsized":"2024-06-06T06:57:52.395590+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397434+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T06:57:52.397434+0000","last_peered":"2024-06-06T06:57:52.397434+0000","last_clean":"2024-06-06T06:57:52.397434+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T06:57:52.397434+0000","last_undegraded":"2024-06-06T06:57:52.397434+0000","last_fullsized":"2024-06-06T06:57:52.397434+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":45,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:58:18.419506+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T06:58:18.419506+0000","last_peered":"2024-06-06T06:58:18.419506+0000","last_clean":"2024-06-06T06:58:18.419506+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T06:58:18.419506+0000","last_undegraded":"2024-06-06T06:58:18.419506+0000","last_fullsized":"2024-06-06T06:58:18.419506+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.396138+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T06:57:52.396138+0000","last_peered":"2024-06-06T06:57:52.396138+0000","last_clean":"2024-06-06T06:57:52.396138+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T06:57:52.396138+0000","last_undegraded":"2024-06-06T06:57:52.396138+0000","last_fullsized":"2024-06-06T06:57:52.396138+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398805+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T06:57:52.398805+0000","last_peered":"2024-06-06T06:57:52.398805+0000","last_clean":"2024-06-06T06:57:52.398805+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T06:57:52.398805+0000","last_undegraded":"2024-06-06T06:57:52.398805+0000","last_fullsized":"2024-06-06T06:57:52.398805+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395230+0000","last_change":"2024-06-06T06:57:07.558536+0000","last_active":"2024-06-06T06:57:52.395230+0000","last_peered":"2024-06-06T06:57:52.395230+0000","last_clean":"2024-06-06T06:57:52.395230+0000","last_became_active":"2024-06-06T06:57:07.558201+0000","last_became_peered":"2024-06-06T06:57:07.558201+0000","last_unstale":"2024-06-06T06:57:52.395230+0000","last_undegraded":"2024-06-06T06:57:52.395230+0000","last_fullsized":"2024-06-06T06:57:52.395230+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:58:01.546923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394926+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T06:57:52.394926+0000","last_peered":"2024-06-06T06:57:52.394926+0000","last_clean":"2024-06-06T06:57:52.394926+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T06:57:52.394926+0000","last_undegraded":"2024-06-06T06:57:52.394926+0000","last_fullsized":"2024-06-06T06:57:52.394926+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":31,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.430470+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T06:57:52.430470+0000","last_peered":"2024-06-06T06:57:52.430470+0000","last_clean":"2024-06-06T06:57:52.430470+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T06:57:52.430470+0000","last_undegraded":"2024-06-06T06:57:52.430470+0000","last_fullsized":"2024-06-06T06:57:52.430470+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431528+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T06:57:52.431528+0000","last_peered":"2024-06-06T06:57:52.431528+0000","last_clean":"2024-06-06T06:57:52.431528+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T06:57:52.431528+0000","last_undegraded":"2024-06-06T06:57:52.431528+0000","last_fullsized":"2024-06-06T06:57:52.431528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.400274+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T06:57:52.400274+0000","last_peered":"2024-06-06T06:57:52.400274+0000","last_clean":"2024-06-06T06:57:52.400274+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T06:57:52.400274+0000","last_undegraded":"2024-06-06T06:57:52.400274+0000","last_fullsized":"2024-06-06T06:57:52.400274+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395966+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T06:57:52.395966+0000","last_peered":"2024-06-06T06:57:52.395966+0000","last_clean":"2024-06-06T06:57:52.395966+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T06:57:52.395966+0000","last_undegraded":"2024-06-06T06:57:52.395966+0000","last_fullsized":"2024-06-06T06:57:52.395966+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.395220+0000","last_change":"2024-06-06T06:57:07.553546+0000","last_active":"2024-06-06T06:57:52.395220+0000","last_peered":"2024-06-06T06:57:52.395220+0000","last_clean":"2024-06-06T06:57:52.395220+0000","last_became_active":"2024-06-06T06:57:07.553173+0000","last_became_peered":"2024-06-06T06:57:07.553173+0000","last_unstale":"2024-06-06T06:57:52.395220+0000","last_undegraded":"2024-06-06T06:57:52.395220+0000","last_fullsized":"2024-06-06T06:57:52.395220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:50:43.222641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":32,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398182+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T06:57:52.398182+0000","last_peered":"2024-06-06T06:57:52.398182+0000","last_clean":"2024-06-06T06:57:52.398182+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T06:57:52.398182+0000","last_undegraded":"2024-06-06T06:57:52.398182+0000","last_fullsized":"2024-06-06T06:57:52.398182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.394863+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T06:57:52.394863+0000","last_peered":"2024-06-06T06:57:52.394863+0000","last_clean":"2024-06-06T06:57:52.394863+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T06:57:52.394863+0000","last_undegraded":"2024-06-06T06:57:52.394863+0000","last_fullsized":"2024-06-06T06:57:52.394863+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.397623+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T06:57:52.397623+0000","last_peered":"2024-06-06T06:57:52.397623+0000","last_clean":"2024-06-06T06:57:52.397623+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:57:52.397623+0000","last_undegraded":"2024-06-06T06:57:52.397623+0000","last_fullsized":"2024-06-06T06:57:52.397623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.431182+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T06:57:52.431182+0000","last_peered":"2024-06-06T06:57:52.431182+0000","last_clean":"2024-06-06T06:57:52.431182+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T06:57:52.431182+0000","last_undegraded":"2024-06-06T06:57:52.431182+0000","last_fullsized":"2024-06-06T06:57:52.431182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":34,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-06-06T06:57:52.398480+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T06:57:52.398480+0000","last_peered":"2024-06-06T06:57:52.398480+0000","last_clean":"2024-06-06T06:57:52.398480+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T06:57:52.398480+0000","last_undegraded":"2024-06-06T06:57:52.398480+0000","last_fullsized":"2024-06-06T06:57:52.398480+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285685,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70127,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":305,"num_read_kb":228,"num_write":250,"num_write_kb":142,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":204128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351011,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36396,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708756,"statfs":{"total":95995035648,"available":95957766144,"internally_reserved":0,"allocated":1085440,"data_stored":555058,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612648,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28224,"kb_used_data":1208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716928,"statfs":{"total":95995035648,"available":95966134272,"internally_reserved":0,"allocated":1236992,"data_stored":1204799,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776172,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":558694,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037811,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28096,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717056,"statfs":{"total":95995035648,"available":95966265344,"internally_reserved":0,"allocated":1105920,"data_stored":554976,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1159168,"data_stored":1151472,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462973,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1085440,"data_stored":550102,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1146880,"data_stored":607804,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738458,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":551948,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411409,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":614045,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149771,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32248,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712904,"statfs":{"total":95995035648,"available":95962013696,"internally_reserved":0,"allocated":1097728,"data_stored":549810,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":555188,"data_compressed":26321,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724609,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1167360,"data_stored":1146664,"data_compressed":62644,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":58424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":64665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T06:58:49.385 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T06:58:49.386 INFO:tasks.cephadm:Setting up client nodes... 2024-06-06T06:58:49.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-06T06:58:49.631 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:49.752 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:49 smithi073 ceph-mon[32303]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:49.752 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:49 smithi073 ceph-mon[32303]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T06:58:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:49 smithi136 ceph-mon[35901]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:49.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:49 smithi136 ceph-mon[35901]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T06:58:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:49 smithi170 ceph-mon[36108]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T06:58:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:49 smithi170 ceph-mon[36108]: pgmap v357: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T06:58:51.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 -- 172.21.15.73:0/3330524407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102610 msgr2=0x7f959c102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:51.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 --2- 172.21.15.73:0/3330524407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102610 0x7f959c102a70 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f95900099c0 tx=0x7f9590009cd0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 -- 172.21.15.73:0/3330524407 shutdown_connections 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 --2- 172.21.15.73:0/3330524407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 0x7f959c101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 --2- 172.21.15.73:0/3330524407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102fb0 0x7f959c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 --2- 172.21.15.73:0/3330524407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102610 0x7f959c102a70 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 -- 172.21.15.73:0/3330524407 >> 172.21.15.73:0/3330524407 conn(0x7f959c0fc9f0 msgr2=0x7f959c0fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 -- 172.21.15.73:0/3330524407 shutdown_connections 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.590+0000 7f95a143f700 1 -- 172.21.15.73:0/3330524407 wait complete. 2024-06-06T06:58:51.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 Processor -- start 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 -- start start 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 0x7f959c1974a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 0x7f959c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10ceb0 con 0x7f959c102fb0 2024-06-06T06:58:51.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10cd30 con 0x7f959c102610 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f95a143f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f959c10d030 con 0x7f959c101410 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f959b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f959b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38162/0 (socket says 172.21.15.73:38162) 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f959b7fe700 1 -- 172.21.15.73:0/3355213659 learned_addr learned my addr 172.21.15.73:0/3355213659 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.591+0000 7f959affd700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 0x7f959c1974a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959a7fc700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 0x7f959c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 -- 172.21.15.73:0/3355213659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 msgr2=0x7f959c1974a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 0x7f959c1974a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 -- 172.21.15.73:0/3355213659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 msgr2=0x7f959c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 0x7f959c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 -- 172.21.15.73:0/3355213659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c19f100 con 0x7f959c102fb0 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959a7fc700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 0x7f959c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:58:51.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f959b7fe700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f959400add0 tx=0x7f959400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594017020 con 0x7f959c102fb0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c19f3f0 con 0x7f959c102fb0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f959400ae00 con 0x7f959c102fb0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9594005350 con 0x7f959c102fb0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.592+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c19f930 con 0x7f959c102fb0 2024-06-06T06:58:51.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.593+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c1081b0 con 0x7f959c102fb0 2024-06-06T06:58:51.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.594+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f959400c940 con 0x7f959c102fb0 2024-06-06T06:58:51.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.595+0000 7f9583fff700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 0x7f95840a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:51.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.595+0000 7f959affd700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 0x7f95840a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:51.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.595+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9092+0+0 (secure 0 0 0) 0x7f95940ced10 con 0x7f959c102fb0 2024-06-06T06:58:51.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.596+0000 7f959affd700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 0x7f95840a5220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f959c102470 tx=0x7f958c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:51.600 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.600+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9594021020 con 0x7f959c102fb0 2024-06-06T06:58:51.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.877+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f959c06b3d0 con 0x7f959c102fb0 2024-06-06T06:58:51.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.881+0000 7f9583fff700 1 -- 172.21.15.73:0/3355213659 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f959409b840 con 0x7f959c102fb0 2024-06-06T06:58:51.882 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-06-06T06:58:51.882 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQArXmFmVwFbNBAA6Y6OvajX3uP7nMHxHal0tQ== 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 msgr2=0x7f95840a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 0x7f95840a5220 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f959c102470 tx=0x7f958c00b410 comp rx=0 tx=0).stop 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 msgr2=0x7f959c19ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f959400add0 tx=0x7f959400c330 comp rx=0 tx=0).stop 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 shutdown_connections 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f95840a2d60 0x7f95840a5220 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f959c101410 0x7f959c1974a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c102fb0 0x7f959c19ea60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 --2- 172.21.15.73:0/3355213659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f959c102610 0x7f959c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 >> 172.21.15.73:0/3355213659 conn(0x7f959c0fc9f0 msgr2=0x7f959c105910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 shutdown_connections 2024-06-06T06:58:51.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:51.884+0000 7f95a143f700 1 -- 172.21.15.73:0/3355213659 wait complete. 2024-06-06T06:58:52.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T06:58:52.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:52 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T06:58:52.640 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T06:58:52.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-06T06:58:52.641 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-06T06:58:52.677 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-06T06:58:52.677 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-06T06:58:52.677 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-06T06:58:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T06:58:52.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:52 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: pgmap v358: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]: dispatch 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-06-06T06:58:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-06T06:58:52.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:52 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3355213659' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-06T06:58:52.955 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T06:58:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-06-06T06:58:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-06-06T06:58:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:53 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-06T06:58:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:53 smithi073 ceph-mon[32303]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T06:58:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T06:58:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-06-06T06:58:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-06-06T06:58:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:53 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-06T06:58:53.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:53 smithi136 ceph-mon[35901]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T06:58:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-06-06T06:58:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [5, 7]}]': finished 2024-06-06T06:58:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-06-06T06:58:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:53 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-06-06T06:58:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:53 smithi170 ceph-mon[36108]: osdmap e99: 12 total, 12 up, 12 in 2024-06-06T06:58:54.725 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:54 smithi073 ceph-mon[32303]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-06-06T06:58:54.725 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:54 smithi073 ceph-mon[32303]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T06:58:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:54 smithi136 ceph-mon[35901]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-06-06T06:58:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:54 smithi136 ceph-mon[35901]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 -- 172.21.15.73:0/1175849811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e806ab30 msgr2=0x7f02e810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 --2- 172.21.15.73:0/1175849811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e806ab30 0x7f02e810f320 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b0a0 tx=0x7f02e000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 -- 172.21.15.73:0/1175849811 shutdown_connections 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 --2- 172.21.15.73:0/1175849811 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e8108500 0x7f02e81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 --2- 172.21.15.73:0/1175849811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e806ab30 0x7f02e810f320 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 --2- 172.21.15.73:0/1175849811 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806a170 0x7f02e806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 -- 172.21.15.73:0/1175849811 >> 172.21.15.73:0/1175849811 conn(0x7f02e8078340 msgr2=0x7f02e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 -- 172.21.15.73:0/1175849811 shutdown_connections 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.825+0000 7f02efad4700 1 -- 172.21.15.73:0/1175849811 wait complete. 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 Processor -- start 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 -- start start 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e806a170 0x7f02e819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 0x7f02e819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8112690 con 0x7f02e8108500 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8112510 con 0x7f02e806ab30 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02efad4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02e8112810 con 0x7f02e806a170 2024-06-06T06:58:54.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38174/0 (socket says 172.21.15.73:38174) 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 -- 172.21.15.73:0/3254720459 learned_addr learned my addr 172.21.15.73:0/3254720459 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ed870700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e806a170 0x7f02e819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ed06f700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 0x7f02e819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 -- 172.21.15.73:0/3254720459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e806a170 msgr2=0x7f02e819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e806a170 0x7f02e819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 -- 172.21.15.73:0/3254720459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 msgr2=0x7f02e819ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 0x7f02e819ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ee071700 1 -- 172.21.15.73:0/3254720459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02e8199130 con 0x7f02e8108500 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.826+0000 7f02ed06f700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 0x7f02e819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02ee071700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b830 tx=0x7f02e0004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:54.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0021070 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02e81993c0 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02e0003c00 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0004790 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.827+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02e81a5690 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.828+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02e810a9e0 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.829+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f02e0007a00 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.829+0000 7f02deffd700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 0x7f02d40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.829+0000 7f02ed870700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 0x7f02d40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.830+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f02e00d07b0 con 0x7f02e8108500 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.830+0000 7f02ed870700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 0x7f02d40a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006fd0 tx=0x7f02e4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:54.836 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:54.834+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f02e009d270 con 0x7f02e8108500 2024-06-06T06:58:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:54 smithi170 ceph-mon[36108]: pgmap v360: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 818 B/s wr, 10 op/s 2024-06-06T06:58:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:54 smithi170 ceph-mon[36108]: osdmap e100: 12 total, 12 up, 12 in 2024-06-06T06:58:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.089+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f02e8199eb0 con 0x7f02e8108500 2024-06-06T06:58:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.094+0000 7f02deffd700 1 -- 172.21.15.73:0/3254720459 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263198 (secure 0 0 0) 0x7f02e009cb70 con 0x7f02e8108500 2024-06-06T06:58:55.098 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 msgr2=0x7f02d40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 0x7f02d40a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f02e4006fd0 tx=0x7f02e4008040 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 msgr2=0x7f02e81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f02e000b830 tx=0x7f02e0004d90 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 shutdown_connections 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f02d40a2cb0 0x7f02d40a5170 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02e806a170 0x7f02e819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02e8108500 0x7f02e81989d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 --2- 172.21.15.73:0/3254720459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02e806ab30 0x7f02e819ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 >> 172.21.15.73:0/3254720459 conn(0x7f02e8078340 msgr2=0x7f02e8100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:55.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 shutdown_connections 2024-06-06T06:58:55.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:55.098+0000 7f02efad4700 1 -- 172.21.15.73:0/3254720459 wait complete. 2024-06-06T06:58:55.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:55 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3254720459' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T06:58:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:55 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3254720459' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T06:58:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:55 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3254720459' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-06T06:58:55.891 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":793833713},{"type":"v1","addr":"172.21.15.73:6801","nonce":793833713}]},"active_addr":"172.21.15.73:6801/793833713","active_change":"2024-06-06T06:49:51.216882+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.73:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3113057040}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1478961120}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1117851147}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1988924098}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2220828338}]}]} 2024-06-06T06:58:55.896 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-06T06:58:55.896 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-06T06:58:55.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:58:56.138 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:56.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:56 smithi073 ceph-mon[32303]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-06T06:58:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:56 smithi136 ceph-mon[35901]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-06T06:58:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:56 smithi170 ceph-mon[36108]: pgmap v362: 129 pgs: 129 active+clean; 645 KiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-06-06T06:58:57.769 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:57 smithi073 ceph-mon[32303]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T06:58:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:57 smithi136 ceph-mon[35901]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T06:58:57.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:57 smithi170 ceph-mon[36108]: pgmap v363: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-06-06T06:58:58.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 -- 172.21.15.73:0/2277264250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c102770 msgr2=0x7f711c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 --2- 172.21.15.73:0/2277264250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c102770 0x7f711c102b50 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f711400b0d0 tx=0x7f711400b3e0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 -- 172.21.15.73:0/2277264250 shutdown_connections 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 --2- 172.21.15.73:0/2277264250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c107860 0x7f711c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 --2- 172.21.15.73:0/2277264250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c102770 0x7f711c102b50 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 --2- 172.21.15.73:0/2277264250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 -- 172.21.15.73:0/2277264250 >> 172.21.15.73:0/2277264250 conn(0x7f711c079600 msgr2=0x7f711c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 -- 172.21.15.73:0/2277264250 shutdown_connections 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.010+0000 7f7120bf2700 1 -- 172.21.15.73:0/2277264250 wait complete. 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 Processor -- start 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 -- start start 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c102770 0x7f711c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:58.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f711c1126c0 con 0x7f711c107860 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f711c112540 con 0x7f711c103120 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f7120bf2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f711c112840 con 0x7f711c102770 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f711ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f711ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35594/0 (socket says 172.21.15.73:35594) 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.011+0000 7f711ad9d700 1 -- 172.21.15.73:0/1693135117 learned_addr learned my addr 172.21.15.73:0/1693135117 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711a59c700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c102770 0x7f711c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f7119d9b700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 -- 172.21.15.73:0/1693135117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c102770 msgr2=0x7f711c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c102770 0x7f711c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 -- 172.21.15.73:0/1693135117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 msgr2=0x7f711c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 -- 172.21.15.73:0/1693135117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f711c199100 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f7119d9b700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f711ad9d700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d980 tx=0x7f710c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c017070 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f711c1993f0 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f710c005020 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710c0099b0 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.012+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f711c1a56b0 con 0x7f711c107860 2024-06-06T06:58:58.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.013+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f711c10aaa0 con 0x7f711c107860 2024-06-06T06:58:58.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.015+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f710c0051d0 con 0x7f711c107860 2024-06-06T06:58:58.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.016+0000 7f71137fe700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 0x7f71080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:58:58.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.016+0000 7f711a59c700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 0x7f71080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:58:58.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.016+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f710c0cf810 con 0x7f711c107860 2024-06-06T06:58:58.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.016+0000 7f711a59c700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 0x7f71080a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7114009fd0 tx=0x7f711400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:58:58.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.020+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f710c020020 con 0x7f711c107860 2024-06-06T06:58:58.216 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.216+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f711c10a480 con 0x7f711c107860 2024-06-06T06:58:58.218 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.217+0000 7f71137fe700 1 -- 172.21.15.73:0/1693135117 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24816 (secure 0 0 0) 0x7f710c09c240 con 0x7f711c107860 2024-06-06T06:58:58.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:58:58.219 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:58:53.383839+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 msgr2=0x7f71080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 0x7f71080a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f7114009fd0 tx=0x7f711400b040 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 msgr2=0x7f711c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f710c00d980 tx=0x7f710c00dc90 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 shutdown_connections 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71080a2cb0 0x7f71080a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f711c102770 0x7f711c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f711c107860 0x7f711c1989a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 --2- 172.21.15.73:0/1693135117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f711c103120 0x7f711c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:58:58.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.219+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 >> 172.21.15.73:0/1693135117 conn(0x7f711c079600 msgr2=0x7f711c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:58:58.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.220+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 shutdown_connections 2024-06-06T06:58:58.222 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:58:58.220+0000 7f7120bf2700 1 -- 172.21.15.73:0/1693135117 wait complete. 2024-06-06T06:58:58.641 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:58 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1693135117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:58 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1693135117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:58.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:58 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1693135117' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:58:58.916 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-06T06:58:58.916 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T06:58:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:58:59.530 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:58:59 smithi073 ceph-mon[32303]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T06:58:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:58:59 smithi136 ceph-mon[35901]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T06:58:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:58:59 smithi170 ceph-mon[36108]: pgmap v364: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 -- 172.21.15.73:0/1889876709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 msgr2=0x7fcf4410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 --2- 172.21.15.73:0/1889876709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf4410e240 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00b0a0 tx=0x7fcf3c00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 -- 172.21.15.73:0/1889876709 shutdown_connections 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 --2- 172.21.15.73:0/1889876709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf4410e240 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 --2- 172.21.15.73:0/1889876709 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44107400 0x7fcf441077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 --2- 172.21.15.73:0/1889876709 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44101390 0x7fcf44101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 -- 172.21.15.73:0/1889876709 >> 172.21.15.73:0/1889876709 conn(0x7fcf44078340 msgr2=0x7fcf44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 -- 172.21.15.73:0/1889876709 shutdown_connections 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.897+0000 7fcf49be4700 1 -- 172.21.15.73:0/1889876709 wait complete. 2024-06-06T06:59:00.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 Processor -- start 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 -- start start 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44101390 0x7fcf44199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 0x7fcf441a4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf441115d0 con 0x7fcf44101d50 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44111450 con 0x7fcf44101390 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf49be4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf44111750 con 0x7fcf44107400 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35610/0 (socket says 172.21.15.73:35610) 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf42ffd700 1 -- 172.21.15.73:0/3961657914 learned_addr learned my addr 172.21.15.73:0/3961657914 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.898+0000 7fcf437fe700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44101390 0x7fcf44199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf43fff700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 0x7fcf441a4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:00.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 -- 172.21.15.73:0/3961657914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 msgr2=0x7fcf441a4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 0x7fcf441a4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 -- 172.21.15.73:0/3961657914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44101390 msgr2=0x7fcf44199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44101390 0x7fcf44199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 -- 172.21.15.73:0/3961657914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf441a4f50 con 0x7fcf44101d50 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf43fff700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 0x7fcf441a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf42ffd700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d980 tx=0x7fcf3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34017070 con 0x7fcf44101d50 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf34004830 con 0x7fcf44101d50 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf441977e0 con 0x7fcf44101d50 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf34005440 con 0x7fcf44101d50 2024-06-06T06:59:00.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.899+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf44197cf0 con 0x7fcf44101d50 2024-06-06T06:59:00.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.900+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf44109970 con 0x7fcf44101d50 2024-06-06T06:59:00.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.901+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcf340055e0 con 0x7fcf44101d50 2024-06-06T06:59:00.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.902+0000 7fcf40ff9700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 0x7fcf2c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:00.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.902+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fcf340cf4d0 con 0x7fcf44101d50 2024-06-06T06:59:00.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.902+0000 7fcf437fe700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 0x7fcf2c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:00.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.902+0000 7fcf437fe700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 0x7fcf2c0a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:00.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:00.906+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf34020020 con 0x7fcf44101d50 2024-06-06T06:59:01.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.101+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcf4406a770 con 0x7fcf44101d50 2024-06-06T06:59:01.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.103+0000 7fcf40ff9700 1 -- 172.21.15.73:0/3961657914 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24816 (secure 0 0 0) 0x7fcf3409bf00 con 0x7fcf44101d50 2024-06-06T06:59:01.104 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:59:01.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:58:53.383839+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T06:59:01.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 msgr2=0x7fcf2c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:01.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 0x7fcf2c0a51f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcf38006fd0 tx=0x7fcf38008040 comp rx=0 tx=0).stop 2024-06-06T06:59:01.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 msgr2=0x7fcf44199ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:01.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fcf3400d980 tx=0x7fcf3400dd40 comp rx=0 tx=0).stop 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 shutdown_connections 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fcf2c0a2d30 0x7fcf2c0a51f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf44107400 0x7fcf441a4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf44101390 0x7fcf44199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 --2- 172.21.15.73:0/3961657914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf44101d50 0x7fcf44199ed0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 >> 172.21.15.73:0/3961657914 conn(0x7fcf44078340 msgr2=0x7fcf440fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 shutdown_connections 2024-06-06T06:59:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:01.105+0000 7fcf49be4700 1 -- 172.21.15.73:0/3961657914 wait complete. 2024-06-06T06:59:01.449 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:01 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3961657914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:59:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:01 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3961657914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:59:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:01 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3961657914' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T06:59:01.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-06T06:59:01.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-06T06:59:01.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-06T06:59:01.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-06T06:59:01.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-06T06:59:01.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-06T06:59:01.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-06T06:59:01.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-06T06:59:01.784 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-06T06:59:01.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-06T06:59:01.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-06T06:59:01.786 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-06T06:59:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.498 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:02 smithi170 ceph-mon[36108]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-06T06:59:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:02 smithi073 ceph-mon[32303]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-06T06:59:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:02 smithi136 ceph-mon[35901]: pgmap v365: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s 2024-06-06T06:59:02.568 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.577 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.577 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.581 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.582 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:02.586 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:04 smithi073 ceph-mon[32303]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T06:59:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:04 smithi136 ceph-mon[35901]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T06:59:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:04 smithi170 ceph-mon[36108]: pgmap v366: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-06T06:59:06.681 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:06 smithi073 ceph-mon[32303]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:06 smithi136 ceph-mon[35901]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:06 smithi170 ceph-mon[36108]: pgmap v367: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:07.320 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:07.323 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:07.325 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:08.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:08 smithi170 ceph-mon[36108]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:08.644 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:08 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:59:08.722 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:08 smithi073 ceph-mon[32303]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:08.722 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:08 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:59:08.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:08 smithi136 ceph-mon[35901]: pgmap v368: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:08.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:08 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 -- 172.21.15.73:0/3236680075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006b3c0 msgr2=0x7f65e010f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3236680075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006b3c0 0x7f65e010f310 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009f90 tx=0x7f65d800b330 comp rx=0 tx=0).stop 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 -- 172.21.15.73:0/3236680075 shutdown_connections 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3236680075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006b3c0 0x7f65e010f310 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3236680075 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65e01084f0 0x7f65e01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3236680075 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006aa00 0x7f65e006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 -- 172.21.15.73:0/3236680075 >> 172.21.15.73:0/3236680075 conn(0x7f65e0078300 msgr2=0x7f65e0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 -- 172.21.15.73:0/3236680075 shutdown_connections 2024-06-06T06:59:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.416+0000 7f65e4f18700 1 -- 172.21.15.73:0/3236680075 wait complete. 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 Processor -- start 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 -- start start 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 0x7f65e019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006b3c0 0x7f65e019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65e01084f0 0x7f65e0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0112600 con 0x7f65e006aa00 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0112480 con 0x7f65e01084f0 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65e4f18700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f65e0112780 con 0x7f65e006b3c0 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 0x7f65e019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 0x7f65e019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33694/0 (socket says 172.21.15.73:33694) 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 -- 172.21.15.73:0/3438037079 learned_addr learned my addr 172.21.15.73:0/3438037079 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65ddd9b700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006b3c0 0x7f65e019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65ded9d700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65e01084f0 0x7f65e0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 -- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006b3c0 msgr2=0x7f65e019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006b3c0 0x7f65e019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 -- 172.21.15.73:0/3438037079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65e01084f0 msgr2=0x7f65e0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f65e01084f0 0x7f65e0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65e01990c0 con 0x7f65e006aa00 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65ddd9b700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f65e006b3c0 0x7f65e019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.417+0000 7f65de59c700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 0x7f65e019e750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65d000e9c0 tx=0x7f65d000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.418+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0004a30 con 0x7f65e006aa00 2024-06-06T06:59:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.418+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65d000cc70 con 0x7f65e006aa00 2024-06-06T06:59:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.418+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65e01993b0 con 0x7f65e006aa00 2024-06-06T06:59:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.418+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d00054e0 con 0x7f65e006aa00 2024-06-06T06:59:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.418+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65e0101380 con 0x7f65e006aa00 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.419+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f65e0062c20 con 0x7f65e006aa00 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.420+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65d0004c60 con 0x7f65e006aa00 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65cf7fe700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f65c80a2d30 0x7f65c80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65ddd9b700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f65c80a2d30 0x7f65c80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f65d00cf980 con 0x7f65e006aa00 2024-06-06T06:59:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65cf7fe700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] conn(0x7f65c80ae090 0x7f65c80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65c80b0c00 con 0x7f65c80ae090 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f65d00cfdd0 con 0x7f65e006aa00 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65ded9d700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] conn(0x7f65c80ae090 0x7f65c80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.421+0000 7f65ddd9b700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f65c80a2d30 0x7f65c80a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65e0078970 tx=0x7f65d4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.422+0000 7f65ded9d700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] conn(0x7f65c80ae090 0x7f65c80b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.422+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== osd.2 v2:172.21.15.170:6800/2309158853 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65c80b0c00 con 0x7f65c80ae090 2024-06-06T06:59:09.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.439+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 --> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f65e0063020 con 0x7f65c80ae090 2024-06-06T06:59:09.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.439+0000 7f65cf7fe700 1 -- 172.21.15.73:0/3438037079 <== osd.2 v2:172.21.15.170:6800/2309158853 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f65e0063020 con 0x7f65c80ae090 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] conn(0x7f65c80ae090 msgr2=0x7f65c80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] conn(0x7f65c80ae090 0x7f65c80b0530 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f65c80a2d30 msgr2=0x7f65c80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f65c80a2d30 0x7f65c80a51f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f65e0078970 tx=0x7f65d4009260 comp rx=0 tx=0).stop 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 msgr2=0x7f65e019e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 --2- 172.21.15.73:0/3438037079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65e006aa00 0x7f65e019e750 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f65d000e9c0 tx=0x7f65d000ecd0 comp rx=0 tx=0).stop 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65ded9d700 1 -- 172.21.15.73:0/3438037079 reap_dead start 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 shutdown_connections 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 >> 172.21.15.73:0/3438037079 conn(0x7f65e0078300 msgr2=0x7f65e0100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 shutdown_connections 2024-06-06T06:59:09.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:09.440+0000 7f65e4f18700 1 -- 172.21.15.73:0/3438037079 wait complete. 2024-06-06T06:59:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:10 smithi073 ceph-mon[32303]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:10 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:10 smithi170 ceph-mon[36108]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:10 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:10 smithi136 ceph-mon[35901]: pgmap v369: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:10.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:10 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:12.535 INFO:teuthology.orchestra.run.smithi073.stdout:77309411414 2024-06-06T06:59:12.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-06T06:59:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:12 smithi170 ceph-mon[36108]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:12.635 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:12 smithi073 ceph-mon[32303]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.746+0000 7f9b6c092700 1 -- 172.21.15.73:0/1733284125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b64101d40 msgr2=0x7f9b6410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.746+0000 7f9b6c092700 1 --2- 172.21.15.73:0/1733284125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b64101d40 0x7f9b6410e230 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b0a0 tx=0x7f9b5c00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 -- 172.21.15.73:0/1733284125 shutdown_connections 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 --2- 172.21.15.73:0/1733284125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b641073f0 0x7f9b641077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 --2- 172.21.15.73:0/1733284125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b64101d40 0x7f9b6410e230 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 --2- 172.21.15.73:0/1733284125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b64101380 0x7f9b64101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 -- 172.21.15.73:0/1733284125 >> 172.21.15.73:0/1733284125 conn(0x7f9b64078300 msgr2=0x7f9b64078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 -- 172.21.15.73:0/1733284125 shutdown_connections 2024-06-06T06:59:12.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 -- 172.21.15.73:0/1733284125 wait complete. 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 Processor -- start 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.747+0000 7f9b6c092700 1 -- start start 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b64101380 0x7f9b641998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b64101d40 0x7f9b64199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 0x7f9b641a4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b64111510 con 0x7f9b641073f0 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9b64111390 con 0x7f9b64101d40 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 0x7f9b641a4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6c092700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9b64111690 con 0x7f9b64101380 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 0x7f9b641a4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33734/0 (socket says 172.21.15.73:33734) 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 -- 172.21.15.73:0/3037621433 learned_addr learned my addr 172.21.15.73:0/3037621433 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b69e2e700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b64101380 0x7f9b641998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6962d700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b64101d40 0x7f9b64199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 -- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b64101380 msgr2=0x7f9b641998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9b64101380 0x7f9b641998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 -- 172.21.15.73:0/3037621433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b64101d40 msgr2=0x7f9b64199e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9b64101d40 0x7f9b64199e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b641a4eb0 con 0x7f9b641073f0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.748+0000 7f9b6a62f700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 0x7f9b641a4970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b830 tx=0x7f9b5c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.749+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c021070 con 0x7f9b641073f0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.749+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b641a5080 con 0x7f9b641073f0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.749+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b641a5580 con 0x7f9b641073f0 2024-06-06T06:59:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.749+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b5c007e30 con 0x7f9b641073f0 2024-06-06T06:59:12.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.749+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c004700 con 0x7f9b641073f0 2024-06-06T06:59:12.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.750+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9b64062ba0 con 0x7f9b641073f0 2024-06-06T06:59:12.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.752+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f9b5c007980 con 0x7f9b641073f0 2024-06-06T06:59:12.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.752+0000 7f9b5affd700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9b500a2cb0 0x7f9b500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b69e2e700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9b500a2cb0 0x7f9b500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f9b5c00f020 con 0x7f9b641073f0 2024-06-06T06:59:12.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b5affd700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] conn(0x7f9b500ae010 0x7f9b500b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9b5c00aea0 con 0x7f9b500ae010 2024-06-06T06:59:12.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f9b5c0d1360 con 0x7f9b641073f0 2024-06-06T06:59:12.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b69e2e700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9b500a2cb0 0x7f9b500a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b60005950 tx=0x7f9b6000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.753+0000 7f9b6962d700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] conn(0x7f9b500ae010 0x7f9b500b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.754+0000 7f9b6962d700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] conn(0x7f9b500ae010 0x7f9b500b04b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.755+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== osd.8 v2:172.21.15.170:6816/65881112 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9b5c00aea0 con 0x7f9b500ae010 2024-06-06T06:59:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.776+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 --> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9b64062fa0 con 0x7f9b500ae010 2024-06-06T06:59:12.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b5affd700 1 -- 172.21.15.73:0/3037621433 <== osd.8 v2:172.21.15.170:6816/65881112 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9b64062fa0 con 0x7f9b500ae010 2024-06-06T06:59:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] conn(0x7f9b500ae010 msgr2=0x7f9b500b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] conn(0x7f9b500ae010 0x7f9b500b04b0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9b500a2cb0 msgr2=0x7f9b500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9b500a2cb0 0x7f9b500a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b60005950 tx=0x7f9b6000b410 comp rx=0 tx=0).stop 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 msgr2=0x7f9b641a4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 --2- 172.21.15.73:0/3037621433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b641073f0 0x7f9b641a4970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00b830 tx=0x7f9b5c004ce0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6a62f700 1 -- 172.21.15.73:0/3037621433 reap_dead start 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 shutdown_connections 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 >> 172.21.15.73:0/3037621433 conn(0x7f9b64078300 msgr2=0x7f9b640fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 shutdown_connections 2024-06-06T06:59:12.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:12.777+0000 7f9b6c092700 1 -- 172.21.15.73:0/3037621433 wait complete. 2024-06-06T06:59:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:12 smithi136 ceph-mon[35901]: pgmap v370: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- 172.21.15.73:0/4071445180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 msgr2=0x7f13781088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 --2- 172.21.15.73:0/4071445180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f13781088d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1368009990 tx=0x7f1368009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- 172.21.15.73:0/4071445180 shutdown_connections 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 --2- 172.21.15.73:0/4071445180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f137806b3c0 0x7f137810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 --2- 172.21.15.73:0/4071445180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f13781088d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 --2- 172.21.15.73:0/4071445180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f137806aa00 0x7f137806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- 172.21.15.73:0/4071445180 >> 172.21.15.73:0/4071445180 conn(0x7f1378078300 msgr2=0x7f1378078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:12.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- 172.21.15.73:0/4071445180 shutdown_connections 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- 172.21.15.73:0/4071445180 wait complete. 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 Processor -- start 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.322+0000 7f137dd77700 1 -- start start 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f137806aa00 0x7f137819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f137806b3c0 0x7f137819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f1378198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1378112600 con 0x7f13781084f0 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1378112480 con 0x7f137806aa00 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f137dd77700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1378112780 con 0x7f137806b3c0 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f1378198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f1378198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33708/0 (socket says 172.21.15.73:33708) 2024-06-06T06:59:12.870 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1376ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f137806b3c0 0x7f137819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 -- 172.21.15.73:0/1646044255 learned_addr learned my addr 172.21.15.73:0/1646044255 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f13777fe700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f137806aa00 0x7f137819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 -- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f137806b3c0 msgr2=0x7f137819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f137806b3c0 0x7f137819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 -- 172.21.15.73:0/1646044255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f137806aa00 msgr2=0x7f137819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.871 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f137806aa00 0x7f137819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13781990c0 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1377fff700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f1378198960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1370007ee0 tx=0x7f137000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.323+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1370004bb0 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.324+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1370004d50 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.324+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13781993b0 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.324+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13700058e0 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.324+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13781a5670 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.324+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1378062c20 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.326+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f137001a020 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.326+0000 7f1374ff9700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13600a2cb0 0x7f13600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f13777fe700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13600a2cb0 0x7f13600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f13700cf370 con 0x7f13781084f0 2024-06-06T06:59:12.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1374ff9700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] conn(0x7f13600ae010 0x7f13600b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1370021e70 con 0x7f13600ae010 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f13700cf7c0 con 0x7f13781084f0 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f13777fe700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13600a2cb0 0x7f13600a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f136800b5c0 tx=0x7f1368005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1376ffd700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] conn(0x7f13600ae010 0x7f13600b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.327+0000 7f1376ffd700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] conn(0x7f13600ae010 0x7f13600b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.328+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== osd.11 v2:172.21.15.170:6824/3729886831 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1370021e70 con 0x7f13600ae010 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.345+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 --> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f13780630a0 con 0x7f13600ae010 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f1374ff9700 1 -- 172.21.15.73:0/1646044255 <== osd.11 v2:172.21.15.170:6824/3729886831 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f13780630a0 con 0x7f13600ae010 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] conn(0x7f13600ae010 msgr2=0x7f13600b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] conn(0x7f13600ae010 0x7f13600b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13600a2cb0 msgr2=0x7f13600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13600a2cb0 0x7f13600a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f136800b5c0 tx=0x7f1368005960 comp rx=0 tx=0).stop 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 msgr2=0x7f1378198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:12.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.346+0000 7f137dd77700 1 --2- 172.21.15.73:0/1646044255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13781084f0 0x7f1378198960 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1370007ee0 tx=0x7f137000d3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.347+0000 7f1377fff700 1 -- 172.21.15.73:0/1646044255 reap_dead start 2024-06-06T06:59:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.347+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 shutdown_connections 2024-06-06T06:59:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.347+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 >> 172.21.15.73:0/1646044255 conn(0x7f1378078300 msgr2=0x7f1378100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.347+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 shutdown_connections 2024-06-06T06:59:12.874 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:11.347+0000 7f137dd77700 1 -- 172.21.15.73:0/1646044255 wait complete. 2024-06-06T06:59:13.526 INFO:teuthology.orchestra.run.smithi073.stdout:347892351017 2024-06-06T06:59:13.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-06T06:59:14.178 INFO:teuthology.orchestra.run.smithi073.stdout:257698037817 2024-06-06T06:59:14.178 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-06T06:59:14.463 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:14 smithi073 ceph-mon[32303]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:14.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:14 smithi136 ceph-mon[35901]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:14.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:14 smithi170 ceph-mon[36108]: pgmap v371: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:15.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 -- 172.21.15.73:0/2832050001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102550 msgr2=0x7f4fd41029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/2832050001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102550 0x7f4fd41029d0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80099c0 tx=0x7f4fc8009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 -- 172.21.15.73:0/2832050001 shutdown_connections 2024-06-06T06:59:15.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/2832050001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd4102f10 0x7f4fd410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/2832050001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd41073a0 0x7f4fd4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/2832050001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102550 0x7f4fd41029d0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 -- 172.21.15.73:0/2832050001 >> 172.21.15.73:0/2832050001 conn(0x7f4fd4079650 msgr2=0x7f4fd4079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 -- 172.21.15.73:0/2832050001 shutdown_connections 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.843+0000 7f4fdc64e700 1 -- 172.21.15.73:0/2832050001 wait complete. 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 Processor -- start 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 -- start start 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd4102550 0x7f4fd419e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 0x7f4fd419edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd41073a0 0x7f4fd4198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4112690 con 0x7f4fd4102f10 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4112510 con 0x7f4fd41073a0 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fdc64e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd4112810 con 0x7f4fd4102550 2024-06-06T06:59:15.848 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fd9be9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 0x7f4fd419edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fd9be9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 0x7f4fd419edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46370/0 (socket says 172.21.15.73:46370) 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.844+0000 7f4fd9be9700 1 -- 172.21.15.73:0/4223753585 learned_addr learned my addr 172.21.15.73:0/4223753585 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fda3ea700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd4102550 0x7f4fd419e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fdabeb700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd41073a0 0x7f4fd4198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 -- 172.21.15.73:0/4223753585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd4102550 msgr2=0x7f4fd419e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fd4102550 0x7f4fd419e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 -- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd41073a0 msgr2=0x7f4fd4198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fd41073a0 0x7f4fd4198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd4199190 con 0x7f4fd4102f10 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd9be9700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 0x7f4fd419edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc80037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8027070 con 0x7f4fd4102f10 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc8004140 con 0x7f4fd4102f10 2024-06-06T06:59:15.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800f460 con 0x7f4fd4102f10 2024-06-06T06:59:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd4199420 con 0x7f4fd4102f10 2024-06-06T06:59:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.845+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd41a5740 con 0x7f4fd4102f10 2024-06-06T06:59:15.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.846+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4fd4062ba0 con 0x7f4fd4102f10 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.847+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4fc802a070 con 0x7f4fd4102f10 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fd37fe700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4fbc0a2d30 0x7f4fbc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fda3ea700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4fbc0a2d30 0x7f4fbc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f4fc80d1070 con 0x7f4fd4102f10 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fd37fe700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] conn(0x7f4fbc0ae090 0x7f4fbc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4fc80056b0 con 0x7f4fbc0ae090 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.848+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f4fc80d14c0 con 0x7f4fd4102f10 2024-06-06T06:59:15.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.849+0000 7f4fdabeb700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] conn(0x7f4fbc0ae090 0x7f4fbc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:15.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.849+0000 7f4fda3ea700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4fbc0a2d30 0x7f4fbc0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005950 tx=0x7f4fcc005ed0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:15.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.849+0000 7f4fdabeb700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] conn(0x7f4fbc0ae090 0x7f4fbc0b0530 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:15.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.850+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== osd.10 v2:172.21.15.136:6824/2185418171 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4fc80056b0 con 0x7f4fbc0ae090 2024-06-06T06:59:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.866+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 --> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4fd4062ff0 con 0x7f4fbc0ae090 2024-06-06T06:59:15.867 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fd37fe700 1 -- 172.21.15.73:0/4223753585 <== osd.10 v2:172.21.15.136:6824/2185418171 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4fd4062ff0 con 0x7f4fbc0ae090 2024-06-06T06:59:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] conn(0x7f4fbc0ae090 msgr2=0x7f4fbc0b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] conn(0x7f4fbc0ae090 0x7f4fbc0b0530 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4fbc0a2d30 msgr2=0x7f4fbc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4fbc0a2d30 0x7f4fbc0a51f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc005950 tx=0x7f4fcc005ed0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 msgr2=0x7f4fd419edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 --2- 172.21.15.73:0/4223753585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fd4102f10 0x7f4fd419edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009fd0 tx=0x7f4fc80037a0 comp rx=0 tx=0).stop 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdabeb700 1 -- 172.21.15.73:0/4223753585 reap_dead start 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 shutdown_connections 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 >> 172.21.15.73:0/4223753585 conn(0x7f4fd4079650 msgr2=0x7f4fd41006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 shutdown_connections 2024-06-06T06:59:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:15.867+0000 7f4fdc64e700 1 -- 172.21.15.73:0/4223753585 wait complete. 2024-06-06T06:59:16.772 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:16 smithi073 ceph-mon[32303]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:16.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:16 smithi136 ceph-mon[35901]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:16 smithi170 ceph-mon[36108]: pgmap v372: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 -- 172.21.15.73:0/281742458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c107840 msgr2=0x7f401c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 --2- 172.21.15.73:0/281742458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c107840 0x7f401c10f3c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f401400b0d0 tx=0x7f401400b3e0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 -- 172.21.15.73:0/281742458 shutdown_connections 2024-06-06T06:59:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 --2- 172.21.15.73:0/281742458 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c107840 0x7f401c10f3c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 --2- 172.21.15.73:0/281742458 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c102740 0x7f401c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 --2- 172.21.15.73:0/281742458 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f401c1030f0 0x7f401c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 -- 172.21.15.73:0/281742458 >> 172.21.15.73:0/281742458 conn(0x7f401c0795f0 msgr2=0x7f401c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 -- 172.21.15.73:0/281742458 shutdown_connections 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.194+0000 7f4020f1e700 1 -- 172.21.15.73:0/281742458 wait complete. 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 Processor -- start 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 -- start start 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f401c102740 0x7f401c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 0x7f401c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c107840 0x7f401c198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f401c112630 con 0x7f401c1030f0 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f401c1124b0 con 0x7f401c107840 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4020f1e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f401c1127b0 con 0x7f401c102740 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 0x7f401c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 0x7f401c19ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46390/0 (socket says 172.21.15.73:46390) 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 -- 172.21.15.73:0/996160461 learned_addr learned my addr 172.21.15.73:0/996160461 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f401a59c700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f401c102740 0x7f401c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 -- 172.21.15.73:0/996160461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f401c102740 msgr2=0x7f401c19e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f401c102740 0x7f401c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 -- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c107840 msgr2=0x7f401c198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f401ad9d700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c107840 0x7f401c198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f401c107840 0x7f401c198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.195+0000 7f4019d9b700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401c198fd0 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f4019d9b700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 0x7f401c19ebf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f401000c970 tx=0x7f401000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010007aa0 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4010004ce0 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f401c1992c0 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4010005620 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f401c1a5580 con 0x7f401c1030f0 2024-06-06T06:59:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.196+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f401c062be0 con 0x7f401c1030f0 2024-06-06T06:59:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.198+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f4010004750 con 0x7f401c1030f0 2024-06-06T06:59:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.198+0000 7f400b7fe700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f40040a2cb0 0x7f40040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f401a59c700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f40040a2cb0 0x7f40040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f40100cf860 con 0x7f401c1030f0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f400b7fe700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] conn(0x7f40040ae010 0x7f40040b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f40040b0b80 con 0x7f40040ae010 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f40100cfcb0 con 0x7f401c1030f0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f401a59c700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f40040a2cb0 0x7f40040a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f400c005950 tx=0x7f400c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f401ad9d700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] conn(0x7f40040ae010 0x7f40040b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.199+0000 7f401ad9d700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] conn(0x7f40040ae010 0x7f40040b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.200+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== osd.1 v2:172.21.15.136:6800/2860362689 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f40040b0b80 con 0x7f40040ae010 2024-06-06T06:59:17.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.218+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 --> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f401c063060 con 0x7f40040ae010 2024-06-06T06:59:17.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f400b7fe700 1 -- 172.21.15.73:0/996160461 <== osd.1 v2:172.21.15.136:6800/2860362689 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f401c063060 con 0x7f40040ae010 2024-06-06T06:59:17.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] conn(0x7f40040ae010 msgr2=0x7f40040b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] conn(0x7f40040ae010 0x7f40040b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f40040a2cb0 msgr2=0x7f40040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f40040a2cb0 0x7f40040a5170 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f400c005950 tx=0x7f400c0058e0 comp rx=0 tx=0).stop 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 msgr2=0x7f401c19ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 --2- 172.21.15.73:0/996160461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f401c1030f0 0x7f401c19ebf0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f401000c970 tx=0x7f401000cd30 comp rx=0 tx=0).stop 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f401ad9d700 1 -- 172.21.15.73:0/996160461 reap_dead start 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 shutdown_connections 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 >> 172.21.15.73:0/996160461 conn(0x7f401c0795f0 msgr2=0x7f401c100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 shutdown_connections 2024-06-06T06:59:17.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:17.219+0000 7f4020f1e700 1 -- 172.21.15.73:0/996160461 wait complete. 2024-06-06T06:59:18.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:18 smithi073 ceph-mon[32303]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-06T06:59:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:18 smithi136 ceph-mon[35901]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-06T06:59:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:18 smithi170 ceph-mon[36108]: pgmap v373: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 10 op/s 2024-06-06T06:59:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 -- 172.21.15.73:0/1636342203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc1073b0 msgr2=0x7faafc06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 --2- 172.21.15.73:0/1636342203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc1073b0 0x7faafc06aa30 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b0a0 tx=0x7faaec00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 -- 172.21.15.73:0/1636342203 shutdown_connections 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 --2- 172.21.15.73:0/1636342203 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faafc104ec0 0x7faafc105350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 --2- 172.21.15.73:0/1636342203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc1073b0 0x7faafc06aa30 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 --2- 172.21.15.73:0/1636342203 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc06af70 0x7faafc06b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.232+0000 7fab0417c700 1 -- 172.21.15.73:0/1636342203 >> 172.21.15.73:0/1636342203 conn(0x7faafc078300 msgr2=0x7faafc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- 172.21.15.73:0/1636342203 shutdown_connections 2024-06-06T06:59:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- 172.21.15.73:0/1636342203 wait complete. 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 Processor -- start 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- start start 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faafc06af70 0x7faafc19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 0x7faafc19f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc1073b0 0x7faafc1a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faafc118c70 con 0x7faafc104ec0 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7faafc118af0 con 0x7faafc06af70 2024-06-06T06:59:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.233+0000 7fab0417c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7faafc118df0 con 0x7faafc1073b0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 0x7faafc19f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 0x7faafc19f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46402/0 (socket says 172.21.15.73:46402) 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 -- 172.21.15.73:0/762252092 learned_addr learned my addr 172.21.15.73:0/762252092 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab02719700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc1073b0 0x7faafc1a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01f18700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faafc06af70 0x7faafc19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 -- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc1073b0 msgr2=0x7faafc1a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc1073b0 0x7faafc1a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 -- 172.21.15.73:0/762252092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faafc06af70 msgr2=0x7faafc19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7faafc06af70 0x7faafc19f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc1a4150 con 0x7faafc104ec0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab02719700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7faafc1073b0 0x7faafc1a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab01717700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 0x7faafc19f680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b530 tx=0x7faaf000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0015020 con 0x7faafc104ec0 2024-06-06T06:59:19.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaf0011ce0 con 0x7faafc104ec0 2024-06-06T06:59:19.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc1a4440 con 0x7faafc104ec0 2024-06-06T06:59:19.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0010d60 con 0x7faafc104ec0 2024-06-06T06:59:19.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.234+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1abcc0 con 0x7faafc104ec0 2024-06-06T06:59:19.238 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.235+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faafc100dd0 con 0x7faafc104ec0 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.236+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faaf0018020 con 0x7faafc104ec0 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7faafaffd700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7faae40a2cb0 0x7faae40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7faaf00cf490 con 0x7faafc104ec0 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7fab01f18700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7faae40a2cb0 0x7faae40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7faafaffd700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] conn(0x7faae40ae010 0x7faae40b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faaf0018ec0 con 0x7faae40ae010 2024-06-06T06:59:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7faaf00cf8e0 con 0x7faafc104ec0 2024-06-06T06:59:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7fab01f18700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7faae40a2cb0 0x7faae40a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b070 tx=0x7faaec0060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.237+0000 7fab02719700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] conn(0x7faae40ae010 0x7faae40b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.238+0000 7fab02719700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] conn(0x7faae40ae010 0x7faae40b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.239+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== osd.5 v2:172.21.15.170:6808/686557154 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faaf0018ec0 con 0x7faae40ae010 2024-06-06T06:59:19.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.255+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 --> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faafc1019a0 con 0x7faae40ae010 2024-06-06T06:59:19.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7faafaffd700 1 -- 172.21.15.73:0/762252092 <== osd.5 v2:172.21.15.170:6808/686557154 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faafc1019a0 con 0x7faae40ae010 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] conn(0x7faae40ae010 msgr2=0x7faae40b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] conn(0x7faae40ae010 0x7faae40b04b0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7faae40a2cb0 msgr2=0x7faae40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7faae40a2cb0 0x7faae40a5170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7faaec00b070 tx=0x7faaec0060c0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 msgr2=0x7faafc19f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:19.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 --2- 172.21.15.73:0/762252092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faafc104ec0 0x7faafc19f680 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7faaf000b530 tx=0x7faaf000b8f0 comp rx=0 tx=0).stop 2024-06-06T06:59:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab02719700 1 -- 172.21.15.73:0/762252092 reap_dead start 2024-06-06T06:59:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 shutdown_connections 2024-06-06T06:59:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 >> 172.21.15.73:0/762252092 conn(0x7faafc078300 msgr2=0x7faafc100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 shutdown_connections 2024-06-06T06:59:19.258 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:19.256+0000 7fab0417c700 1 -- 172.21.15.73:0/762252092 wait complete. 2024-06-06T06:59:20.464 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:20 smithi073 ceph-mon[32303]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T06:59:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:20 smithi136 ceph-mon[35901]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T06:59:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:20 smithi170 ceph-mon[36108]: pgmap v374: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-06-06T06:59:21.202 INFO:teuthology.orchestra.run.smithi073.stdout:167503724616 2024-06-06T06:59:21.202 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-06T06:59:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 -- 172.21.15.73:0/3414467270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc06a170 msgr2=0x7f23cc06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 --2- 172.21.15.73:0/3414467270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc06a170 0x7f23cc06a5f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009990 tx=0x7f23bc009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 -- 172.21.15.73:0/3414467270 shutdown_connections 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 --2- 172.21.15.73:0/3414467270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23cc06ab30 0x7f23cc10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 --2- 172.21.15.73:0/3414467270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc108500 0x7f23cc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 --2- 172.21.15.73:0/3414467270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc06a170 0x7f23cc06a5f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 -- 172.21.15.73:0/3414467270 >> 172.21.15.73:0/3414467270 conn(0x7f23cc078340 msgr2=0x7f23cc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 -- 172.21.15.73:0/3414467270 shutdown_connections 2024-06-06T06:59:21.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.891+0000 7f23d33df700 1 -- 172.21.15.73:0/3414467270 wait complete. 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 Processor -- start 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 -- start start 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc06a170 0x7f23cc19e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23cc06ab30 0x7f23cc19ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 0x7f23cc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112710 con 0x7f23cc108500 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112590 con 0x7f23cc06a170 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d33df700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f23cc112890 con 0x7f23cc06ab30 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d197c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 0x7f23cc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:21.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d197c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 0x7f23cc198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46426/0 (socket says 172.21.15.73:46426) 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d197c700 1 -- 172.21.15.73:0/823577722 learned_addr learned my addr 172.21.15.73:0/823577722 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.892+0000 7f23d117b700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc06a170 0x7f23cc19e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d097a700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23cc06ab30 0x7f23cc19ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 -- 172.21.15.73:0/823577722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23cc06ab30 msgr2=0x7f23cc19ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f23cc06ab30 0x7f23cc19ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 -- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc06a170 msgr2=0x7f23cc19e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f23cc06a170 0x7f23cc19e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23cc199170 con 0x7f23cc108500 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d197c700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 0x7f23cc198a10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b970 tx=0x7f23c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4019070 con 0x7f23cc108500 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23c400cde0 con 0x7f23cc108500 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23cc199460 con 0x7f23cc108500 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23c4007370 con 0x7f23cc108500 2024-06-06T06:59:21.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.893+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23cc1a5720 con 0x7f23cc108500 2024-06-06T06:59:21.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.894+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23cc062c20 con 0x7f23cc108500 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.895+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f23c401c070 con 0x7f23cc108500 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23c27fc700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f23b80a2d30 0x7f23b80a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23d117b700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f23b80a2d30 0x7f23b80a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f23c40cf2e0 con 0x7f23cc108500 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23c27fc700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] conn(0x7f23b80ae090 0x7f23b80b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23c401ce50 con 0x7f23b80ae090 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f23c40cf730 con 0x7f23cc108500 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.896+0000 7f23d117b700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f23b80a2d30 0x7f23b80a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f23cc199b50 tx=0x7f23c8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.897+0000 7f23d097a700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] conn(0x7f23b80ae090 0x7f23b80b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:21.898 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.897+0000 7f23d097a700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] conn(0x7f23b80ae090 0x7f23b80b0530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:21.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.898+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== osd.4 v2:172.21.15.136:6808/846852962 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23c401ce50 con 0x7f23b80ae090 2024-06-06T06:59:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.920+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 --> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f23cc063020 con 0x7f23b80ae090 2024-06-06T06:59:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23c27fc700 1 -- 172.21.15.73:0/823577722 <== osd.4 v2:172.21.15.136:6808/846852962 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f23cc063020 con 0x7f23b80ae090 2024-06-06T06:59:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] conn(0x7f23b80ae090 msgr2=0x7f23b80b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] conn(0x7f23b80ae090 0x7f23b80b0530 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f23b80a2d30 msgr2=0x7f23b80a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f23b80a2d30 0x7f23b80a51f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f23cc199b50 tx=0x7f23c8009190 comp rx=0 tx=0).stop 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 msgr2=0x7f23cc198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 --2- 172.21.15.73:0/823577722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f23cc108500 0x7f23cc198a10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f23c400b970 tx=0x7f23c400bc80 comp rx=0 tx=0).stop 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d197c700 1 -- 172.21.15.73:0/823577722 reap_dead start 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 shutdown_connections 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 >> 172.21.15.73:0/823577722 conn(0x7f23cc078340 msgr2=0x7f23cc100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 shutdown_connections 2024-06-06T06:59:21.923 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.921+0000 7f23d33df700 1 -- 172.21.15.73:0/823577722 wait complete. 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 -- 172.21.15.73:0/3889864686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea80102f10 msgr2=0x7fea8010f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 --2- 172.21.15.73:0/3889864686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea80102f10 0x7fea8010f330 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fea7800b0a0 tx=0x7fea7800b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 -- 172.21.15.73:0/3889864686 shutdown_connections 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 --2- 172.21.15.73:0/3889864686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea801084f0 0x7fea801088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 --2- 172.21.15.73:0/3889864686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea80102f10 0x7fea8010f330 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 --2- 172.21.15.73:0/3889864686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80102550 0x7fea801029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 -- 172.21.15.73:0/3889864686 >> 172.21.15.73:0/3889864686 conn(0x7fea80079650 msgr2=0x7fea80079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:22.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 -- 172.21.15.73:0/3889864686 shutdown_connections 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.779+0000 7fea86305700 1 -- 172.21.15.73:0/3889864686 wait complete. 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 Processor -- start 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 -- start start 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80102550 0x7fea80074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80102f10 0x7fea80074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 0x7fea80075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea80112620 con 0x7fea801084f0 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea801124a0 con 0x7fea80102550 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea86305700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea801127a0 con 0x7fea80102f10 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea848a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 0x7fea80075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.780+0000 7fea848a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 0x7fea80075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46408/0 (socket says 172.21.15.73:46408) 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 -- 172.21.15.73:0/677035412 learned_addr learned my addr 172.21.15.73:0/677035412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:22.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7f7fe700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80102f10 0x7fea80074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7ffff700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80102550 0x7fea80074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 -- 172.21.15.73:0/677035412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80102f10 msgr2=0x7fea80074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80102f10 0x7fea80074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 -- 172.21.15.73:0/677035412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80102550 msgr2=0x7fea80074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea80102550 0x7fea80074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea801a6f80 con 0x7fea801084f0 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7f7fe700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea80102f10 0x7fea80074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea848a2700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 0x7fea80075210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea780062a0 tx=0x7fea78004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:22.023 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78021070 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea801a7210 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea78007d00 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7801d450 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.781+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea801a7710 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.782+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea80062ba0 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.783+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fea78003a00 con 0x7fea801084f0 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7d7fa700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea6c0a2cb0 0x7fea6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:22.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7ffff700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea6c0a2cb0 0x7fea6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fea7800f020 con 0x7fea801084f0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7d7fa700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] conn(0x7fea6c0ae010 0x7fea6c0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fea7800aea0 con 0x7fea6c0ae010 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fea780d1580 con 0x7fea801084f0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7f7fe700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] conn(0x7fea6c0ae010 0x7fea6c0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.784+0000 7fea7ffff700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea6c0a2cb0 0x7fea6c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea68005950 tx=0x7fea68005c70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.785+0000 7fea7f7fe700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] conn(0x7fea6c0ae010 0x7fea6c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.785+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== osd.3 v2:172.21.15.73:6810/3983384250 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fea7800aea0 con 0x7fea6c0ae010 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.802+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 --> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea80062fa0 con 0x7fea6c0ae010 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.802+0000 7fea7d7fa700 1 -- 172.21.15.73:0/677035412 <== osd.3 v2:172.21.15.73:6810/3983384250 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fea80062fa0 con 0x7fea6c0ae010 2024-06-06T06:59:22.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.802+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] conn(0x7fea6c0ae010 msgr2=0x7fea6c0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.802+0000 7fea86305700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] conn(0x7fea6c0ae010 0x7fea6c0b04b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea6c0a2cb0 msgr2=0x7fea6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fea6c0a2cb0 0x7fea6c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fea68005950 tx=0x7fea68005c70 comp rx=0 tx=0).stop 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 msgr2=0x7fea80075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 --2- 172.21.15.73:0/677035412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea801084f0 0x7fea80075210 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fea780062a0 tx=0x7fea78004ed0 comp rx=0 tx=0).stop 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea848a2700 1 -- 172.21.15.73:0/677035412 reap_dead start 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 shutdown_connections 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 >> 172.21.15.73:0/677035412 conn(0x7fea80079650 msgr2=0x7fea80100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 shutdown_connections 2024-06-06T06:59:22.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:21.803+0000 7fea86305700 1 -- 172.21.15.73:0/677035412 wait complete. 2024-06-06T06:59:22.601 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:22 smithi073 ceph-mon[32303]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:22.630 INFO:teuthology.orchestra.run.smithi073.stdout:111669149779 2024-06-06T06:59:22.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-06T06:59:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:22 smithi136 ceph-mon[35901]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:22 smithi170 ceph-mon[36108]: pgmap v375: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:23.618 INFO:teuthology.orchestra.run.smithi073.stdout:146028888143 2024-06-06T06:59:23.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-06T06:59:23.752 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:24.305 INFO:teuthology.orchestra.run.smithi073.stdout:313532612655 2024-06-06T06:59:24.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.496+0000 7f3bf4b46700 1 -- 172.21.15.73:0/253421219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101380 msgr2=0x7f3bf0101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/253421219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101380 0x7f3bf0101800 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8009990 tx=0x7f3bd8009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 -- 172.21.15.73:0/253421219 shutdown_connections 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/253421219 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf0101d40 0x7f3bf010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/253421219 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bf01073f0 0x7f3bf01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/253421219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101380 0x7f3bf0101800 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 -- 172.21.15.73:0/253421219 >> 172.21.15.73:0/253421219 conn(0x7f3bf0078300 msgr2=0x7f3bf0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:24.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 -- 172.21.15.73:0/253421219 shutdown_connections 2024-06-06T06:59:24.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.497+0000 7f3bf4b46700 1 -- 172.21.15.73:0/253421219 wait complete. 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 Processor -- start 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 -- start start 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bf0101380 0x7f3bf0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 0x7f3bf0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01073f0 0x7f3bf01a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf0111540 con 0x7f3bf0101d40 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01113c0 con 0x7f3bf0101380 2024-06-06T06:59:24.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3bf4b46700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf01116c0 con 0x7f3bf01073f0 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3be7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 0x7f3bf0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3be7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 0x7f3bf0199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46446/0 (socket says 172.21.15.73:46446) 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.498+0000 7f3be7fff700 1 -- 172.21.15.73:0/547777061 learned_addr learned my addr 172.21.15.73:0/547777061 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3beed9d700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01073f0 0x7f3bf01a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3bee59c700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bf0101380 0x7f3bf0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 -- 172.21.15.73:0/547777061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01073f0 msgr2=0x7f3bf01a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3bf01073f0 0x7f3bf01a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 -- 172.21.15.73:0/547777061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bf0101380 msgr2=0x7f3bf0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3bf0101380 0x7f3bf0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf01a4fc0 con 0x7f3bf0101d40 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be7fff700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 0x7f3bf0199f40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8000c00 tx=0x7f3bd8003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8027070 con 0x7f3bf0101d40 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf01a5190 con 0x7f3bf0101d40 2024-06-06T06:59:24.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3bd800bdc0 con 0x7f3bf0101d40 2024-06-06T06:59:24.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8022cd0 con 0x7f3bf0101d40 2024-06-06T06:59:24.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.499+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf01a55e0 con 0x7f3bf0101d40 2024-06-06T06:59:24.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.500+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3bf0062b60 con 0x7f3bf0101d40 2024-06-06T06:59:24.502 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.502+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3bd802a070 con 0x7f3bf0101d40 2024-06-06T06:59:24.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.502+0000 7f3be77fe700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3bdc0a2cb0 0x7f3bdc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:24.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.502+0000 7f3bee59c700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3bdc0a2cb0 0x7f3bdc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:24.503 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.502+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f3bd80d0890 con 0x7f3bf0101d40 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3be77fe700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] conn(0x7f3bdc0ae010 0x7f3bdc0b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3bd80095d0 con 0x7f3bdc0ae010 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f3bd80d0ce0 con 0x7f3bf0101d40 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3beed9d700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] conn(0x7f3bdc0ae010 0x7f3bdc0b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3bee59c700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3bdc0a2cb0 0x7f3bdc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3be0005950 tx=0x7f3be000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.503+0000 7f3beed9d700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] conn(0x7f3bdc0ae010 0x7f3bdc0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:24.504 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.504+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== osd.9 v2:172.21.15.73:6826/2353282105 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3bd80095d0 con 0x7f3bdc0ae010 2024-06-06T06:59:24.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 --> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3bf0062f60 con 0x7f3bdc0ae010 2024-06-06T06:59:24.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3be77fe700 1 -- 172.21.15.73:0/547777061 <== osd.9 v2:172.21.15.73:6826/2353282105 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3bf0062f60 con 0x7f3bdc0ae010 2024-06-06T06:59:24.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] conn(0x7f3bdc0ae010 msgr2=0x7f3bdc0b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] conn(0x7f3bdc0ae010 0x7f3bdc0b04b0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3bdc0a2cb0 msgr2=0x7f3bdc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f3bdc0a2cb0 0x7f3bdc0a5170 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3be0005950 tx=0x7f3be000b410 comp rx=0 tx=0).stop 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 msgr2=0x7f3bf0199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3bf4b46700 1 --2- 172.21.15.73:0/547777061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bf0101d40 0x7f3bf0199f40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8000c00 tx=0x7f3bd8003a80 comp rx=0 tx=0).stop 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.521+0000 7f3beed9d700 1 -- 172.21.15.73:0/547777061 reap_dead start 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.522+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 shutdown_connections 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.522+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 >> 172.21.15.73:0/547777061 conn(0x7f3bf0078300 msgr2=0x7f3bf0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.522+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 shutdown_connections 2024-06-06T06:59:24.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:24.522+0000 7f3bf4b46700 1 -- 172.21.15.73:0/547777061 wait complete. 2024-06-06T06:59:24.693 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:24 smithi073 ceph-mon[32303]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:24 smithi136 ceph-mon[35901]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:24.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:24 smithi170 ceph-mon[36108]: pgmap v376: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:25.035 INFO:teuthology.orchestra.run.smithi073.stdout:55834574940 2024-06-06T06:59:25.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- 172.21.15.73:0/2135354347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c1066d0 msgr2=0x7fab8c10e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 --2- 172.21.15.73:0/2135354347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c1066d0 0x7fab8c10e2a0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fab8400b0d0 tx=0x7fab8400b3e0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- 172.21.15.73:0/2135354347 shutdown_connections 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 --2- 172.21.15.73:0/2135354347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c1066d0 0x7fab8c10e2a0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 --2- 172.21.15.73:0/2135354347 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c101680 0x7fab8c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 --2- 172.21.15.73:0/2135354347 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab8c102030 0x7fab8c105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- 172.21.15.73:0/2135354347 >> 172.21.15.73:0/2135354347 conn(0x7fab8c0773e0 msgr2=0x7fab8c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- 172.21.15.73:0/2135354347 shutdown_connections 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- 172.21.15.73:0/2135354347 wait complete. 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 Processor -- start 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.658+0000 7fab933cb700 1 -- start start 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 0x7fab8c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab8c102030 0x7fab8c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c1066d0 0x7fab8c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c1115e0 con 0x7fab8c101680 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111460 con 0x7fab8c1066d0 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab8c111760 con 0x7fab8c102030 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 0x7fab8c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 0x7fab8c19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39882/0 (socket says 172.21.15.73:39882) 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 -- 172.21.15.73:0/1834571958 learned_addr learned my addr 172.21.15.73:0/1834571958 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:25.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91968700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c1066d0 0x7fab8c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab90966700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab8c102030 0x7fab8c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 -- 172.21.15.73:0/1834571958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab8c102030 msgr2=0x7fab8c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab8c102030 0x7fab8c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 -- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c1066d0 msgr2=0x7fab8c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c1066d0 0x7fab8c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab8c199120 con 0x7fab8c101680 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91968700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab8c1066d0 0x7fab8c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab91167700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 0x7fab8c19e7b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab8800d950 tx=0x7fab8800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab88016070 con 0x7fab8c101680 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab8c199410 con 0x7fab8c101680 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab88010460 con 0x7fab8c101680 2024-06-06T06:59:25.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8800f620 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.659+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8c1a5680 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.660+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab8c06af50 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fab88019070 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab780a2cb0 0x7fab780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab90966700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab780a2cb0 0x7fab780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fab880cf720 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] conn(0x7fab780ae010 0x7fab780b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab780b0b80 con 0x7fab780ae010 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.662+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7fab880cfb70 con 0x7fab8c101680 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.663+0000 7fab90966700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab780a2cb0 0x7fab780a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.663+0000 7fab91968700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] conn(0x7fab780ae010 0x7fab780b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:25.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.663+0000 7fab91968700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] conn(0x7fab780ae010 0x7fab780b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:25.668 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.664+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== osd.7 v2:172.21.15.136:6816/4094155747 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab780b0b80 con 0x7fab780ae010 2024-06-06T06:59:25.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.681+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 --> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab8c102ad0 con 0x7fab780ae010 2024-06-06T06:59:25.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab827fc700 1 -- 172.21.15.73:0/1834571958 <== osd.7 v2:172.21.15.136:6816/4094155747 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab8c102ad0 con 0x7fab780ae010 2024-06-06T06:59:25.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] conn(0x7fab780ae010 msgr2=0x7fab780b04b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.683 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] conn(0x7fab780ae010 0x7fab780b04b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab780a2cb0 msgr2=0x7fab780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab780a2cb0 0x7fab780a5170 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fab7c006fd0 tx=0x7fab7c008040 comp rx=0 tx=0).stop 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 msgr2=0x7fab8c19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 --2- 172.21.15.73:0/1834571958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab8c101680 0x7fab8c19e7b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fab8800d950 tx=0x7fab8800dd10 comp rx=0 tx=0).stop 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab91968700 1 -- 172.21.15.73:0/1834571958 reap_dead start 2024-06-06T06:59:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 shutdown_connections 2024-06-06T06:59:25.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 >> 172.21.15.73:0/1834571958 conn(0x7fab8c0773e0 msgr2=0x7fab8c105310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:25.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 shutdown_connections 2024-06-06T06:59:25.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:25.682+0000 7fab933cb700 1 -- 172.21.15.73:0/1834571958 wait complete. 2024-06-06T06:59:26.744 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:26.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:26 smithi073 ceph-mon[32303]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:26.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:26 smithi136 ceph-mon[35901]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:26.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:26 smithi170 ceph-mon[36108]: pgmap v377: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 10 op/s 2024-06-06T06:59:27.206 INFO:teuthology.orchestra.run.smithi073.stdout:223338299456 2024-06-06T06:59:27.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-06T06:59:27.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 -- 172.21.15.73:0/2057943084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788101440 msgr2=0x7f778810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 --2- 172.21.15.73:0/2057943084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788101440 0x7f778810f330 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f778000b0a0 tx=0x7f778000b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 -- 172.21.15.73:0/2057943084 shutdown_connections 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 --2- 172.21.15.73:0/2057943084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788101440 0x7f778810f330 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 --2- 172.21.15.73:0/2057943084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7788108510 0x7f77881088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 --2- 172.21.15.73:0/2057943084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7788100a80 0x7f7788100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 -- 172.21.15.73:0/2057943084 >> 172.21.15.73:0/2057943084 conn(0x7f7788077a00 msgr2=0x7f77880fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 -- 172.21.15.73:0/2057943084 shutdown_connections 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.416+0000 7f778cc8b700 1 -- 172.21.15.73:0/2057943084 wait complete. 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.417+0000 7f778cc8b700 1 Processor -- start 2024-06-06T06:59:27.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.417+0000 7f778cc8b700 1 -- start start 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.417+0000 7f778cc8b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 0x7f778819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.417+0000 7f778cc8b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7788101440 0x7f778819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.417+0000 7f778cc8b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7788108510 0x7f7788198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778cc8b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7788112640 con 0x7f7788100a80 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778cc8b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f77881124c0 con 0x7f7788108510 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778cc8b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f77881127c0 con 0x7f7788101440 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 0x7f778819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 0x7f778819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39902/0 (socket says 172.21.15.73:39902) 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 -- 172.21.15.73:0/1622688200 learned_addr learned my addr 172.21.15.73:0/1622688200 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f7786d9d700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7788108510 0x7f7788198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f7785d9b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7788101440 0x7f778819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 -- 172.21.15.73:0/1622688200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7788101440 msgr2=0x7f778819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7788101440 0x7f778819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 -- 172.21.15.73:0/1622688200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7788108510 msgr2=0x7f7788198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7788108510 0x7f7788198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77881990a0 con 0x7f7788100a80 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778659c700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 0x7f778819e730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f777800c920 tx=0x7f777800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778004a30 con 0x7f7788100a80 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77780054d0 con 0x7f7788100a80 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7788199390 con 0x7f7788100a80 2024-06-06T06:59:27.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7778007c30 con 0x7f7788100a80 2024-06-06T06:59:27.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.418+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77881a5600 con 0x7f7788100a80 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.419+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7788062c20 con 0x7f7788100a80 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.421+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f7778004bd0 con 0x7f7788100a80 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.421+0000 7f77777fe700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f77700a2d30 0x7f77700a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f7785d9b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f77700a2d30 0x7f77700a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f77780cf4c0 con 0x7f7788100a80 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f77777fe700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] conn(0x7f77700ae090 0x7f77700b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77700b0c00 con 0x7f77700ae090 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f77780cf910 con 0x7f7788100a80 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f7786d9d700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] conn(0x7f77700ae090 0x7f77700b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f7785d9b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f77700a2d30 0x7f77700a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788107c20 tx=0x7f777c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:27.446 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.422+0000 7f7786d9d700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] conn(0x7f77700ae090 0x7f77700b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.423+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== osd.0 v2:172.21.15.73:6802/3665933649 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77700b0c00 con 0x7f77700ae090 2024-06-06T06:59:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.440+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 --> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7788063020 con 0x7f77700ae090 2024-06-06T06:59:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f77777fe700 1 -- 172.21.15.73:0/1622688200 <== osd.0 v2:172.21.15.73:6802/3665933649 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7788063020 con 0x7f77700ae090 2024-06-06T06:59:27.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] conn(0x7f77700ae090 msgr2=0x7f77700b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] conn(0x7f77700ae090 0x7f77700b0530 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f77700a2d30 msgr2=0x7f77700a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f77700a2d30 0x7f77700a51f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7788107c20 tx=0x7f777c00a300 comp rx=0 tx=0).stop 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 msgr2=0x7f778819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f778cc8b700 1 --2- 172.21.15.73:0/1622688200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7788100a80 0x7f778819e730 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f777800c920 tx=0x7f777800cc30 comp rx=0 tx=0).stop 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.441+0000 7f7786d9d700 1 -- 172.21.15.73:0/1622688200 reap_dead start 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.442+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 shutdown_connections 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.442+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 >> 172.21.15.73:0/1622688200 conn(0x7f7788077a00 msgr2=0x7f77880ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.442+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 shutdown_connections 2024-06-06T06:59:27.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:27.442+0000 7f778cc8b700 1 -- 172.21.15.73:0/1622688200 wait complete. 2024-06-06T06:59:28.059 INFO:teuthology.orchestra.run.smithi073.stdout:34359738467 2024-06-06T06:59:28.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-06T06:59:28.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:28 smithi073 ceph-mon[32303]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-06T06:59:28.756 INFO:teuthology.orchestra.run.smithi073.stdout:292057776181 2024-06-06T06:59:28.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-06T06:59:28.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:28 smithi136 ceph-mon[35901]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-06T06:59:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:28 smithi170 ceph-mon[36108]: pgmap v378: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-06T06:59:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 -- 172.21.15.73:0/2127347424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 msgr2=0x7ff4a81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 --2- 172.21.15.73:0/2127347424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a81088f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 -- 172.21.15.73:0/2127347424 shutdown_connections 2024-06-06T06:59:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 --2- 172.21.15.73:0/2127347424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 0x7ff4a810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 --2- 172.21.15.73:0/2127347424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a81088f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 --2- 172.21.15.73:0/2127347424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 0x7ff4a8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 -- 172.21.15.73:0/2127347424 >> 172.21.15.73:0/2127347424 conn(0x7ff4a8077a00 msgr2=0x7ff4a80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 -- 172.21.15.73:0/2127347424 shutdown_connections 2024-06-06T06:59:29.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.785+0000 7ff4ae289700 1 -- 172.21.15.73:0/2127347424 wait complete. 2024-06-06T06:59:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 Processor -- start 2024-06-06T06:59:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 -- start start 2024-06-06T06:59:29.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 0x7ff4a81a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 0x7ff4a81a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a8112620 con 0x7ff4a8108510 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81124a0 con 0x7ff4a8101440 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ae289700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4a81127a0 con 0x7ff4a8100a80 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ac826700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ac826700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39916/0 (socket says 172.21.15.73:39916) 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.786+0000 7ff4ac826700 1 -- 172.21.15.73:0/903307682 learned_addr learned my addr 172.21.15.73:0/903307682 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4a77fe700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 0x7ff4a81a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4a7fff700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 0x7ff4a81a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 -- 172.21.15.73:0/903307682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 msgr2=0x7ff4a81a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:29.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 0x7ff4a81a4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 -- 172.21.15.73:0/903307682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 msgr2=0x7ff4a81a5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 0x7ff4a81a5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 -- 172.21.15.73:0/903307682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4a810ecd0 con 0x7ff4a8108510 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ac826700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007ee0 tx=0x7ff4a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a000dbd0 con 0x7ff4a8108510 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4a000dd70 con 0x7ff4a8108510 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0005990 con 0x7ff4a8108510 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4a810efc0 con 0x7ff4a8108510 2024-06-06T06:59:29.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.787+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4a81abc20 con 0x7ff4a8108510 2024-06-06T06:59:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.789+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4a8069180 con 0x7ff4a8108510 2024-06-06T06:59:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.789+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7ff4a001a020 con 0x7ff4a8108510 2024-06-06T06:59:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.790+0000 7ff4a57fa700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 0x7ff4900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.790+0000 7ff4a7fff700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 0x7ff4900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:29.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.790+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7ff4a00cec90 con 0x7ff4a8108510 2024-06-06T06:59:29.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.790+0000 7ff4a7fff700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 0x7ff4900a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff498006010 tx=0x7ff49801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:29.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.795+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4a00d4020 con 0x7ff4a8108510 2024-06-06T06:59:29.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.995+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7ff4a81abf10 con 0x7ff4a8108510 2024-06-06T06:59:29.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.995+0000 7ff4a57fa700 1 -- 172.21.15.73:0/903307682 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7ff4a0010070 con 0x7ff4a8108510 2024-06-06T06:59:29.997 INFO:teuthology.orchestra.run.smithi073.stdout:347892351020 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 msgr2=0x7ff4900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 0x7ff4900a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff498006010 tx=0x7ff49801a040 comp rx=0 tx=0).stop 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 msgr2=0x7ff4a810e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0007ee0 tx=0x7ff4a000d3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 shutdown_connections 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4900a2cb0 0x7ff4900a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4a8100a80 0x7ff4a81a4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:29.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a8108510 0x7ff4a810e500 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:30.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 --2- 172.21.15.73:0/903307682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4a8101440 0x7ff4a81a5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:30.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 >> 172.21.15.73:0/903307682 conn(0x7ff4a8077a00 msgr2=0x7ff4a80ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:30.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 shutdown_connections 2024-06-06T06:59:30.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:29.997+0000 7ff4ae289700 1 -- 172.21.15.73:0/903307682 wait complete. 2024-06-06T06:59:30.662 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:30 smithi073 ceph-mon[32303]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T06:59:30.662 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:30 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/903307682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T06:59:30.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:30 smithi136 ceph-mon[35901]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T06:59:30.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:30 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/903307682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T06:59:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:30 smithi170 ceph-mon[36108]: pgmap v379: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 8 op/s 2024-06-06T06:59:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:30 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/903307682' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.288+0000 7f645c472700 1 -- 172.21.15.73:0/2325272004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 msgr2=0x7f645410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.288+0000 7f645c472700 1 --2- 172.21.15.73:0/2325272004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645410f310 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f644c00b0a0 tx=0x7f644c00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 -- 172.21.15.73:0/2325272004 shutdown_connections 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 --2- 172.21.15.73:0/2325272004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645410f310 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 --2- 172.21.15.73:0/2325272004 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64541084f0 0x7f64541088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 --2- 172.21.15.73:0/2325272004 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64541009e0 0x7f6454100e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 -- 172.21.15.73:0/2325272004 >> 172.21.15.73:0/2325272004 conn(0x7f6454077960 msgr2=0x7f64540fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:31.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 -- 172.21.15.73:0/2325272004 shutdown_connections 2024-06-06T06:59:31.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 -- 172.21.15.73:0/2325272004 wait complete. 2024-06-06T06:59:31.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 Processor -- start 2024-06-06T06:59:31.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.289+0000 7f645c472700 1 -- start start 2024-06-06T06:59:31.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64541009e0 0x7f645419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:31.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:31.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64541084f0 0x7f64541989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:31.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6454112700 con 0x7f64541013a0 2024-06-06T06:59:31.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6454112580 con 0x7f64541084f0 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6454112880 con 0x7f64541009e0 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645419ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39946/0 (socket says 172.21.15.73:39946) 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 -- 172.21.15.73:0/991819731 learned_addr learned my addr 172.21.15.73:0/991819731 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645aa0f700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64541084f0 0x7f64541989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645a20e700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64541009e0 0x7f645419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 -- 172.21.15.73:0/991819731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64541009e0 msgr2=0x7f645419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f64541009e0 0x7f645419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 -- 172.21.15.73:0/991819731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64541084f0 msgr2=0x7f64541989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f64541084f0 0x7f64541989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6454199120 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f6459a0d700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645419ed40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f644400b530 tx=0x7f644400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444015020 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6444011ce0 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6454199410 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6444010e40 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.290+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64541a56d0 con 0x7f64541013a0 2024-06-06T06:59:31.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.291+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6454062c20 con 0x7f64541013a0 2024-06-06T06:59:31.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6444018020 con 0x7f64541013a0 2024-06-06T06:59:31.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f64400a2d30 0x7f64400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:31.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f645a20e700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f64400a2d30 0x7f64400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f64440cf3b0 con 0x7f64541013a0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] conn(0x7f64400ae090 0x7f64400b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6444019e70 con 0x7f64400ae090 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.293+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=100) ==== 24+0+0 (secure 0 0 0) 0x7f64440cf800 con 0x7f64541013a0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.294+0000 7f645aa0f700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] conn(0x7f64400ae090 0x7f64400b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.294+0000 7f645a20e700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f64400a2d30 0x7f64400a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6450006fd0 tx=0x7f6450008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.294+0000 7f645aa0f700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] conn(0x7f64400ae090 0x7f64400b0530 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:31.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.295+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== osd.6 v2:172.21.15.73:6818/135647786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6444019e70 con 0x7f64400ae090 2024-06-06T06:59:31.313 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.311+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 --> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f64540630a0 con 0x7f64400ae090 2024-06-06T06:59:31.314 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f644b7fe700 1 -- 172.21.15.73:0/991819731 <== osd.6 v2:172.21.15.73:6818/135647786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f64540630a0 con 0x7f64400ae090 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] conn(0x7f64400ae090 msgr2=0x7f64400b0530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] conn(0x7f64400ae090 0x7f64400b0530 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f64400a2d30 msgr2=0x7f64400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f64400a2d30 0x7f64400a51f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6450006fd0 tx=0x7f6450008040 comp rx=0 tx=0).stop 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 msgr2=0x7f645419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 --2- 172.21.15.73:0/991819731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64541013a0 0x7f645419ed40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f644400b530 tx=0x7f644400b840 comp rx=0 tx=0).stop 2024-06-06T06:59:31.315 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645aa0f700 1 -- 172.21.15.73:0/991819731 reap_dead start 2024-06-06T06:59:31.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 shutdown_connections 2024-06-06T06:59:31.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.312+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 >> 172.21.15.73:0/991819731 conn(0x7f6454077960 msgr2=0x7f64540ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:31.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.313+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 shutdown_connections 2024-06-06T06:59:31.316 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:31.313+0000 7f645c472700 1 -- 172.21.15.73:0/991819731 wait complete. 2024-06-06T06:59:31.546 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:32.402 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351017 got 347892351020 for osd.11 2024-06-06T06:59:32.402 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:32.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:32 smithi170 ceph-mon[36108]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T06:59:32.664 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:32 smithi073 ceph-mon[32303]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T06:59:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:32 smithi136 ceph-mon[35901]: pgmap v380: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-06T06:59:33.065 INFO:teuthology.orchestra.run.smithi073.stdout:201863462982 2024-06-06T06:59:33.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-06T06:59:33.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 -- 172.21.15.73:0/2259155301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89081027a0 msgr2=0x7f8908102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:33.468 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 --2- 172.21.15.73:0/2259155301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89081027a0 0x7f8908102b80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f88f80099c0 tx=0x7f88f8009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 -- 172.21.15.73:0/2259155301 shutdown_connections 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 --2- 172.21.15.73:0/2259155301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8908107870 0x7f890810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 --2- 172.21.15.73:0/2259155301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f89081027a0 0x7f8908102b80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 --2- 172.21.15.73:0/2259155301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 0x7f8908107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.463+0000 7f890e431700 1 -- 172.21.15.73:0/2259155301 >> 172.21.15.73:0/2259155301 conn(0x7f8908079570 msgr2=0x7f8908079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 -- 172.21.15.73:0/2259155301 shutdown_connections 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 -- 172.21.15.73:0/2259155301 wait complete. 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 Processor -- start 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 -- start start 2024-06-06T06:59:33.469 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 0x7f8908074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 0x7f8908074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.464+0000 7f890e431700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890e431700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8908112660 con 0x7f8908107870 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890e431700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f89081124e0 con 0x7f89081027a0 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890e431700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f89081127e0 con 0x7f8908103150 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f8907fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 0x7f8908074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:33.470 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39954/0 (socket says 172.21.15.73:39954) 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f8907fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 0x7f8908074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:36182/0 (socket says 172.21.15.73:36182) 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 -- 172.21.15.73:0/3292721960 learned_addr learned my addr 172.21.15.73:0/3292721960 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f89077fe700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 0x7f8908074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 -- 172.21.15.73:0/3292721960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 msgr2=0x7f8908074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 0x7f8908074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 -- 172.21.15.73:0/3292721960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 msgr2=0x7f8908074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 0x7f8908074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 -- 172.21.15.73:0/3292721960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89081a6f90 con 0x7f8908107870 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890c9ce700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8900007d70 tx=0x7f890000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8900016020 con 0x7f8908107870 2024-06-06T06:59:33.471 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f890000ad40 con 0x7f8908107870 2024-06-06T06:59:33.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89081a7280 con 0x7f8908107870 2024-06-06T06:59:33.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89000115e0 con 0x7f8908107870 2024-06-06T06:59:33.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.465+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89081a7810 con 0x7f8908107870 2024-06-06T06:59:33.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.466+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f890810aab0 con 0x7f8908107870 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.467+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8900019020 con 0x7f8908107870 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.468+0000 7f89057fa700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 0x7f88f00a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.468+0000 7f8907fff700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 0x7f88f00a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.468+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f89000cf710 con 0x7f8908107870 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.469+0000 7f8907fff700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 0x7f88f00a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:33.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.472+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8900020020 con 0x7f8908107870 2024-06-06T06:59:33.685 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.683+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f890810a490 con 0x7f8908107870 2024-06-06T06:59:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.684+0000 7f89057fa700 1 -- 172.21.15.73:0/3292721960 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f890009c250 con 0x7f8908107870 2024-06-06T06:59:33.686 INFO:teuthology.orchestra.run.smithi073.stdout:77309411419 2024-06-06T06:59:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 msgr2=0x7f88f00a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 0x7f88f00a51f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f88f8000c00 tx=0x7f88f8005fb0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 msgr2=0x7f8908075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f8900007d70 tx=0x7f890000a340 comp rx=0 tx=0).stop 2024-06-06T06:59:33.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 shutdown_connections 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8908107870 0x7f8908075210 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f89081027a0 0x7f8908074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f88f00a2d30 0x7f88f00a51f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 --2- 172.21.15.73:0/3292721960 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8908103150 0x7f8908074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 >> 172.21.15.73:0/3292721960 conn(0x7f8908079570 msgr2=0x7f8908101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 shutdown_connections 2024-06-06T06:59:33.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:33.686+0000 7f890e431700 1 -- 172.21.15.73:0/3292721960 wait complete. 2024-06-06T06:59:34.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:34 smithi073 ceph-mon[32303]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:34.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:34 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3292721960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T06:59:34.649 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.675 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411414 got 77309411419 for osd.2 2024-06-06T06:59:34.676 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:34.678 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.679 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.681 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.684 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:34 smithi136 ceph-mon[35901]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:34.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:34 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3292721960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T06:59:34.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:34 smithi170 ceph-mon[36108]: pgmap v381: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:34.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:34 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3292721960' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-06T06:59:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:36.706 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:36 smithi073 ceph-mon[32303]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:36 smithi136 ceph-mon[35901]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:36 smithi170 ceph-mon[36108]: pgmap v382: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:37.333 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:38.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:38 smithi073 ceph-mon[32303]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-06T06:59:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:38 smithi136 ceph-mon[35901]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-06T06:59:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:38 smithi170 ceph-mon[36108]: pgmap v383: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-06T06:59:40.171 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 -- 172.21.15.73:0/1761002550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 msgr2=0x7fe1541077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 --2- 172.21.15.73:0/1761002550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541077b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe14c00b0a0 tx=0x7fe14c00b3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 -- 172.21.15.73:0/1761002550 shutdown_connections 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 --2- 172.21.15.73:0/1761002550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 0x7fe15410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 --2- 172.21.15.73:0/1761002550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541077b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 --2- 172.21.15.73:0/1761002550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 0x7fe1541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 -- 172.21.15.73:0/1761002550 >> 172.21.15.73:0/1761002550 conn(0x7fe1540782c0 msgr2=0x7fe1540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 -- 172.21.15.73:0/1761002550 shutdown_connections 2024-06-06T06:59:40.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.170+0000 7fe15b321700 1 -- 172.21.15.73:0/1761002550 wait complete. 2024-06-06T06:59:40.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 Processor -- start 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 -- start start 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 0x7fe154199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 0x7fe154199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1541114c0 con 0x7fe1541073d0 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111340 con 0x7fe154101390 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe15b321700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe154111640 con 0x7fe154101d30 2024-06-06T06:59:40.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.171+0000 7fe1598be700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51190/0 (socket says 172.21.15.73:51190) 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 -- 172.21.15.73:0/3961989829 learned_addr learned my addr 172.21.15.73:0/3961989829 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1590bd700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 0x7fe154199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1588bc700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 0x7fe154199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 -- 172.21.15.73:0/3961989829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 msgr2=0x7fe154199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 0x7fe154199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 -- 172.21.15.73:0/3961989829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 msgr2=0x7fe154199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 0x7fe154199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 -- 172.21.15.73:0/3961989829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1541a4ec0 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1598be700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c900 tx=0x7fe14400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144007b00 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe14400ce50 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1541a50f0 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe144018660 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.172+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1541a5600 con 0x7fe1541073d0 2024-06-06T06:59:40.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.173+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe154069180 con 0x7fe1541073d0 2024-06-06T06:59:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.175+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe14400f450 con 0x7fe1541073d0 2024-06-06T06:59:40.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.175+0000 7fe1527fc700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 0x7fe13c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:40.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.175+0000 7fe1590bd700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 0x7fe13c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:40.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.175+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fe1440cf480 con 0x7fe1541073d0 2024-06-06T06:59:40.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.176+0000 7fe1590bd700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 0x7fe13c0a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009fd0 tx=0x7fe14c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:40.179 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.179+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe144014030 con 0x7fe1541073d0 2024-06-06T06:59:40.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.383+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fe15402de60 con 0x7fe1541073d0 2024-06-06T06:59:40.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.383+0000 7fe1527fc700 1 -- 172.21.15.73:0/3961989829 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe14409beb0 con 0x7fe1541073d0 2024-06-06T06:59:40.384 INFO:teuthology.orchestra.run.smithi073.stdout:257698037822 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.385+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 msgr2=0x7fe13c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.385+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 0x7fe13c0a5170 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009fd0 tx=0x7fe14c00b7b0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.385+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 msgr2=0x7fe1541a4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.385+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fe14400c900 tx=0x7fe14400cc10 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 shutdown_connections 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1541073d0 0x7fe1541a4980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe154101390 0x7fe154199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe13c0a2cb0 0x7fe13c0a5170 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 --2- 172.21.15.73:0/3961989829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe154101d30 0x7fe154199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 >> 172.21.15.73:0/3961989829 conn(0x7fe1540782c0 msgr2=0x7fe154100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:40.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 shutdown_connections 2024-06-06T06:59:40.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:40.386+0000 7fe15b321700 1 -- 172.21.15.73:0/3961989829 wait complete. 2024-06-06T06:59:40.626 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:40 smithi073 ceph-mon[32303]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-06T06:59:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:40 smithi136 ceph-mon[35901]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-06T06:59:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:40 smithi170 ceph-mon[36108]: pgmap v384: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-06-06T06:59:41.213 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037817 got 257698037822 for osd.8 2024-06-06T06:59:41.214 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:41.600 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:41 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3961989829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T06:59:41.646 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:41 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3961989829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T06:59:41.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:41 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3961989829' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-06T06:59:42.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:42 smithi073 ceph-mon[32303]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-06T06:59:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:42 smithi136 ceph-mon[35901]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-06T06:59:42.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:42 smithi170 ceph-mon[36108]: pgmap v385: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-06T06:59:44.648 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:44 smithi073 ceph-mon[32303]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:44 smithi136 ceph-mon[35901]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:44 smithi170 ceph-mon[36108]: pgmap v386: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 -- 172.21.15.73:0/2603602771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188101a60 msgr2=0x7f0188105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 --2- 172.21.15.73:0/2603602771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188101a60 0x7f0188105820 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f017c00aaa0 tx=0x7f017c00adb0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 -- 172.21.15.73:0/2603602771 shutdown_connections 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 --2- 172.21.15.73:0/2603602771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0188105f50 0x7f018810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 --2- 172.21.15.73:0/2603602771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f01881010b0 0x7f0188101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 --2- 172.21.15.73:0/2603602771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188101a60 0x7f0188105820 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 -- 172.21.15.73:0/2603602771 >> 172.21.15.73:0/2603602771 conn(0x7f01880766b0 msgr2=0x7f0188076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 -- 172.21.15.73:0/2603602771 shutdown_connections 2024-06-06T06:59:45.881 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.879+0000 7f018e271700 1 -- 172.21.15.73:0/2603602771 wait complete. 2024-06-06T06:59:45.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.880+0000 7f018e271700 1 Processor -- start 2024-06-06T06:59:45.882 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.880+0000 7f018e271700 1 -- start start 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0188101a60 0x7f018819edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 0x7f0188198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0188110560 con 0x7f0188105f50 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f0187fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f0187fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35490/0 (socket says 172.21.15.73:35490) 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f0187fff700 1 -- 172.21.15.73:0/200323926 learned_addr learned my addr 172.21.15.73:0/200323926 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:45.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018c80e700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 0x7f0188198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f01881103e0 con 0x7f0188101a60 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.881+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f01881106e0 con 0x7f01881010b0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f01877fe700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0188101a60 0x7f018819edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 -- 172.21.15.73:0/200323926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0188101a60 msgr2=0x7f018819edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0188101a60 0x7f018819edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 -- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 msgr2=0x7f0188198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 0x7f0188198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0188199190 con 0x7f01881010b0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f018c80e700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 0x7f0188198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f0187fff700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f017800c920 tx=0x7f017800cc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0178004a30 con 0x7f01881010b0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0188199480 con 0x7f01881010b0 2024-06-06T06:59:45.885 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.882+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f01780054d0 con 0x7f01881010b0 2024-06-06T06:59:45.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.883+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f017801d060 con 0x7f01881010b0 2024-06-06T06:59:45.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.883+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01881a5740 con 0x7f01881010b0 2024-06-06T06:59:45.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.884+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0188069180 con 0x7f01881010b0 2024-06-06T06:59:45.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.885+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0178007bf0 con 0x7f01881010b0 2024-06-06T06:59:45.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.885+0000 7f01857fa700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 0x7f01700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:45.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.885+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f01780cf100 con 0x7f01881010b0 2024-06-06T06:59:45.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.885+0000 7f01877fe700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 0x7f01700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:45.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.886+0000 7f01877fe700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 0x7f01700a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f017c006010 tx=0x7f017c00c650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:45.891 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.890+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0178014030 con 0x7f01881010b0 2024-06-06T06:59:46.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.098+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f0188199f90 con 0x7f01881010b0 2024-06-06T06:59:46.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.098+0000 7f01857fa700 1 -- 172.21.15.73:0/200323926 <== mon.1 v2:172.21.15.170:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f0188199f90 con 0x7f01881010b0 2024-06-06T06:59:46.099 INFO:teuthology.orchestra.run.smithi073.stdout:167503724622 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 msgr2=0x7f01700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 0x7f01700a5170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f017c006010 tx=0x7f017c00c650 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 msgr2=0x7f018819e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f017800c920 tx=0x7f017800cc30 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 shutdown_connections 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f01700a2cb0 0x7f01700a5170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f01881010b0 0x7f018819e870 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0188105f50 0x7f0188198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 --2- 172.21.15.73:0/200323926 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0188101a60 0x7f018819edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:46.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 >> 172.21.15.73:0/200323926 conn(0x7f01880766b0 msgr2=0x7f01880ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:46.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 shutdown_connections 2024-06-06T06:59:46.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.101+0000 7f018e271700 1 -- 172.21.15.73:0/200323926 wait complete. 2024-06-06T06:59:46.633 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:46 smithi073 ceph-mon[32303]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:46.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2361776561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T06:59:46.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/200323926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T06:59:46.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:46 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3611800272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T06:59:46.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:46 smithi136 ceph-mon[35901]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:46.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:46 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2361776561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T06:59:46.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:46 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/200323926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T06:59:46.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:46 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3611800272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T06:59:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:46 smithi170 ceph-mon[36108]: pgmap v387: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T06:59:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:46 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2361776561' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-06T06:59:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:46 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/200323926' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-06T06:59:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:46 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3611800272' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-06T06:59:47.145 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 -- 172.21.15.73:0/1803827511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe574101390 msgr2=0x7fe5741017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 --2- 172.21.15.73:0/1803827511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe574101390 0x7fe5741017f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe560009990 tx=0x7fe560009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 -- 172.21.15.73:0/1803827511 shutdown_connections 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 --2- 172.21.15.73:0/1803827511 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5741073d0 0x7fe5741077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 --2- 172.21.15.73:0/1803827511 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 0x7fe57410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 --2- 172.21.15.73:0/1803827511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe574101390 0x7fe5741017f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 -- 172.21.15.73:0/1803827511 >> 172.21.15.73:0/1803827511 conn(0x7fe5740782c0 msgr2=0x7fe5740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 -- 172.21.15.73:0/1803827511 shutdown_connections 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.626+0000 7fe57afcb700 1 -- 172.21.15.73:0/1803827511 wait complete. 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 Processor -- start 2024-06-06T06:59:47.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 -- start start 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe574101390 0x7fe57419e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 0x7fe57419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5741114e0 con 0x7fe5741073d0 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe574111360 con 0x7fe574101d30 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe57afcb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe574111660 con 0x7fe574101390 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe579568700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.627+0000 7fe579568700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45384/0 (socket says 172.21.15.73:45384) 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 -- 172.21.15.73:0/2361776561 learned_addr learned my addr 172.21.15.73:0/2361776561 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe578d67700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe574101390 0x7fe57419e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe573fff700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 0x7fe57419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 -- 172.21.15.73:0/2361776561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe574101390 msgr2=0x7fe57419e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:47.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe574101390 0x7fe57419e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stdout:146028888147 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 -- 172.21.15.73:0/2361776561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 msgr2=0x7fe57419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 0x7fe57419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 -- 172.21.15.73:0/2361776561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe574199060 con 0x7fe5741073d0 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe579568700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007ee0 tx=0x7fe56c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:47.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c004bb0 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe56c004d50 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe574199350 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56c0058e0 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.628+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5741a55c0 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.629+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe574069180 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.631+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fe56c01a020 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.631+0000 7fe571ffb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 0x7fe5640a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.632+0000 7fe578d67700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 0x7fe5640a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.632+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fe56c0cec80 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.632+0000 7fe578d67700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 0x7fe5640a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe574106f50 tx=0x7fe56800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.636+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe56c010070 con 0x7fe5741073d0 2024-06-06T06:59:47.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.842+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe5741a5790 con 0x7fe5741073d0 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.842+0000 7fe571ffb700 1 -- 172.21.15.73:0/2361776561 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe56c09b6b0 con 0x7fe5741073d0 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 msgr2=0x7fe5640a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 0x7fe5640a51f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe574106f50 tx=0x7fe56800a380 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 msgr2=0x7fe574198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fe56c007ee0 tx=0x7fe56c00d3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 shutdown_connections 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe5640a2d30 0x7fe5640a51f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe574101390 0x7fe57419e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5741073d0 0x7fe574198900 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 --2- 172.21.15.73:0/2361776561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe574101d30 0x7fe57419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 >> 172.21.15.73:0/2361776561 conn(0x7fe5740782c0 msgr2=0x7fe574100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:47.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 shutdown_connections 2024-06-06T06:59:47.151 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:45.845+0000 7fe57afcb700 1 -- 172.21.15.73:0/2361776561 wait complete. 2024-06-06T06:59:48.632 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724616 got 167503724622 for osd.5 2024-06-06T06:59:48.632 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:48.708 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:48 smithi073 ceph-mon[32303]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:48 smithi136 ceph-mon[35901]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 -- 172.21.15.73:0/1327781351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8354101390 msgr2=0x7f83541017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 --2- 172.21.15.73:0/1327781351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8354101390 0x7f83541017f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8344009990 tx=0x7f8344009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 -- 172.21.15.73:0/1327781351 shutdown_connections 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 --2- 172.21.15.73:0/1327781351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101d30 0x7f835410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 --2- 172.21.15.73:0/1327781351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83541073d0 0x7f83541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 --2- 172.21.15.73:0/1327781351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8354101390 0x7f83541017f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 -- 172.21.15.73:0/1327781351 >> 172.21.15.73:0/1327781351 conn(0x7f83540782c0 msgr2=0x7f83540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:48.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 -- 172.21.15.73:0/1327781351 shutdown_connections 2024-06-06T06:59:48.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.814+0000 7f835b4ae700 1 -- 172.21.15.73:0/1327781351 wait complete. 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 Processor -- start 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 -- start start 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101390 0x7f835419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8354101d30 0x7f835419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8354111580 con 0x7f83541073d0 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8354111400 con 0x7f8354101d30 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835b4ae700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8354111700 con 0x7f8354101390 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f8359a4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f8359a4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45428/0 (socket says 172.21.15.73:45428) 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f8359a4b700 1 -- 172.21.15.73:0/2036752940 learned_addr learned my addr 172.21.15.73:0/2036752940 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.815+0000 7f835924a700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101390 0x7f835419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8358a49700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8354101d30 0x7f835419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 -- 172.21.15.73:0/2036752940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101390 msgr2=0x7f835419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101390 0x7f835419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 -- 172.21.15.73:0/2036752940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8354101d30 msgr2=0x7f835419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8354101d30 0x7f835419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 -- 172.21.15.73:0/2036752940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83541990f0 con 0x7f83541073d0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f8359a4b700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b970 tx=0x7f834c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c019070 con 0x7f83541073d0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83541993e0 con 0x7f83541073d0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f834c00cde0 con 0x7f83541073d0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834c007370 con 0x7f83541073d0 2024-06-06T06:59:48.819 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.816+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83541a56a0 con 0x7f83541073d0 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.817+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8354069180 con 0x7f83541073d0 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.818+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f834c01c070 con 0x7f83541073d0 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.819+0000 7f834a7fc700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 0x7f83400a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.819+0000 7f835924a700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 0x7f83400a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.819+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f834c0ce9d0 con 0x7f83541073d0 2024-06-06T06:59:48.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.819+0000 7f835924a700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 0x7f83400a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8350005950 tx=0x7f835000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:48.824 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:48.824+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f834c09b510 con 0x7f83541073d0 2024-06-06T06:59:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:48 smithi170 ceph-mon[36108]: pgmap v388: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T06:59:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 -- 172.21.15.73:0/252419301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 msgr2=0x7fd3441088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 --2- 172.21.15.73:0/252419301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd3441088d0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd340009990 tx=0x7fd340009ca0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 -- 172.21.15.73:0/252419301 shutdown_connections 2024-06-06T06:59:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 --2- 172.21.15.73:0/252419301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 0x7fd34410f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 --2- 172.21.15.73:0/252419301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd3441088d0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 --2- 172.21.15.73:0/252419301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd34406ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 -- 172.21.15.73:0/252419301 >> 172.21.15.73:0/252419301 conn(0x7fd344078300 msgr2=0x7fd344078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 -- 172.21.15.73:0/252419301 shutdown_connections 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.072+0000 7fd34c588700 1 -- 172.21.15.73:0/252419301 wait complete. 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 Processor -- start 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 -- start start 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd344101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 0x7fd344101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112600 con 0x7fd3441084f0 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112480 con 0x7fd34406aa00 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34c588700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd344112780 con 0x7fd34406b3c0 2024-06-06T06:59:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45412/0 (socket says 172.21.15.73:45412) 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 -- 172.21.15.73:0/3611800272 learned_addr learned my addr 172.21.15.73:0/3611800272 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd349b23700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 0x7fd344101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stdout:111669149783 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34a324700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd344101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 -- 172.21.15.73:0/3611800272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 msgr2=0x7fd344101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 0x7fd344101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 -- 172.21.15.73:0/3611800272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 msgr2=0x7fd344101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd344101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34ab25700 1 -- 172.21.15.73:0/3611800272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3441a7080 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.073+0000 7fd34a324700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd344101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd34ab25700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007ee0 tx=0x7fd33c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c00dbd0 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd33c00dd70 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd33c005990 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3441a7370 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.074+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3441a7860 con 0x7fd3441084f0 2024-06-06T06:59:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.075+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd34410a9d0 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.076+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fd33c01a020 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.077+0000 7fd33b7fe700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 0x7fd3300a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.077+0000 7fd34a324700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 0x7fd3300a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.077+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd33c0cf300 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.078+0000 7fd34a324700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 0x7fd3300a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd340006010 tx=0x7fd34001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.080+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd33c010070 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.290+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd34410a3b0 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.291+0000 7fd33b7fe700 1 -- 172.21.15.73:0/3611800272 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd33c09bcd0 con 0x7fd3441084f0 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 msgr2=0x7fd3300a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 0x7fd3300a5170 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd340006010 tx=0x7fd34001a040 comp rx=0 tx=0).stop 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 msgr2=0x7fd344101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd33c007ee0 tx=0x7fd33c00d3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 shutdown_connections 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3441084f0 0x7fd344101ed0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd34406aa00 0x7fd344101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd3300a2cb0 0x7fd3300a5170 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 --2- 172.21.15.73:0/3611800272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd34406b3c0 0x7fd344101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 >> 172.21.15.73:0/3611800272 conn(0x7fd344078300 msgr2=0x7fd344100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 shutdown_connections 2024-06-06T06:59:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:46.293+0000 7fd34c588700 1 -- 172.21.15.73:0/3611800272 wait complete. 2024-06-06T06:59:49.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.024+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f83541a58d0 con 0x7f83541073d0 2024-06-06T06:59:49.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.024+0000 7f834a7fc700 1 -- 172.21.15.73:0/2036752940 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f834c09ae10 con 0x7f83541073d0 2024-06-06T06:59:49.025 INFO:teuthology.orchestra.run.smithi073.stdout:55834574946 2024-06-06T06:59:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 msgr2=0x7f83400a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 0x7f83400a5170 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8350005950 tx=0x7f835000a300 comp rx=0 tx=0).stop 2024-06-06T06:59:49.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 msgr2=0x7f8354198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f834c00b970 tx=0x7f834c00bc80 comp rx=0 tx=0).stop 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 shutdown_connections 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f83400a2cb0 0x7f83400a5170 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8354101390 0x7f835419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83541073d0 0x7f8354198990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 --2- 172.21.15.73:0/2036752940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8354101d30 0x7f835419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 >> 172.21.15.73:0/2036752940 conn(0x7f83540782c0 msgr2=0x7f8354100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 shutdown_connections 2024-06-06T06:59:49.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:49.027+0000 7f835b4ae700 1 -- 172.21.15.73:0/2036752940 wait complete. 2024-06-06T06:59:49.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149779 got 111669149783 for osd.3 2024-06-06T06:59:49.533 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:49 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2036752940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T06:59:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:49 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2036752940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T06:59:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:49 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2036752940' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-06T06:59:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 -- 172.21.15.73:0/3796318587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 msgr2=0x7f45101021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 --2- 172.21.15.73:0/3796318587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f45101021d0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f450c0099c0 tx=0x7f450c009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 -- 172.21.15.73:0/3796318587 shutdown_connections 2024-06-06T06:59:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 --2- 172.21.15.73:0/3796318587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f451010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 --2- 172.21.15.73:0/3796318587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f45101021d0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 --2- 172.21.15.73:0/3796318587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 0x7f4510106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.028+0000 7f4517f3e700 1 -- 172.21.15.73:0/3796318587 >> 172.21.15.73:0/3796318587 conn(0x7f4510077a00 msgr2=0x7f45100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.029+0000 7f4517f3e700 1 -- 172.21.15.73:0/3796318587 shutdown_connections 2024-06-06T06:59:50.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.029+0000 7f4517f3e700 1 -- 172.21.15.73:0/3796318587 wait complete. 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 Processor -- start 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 -- start start 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 0x7f451019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f4510197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4510111540 con 0x7f4510101df0 2024-06-06T06:59:50.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f45101113c0 con 0x7f4510106750 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4517f3e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f45101116c0 con 0x7f45101027a0 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4515cda700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4515cda700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45444/0 (socket says 172.21.15.73:45444) 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.030+0000 7f4515cda700 1 -- 172.21.15.73:0/2233996539 learned_addr learned my addr 172.21.15.73:0/2233996539 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f45154d9700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 0x7f451019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f45164db700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f4510197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 -- 172.21.15.73:0/2233996539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 msgr2=0x7f451019dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 0x7f451019dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 -- 172.21.15.73:0/2233996539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 msgr2=0x7f4510197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f4510197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 -- 172.21.15.73:0/2233996539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4510198020 con 0x7f4510101df0 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f45164db700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f4510197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T06:59:50.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4515cda700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f450c0098e0 tx=0x7f450c0037e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c027070 con 0x7f4510101df0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45101982b0 con 0x7f4510101df0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f450c0041f0 con 0x7f4510101df0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45101a4520 con 0x7f4510101df0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.031+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f450c00f460 con 0x7f4510101df0 2024-06-06T06:59:50.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.032+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44f4005330 con 0x7f4510101df0 2024-06-06T06:59:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.034+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f450c02a070 con 0x7f4510101df0 2024-06-06T06:59:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.035+0000 7f4506ffd700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 0x7f44fc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.035+0000 7f45154d9700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 0x7f44fc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.035+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f450c0d03a0 con 0x7f4510101df0 2024-06-06T06:59:50.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.036+0000 7f45154d9700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 0x7f44fc0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4510103f30 tx=0x7f4500005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:50.041 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.040+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f450c05e850 con 0x7f4510101df0 2024-06-06T06:59:50.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.261+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f44f40058f0 con 0x7f4510101df0 2024-06-06T06:59:50.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.261+0000 7f4506ffd700 1 -- 172.21.15.73:0/2233996539 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f450c013070 con 0x7f4510101df0 2024-06-06T06:59:50.262 INFO:teuthology.orchestra.run.smithi073.stdout:313532612661 2024-06-06T06:59:50.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 msgr2=0x7f44fc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:50.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 0x7f44fc0a5170 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4510103f30 tx=0x7f4500005eb0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 msgr2=0x7f451019d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:50.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f450c0098e0 tx=0x7f450c0037e0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 shutdown_connections 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4510106750 0x7f4510197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4510101df0 0x7f451019d6c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f44fc0a2cb0 0x7f44fc0a5170 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 --2- 172.21.15.73:0/2233996539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f45101027a0 0x7f451019dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.263+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 >> 172.21.15.73:0/2233996539 conn(0x7f4510077a00 msgr2=0x7f45100fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.264+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 shutdown_connections 2024-06-06T06:59:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.264+0000 7f4517f3e700 1 -- 172.21.15.73:0/2233996539 wait complete. 2024-06-06T06:59:50.748 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:50 smithi073 ceph-mon[32303]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:50.749 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:50 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2233996539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T06:59:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:50 smithi136 ceph-mon[35901]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:50 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2233996539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T06:59:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:50 smithi170 ceph-mon[36108]: pgmap v389: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T06:59:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:50 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2233996539' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-06T06:59:51.248 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888143 got 146028888147 for osd.4 2024-06-06T06:59:51.248 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:51.471 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:51 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1451602130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T06:59:51.471 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:51 smithi073 ceph-mon[32303]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:51 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1451602130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T06:59:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:51 smithi136 ceph-mon[35901]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:51 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1451602130' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-06T06:59:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:51 smithi170 ceph-mon[36108]: pgmap v390: 129 pgs: 129 active+clean; 645 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T06:59:52.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.246+0000 7f6a72225700 1 -- 172.21.15.73:0/712045676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 msgr2=0x7f6a6c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:52.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.246+0000 7f6a72225700 1 --2- 172.21.15.73:0/712045676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c102b80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0099c0 tx=0x7f6a5c009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 -- 172.21.15.73:0/712045676 shutdown_connections 2024-06-06T06:59:52.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 --2- 172.21.15.73:0/712045676 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c107870 0x7f6a6c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.077 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 --2- 172.21.15.73:0/712045676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c102b80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 --2- 172.21.15.73:0/712045676 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c103150 0x7f6a6c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 -- 172.21.15.73:0/712045676 >> 172.21.15.73:0/712045676 conn(0x7f6a6c079520 msgr2=0x7f6a6c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 -- 172.21.15.73:0/712045676 shutdown_connections 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 -- 172.21.15.73:0/712045676 wait complete. 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.247+0000 7f6a72225700 1 Processor -- start 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 -- start start 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c103150 0x7f6a6c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c107870 0x7f6a6c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c1126b0 con 0x7f6a6c1027a0 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c112530 con 0x7f6a6c107870 2024-06-06T06:59:52.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a72225700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6a6c112830 con 0x7f6a6c103150 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45454/0 (socket says 172.21.15.73:45454) 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 -- 172.21.15.73:0/1451602130 learned_addr learned my addr 172.21.15.73:0/1451602130 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6bfff700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c107870 0x7f6a6c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6affd700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c103150 0x7f6a6c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 -- 172.21.15.73:0/1451602130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c103150 msgr2=0x7f6a6c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:52.079 INFO:teuthology.orchestra.run.smithi073.stdout:292057776185 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c103150 0x7f6a6c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 -- 172.21.15.73:0/1451602130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c107870 msgr2=0x7f6a6c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c107870 0x7f6a6c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 -- 172.21.15.73:0/1451602130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a6c199030 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.248+0000 7f6a6b7fe700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009fd0 tx=0x7f6a5c0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.249+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c027070 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.249+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a5c004140 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.249+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c022e60 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.249+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a6c1992c0 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.249+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a6c1a55e0 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.250+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a6c069180 con 0x7f6a6c1027a0 2024-06-06T06:59:52.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.251+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6a5c02a070 con 0x7f6a6c1027a0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.252+0000 7f6a68ff9700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 0x7f6a540a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.252+0000 7f6a6affd700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 0x7f6a540a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.252+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f6a5c0d0460 con 0x7f6a6c1027a0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.252+0000 7f6a6affd700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 0x7f6a540a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.256+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6a5c013070 con 0x7f6a6c1027a0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.465+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f6a6c02dca0 con 0x7f6a6c1027a0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.465+0000 7f6a68ff9700 1 -- 172.21.15.73:0/1451602130 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6a5c09cf20 con 0x7f6a6c1027a0 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 msgr2=0x7f6a540a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 0x7f6a540a5170 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6a60006fd0 tx=0x7f6a60008040 comp rx=0 tx=0).stop 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 msgr2=0x7f6a6c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009fd0 tx=0x7f6a5c0037a0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 shutdown_connections 2024-06-06T06:59:52.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6a6c107870 0x7f6a6c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a6c1027a0 0x7f6a6c19e710 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f6a540a2cb0 0x7f6a540a5170 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 --2- 172.21.15.73:0/1451602130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6a6c103150 0x7f6a6c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 >> 172.21.15.73:0/1451602130 conn(0x7f6a6c079520 msgr2=0x7f6a6c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 shutdown_connections 2024-06-06T06:59:52.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:50.468+0000 7f6a72225700 1 -- 172.21.15.73:0/1451602130 wait complete. 2024-06-06T06:59:52.665 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:52 smithi073 ceph-mon[32303]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T06:59:52.699 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776181 got 292057776185 for osd.9 2024-06-06T06:59:52.699 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:52 smithi136 ceph-mon[35901]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T06:59:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:52 smithi170 ceph-mon[36108]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-06-06T06:59:53.498 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:53 smithi073 ceph-mon[32303]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-06T06:59:53.526 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612655 got 313532612661 for osd.10 2024-06-06T06:59:53.526 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:53.635 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574940 got 55834574946 for osd.1 2024-06-06T06:59:53.635 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:53 smithi136 ceph-mon[35901]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-06T06:59:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:53 smithi170 ceph-mon[36108]: pgmap v391: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-06T06:59:53.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 -- 172.21.15.73:0/3686632300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb528107840 msgr2=0x7fb52810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:53.849 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 --2- 172.21.15.73:0/3686632300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb528107840 0x7fb52810f3c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb52000b0d0 tx=0x7fb52000b3e0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 -- 172.21.15.73:0/3686632300 shutdown_connections 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 --2- 172.21.15.73:0/3686632300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb528102740 0x7fb528102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 --2- 172.21.15.73:0/3686632300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb528107840 0x7fb52810f3c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 --2- 172.21.15.73:0/3686632300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb5281030f0 0x7fb528107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 -- 172.21.15.73:0/3686632300 >> 172.21.15.73:0/3686632300 conn(0x7fb5280795f0 msgr2=0x7fb528079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 -- 172.21.15.73:0/3686632300 shutdown_connections 2024-06-06T06:59:53.850 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.811+0000 7fb52f15c700 1 -- 172.21.15.73:0/3686632300 wait complete. 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 Processor -- start 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 -- start start 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb528102740 0x7fb528074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528107840 0x7fb528075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112610 con 0x7fb5281030f0 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112490 con 0x7fb528107840 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb528112790 con 0x7fb528102740 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45480/0 (socket says 172.21.15.73:45480) 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 -- 172.21.15.73:0/740679468 learned_addr learned my addr 172.21.15.73:0/740679468 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 -- 172.21.15.73:0/740679468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb528102740 msgr2=0x7fb528074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:53.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb528102740 0x7fb528074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 -- 172.21.15.73:0/740679468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528107840 msgr2=0x7fb528075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528107840 0x7fb528075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 -- 172.21.15.73:0/740679468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5281a6fc0 con 0x7fb5281030f0 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb527fff700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d950 tx=0x7fb51400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb514016070 con 0x7fb5281030f0 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb514010460 con 0x7fb5281030f0 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5281a72b0 con 0x7fb5281030f0 2024-06-06T06:59:53.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.812+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51400f620 con 0x7fb5281030f0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.813+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5281a7840 con 0x7fb5281030f0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.814+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb528069180 con 0x7fb5281030f0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.815+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb514019070 con 0x7fb5281030f0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.815+0000 7fb525ffb700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 0x7fb5180a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.815+0000 7fb52cef8700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 0x7fb5180a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.815+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fb5140cefa0 con 0x7fb5281030f0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.816+0000 7fb52cef8700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 0x7fb5180a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:53.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:53.820+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb51409bb30 con 0x7fb5281030f0 2024-06-06T06:59:54.024 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.023+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb52802df00 con 0x7fb5281030f0 2024-06-06T06:59:54.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.024+0000 7fb525ffb700 1 -- 172.21.15.73:0/740679468 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb51409b430 con 0x7fb5281030f0 2024-06-06T06:59:54.025 INFO:teuthology.orchestra.run.smithi073.stdout:223338299462 2024-06-06T06:59:54.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 msgr2=0x7fb5180a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 0x7fb5180a5160 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb51c005fd0 tx=0x7fb51c005d00 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 msgr2=0x7fb528074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fb51400d950 tx=0x7fb51400dd10 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 shutdown_connections 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb5180a2ca0 0x7fb5180a5160 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb528102740 0x7fb528074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb528107840 0x7fb528075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 --2- 172.21.15.73:0/740679468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5281030f0 0x7fb528074cd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 >> 172.21.15.73:0/740679468 conn(0x7fb5280795f0 msgr2=0x7fb528100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 shutdown_connections 2024-06-06T06:59:54.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.026+0000 7fb52f15c700 1 -- 172.21.15.73:0/740679468 wait complete. 2024-06-06T06:59:54.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 -- 172.21.15.73:0/1206689376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56981085c0 msgr2=0x7f56981089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 --2- 172.21.15.73:0/1206689376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56981085c0 0x7f56981089a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f56880099c0 tx=0x7f5688009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 -- 172.21.15.73:0/1206689376 shutdown_connections 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 --2- 172.21.15.73:0/1206689376 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 0x7f569810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 --2- 172.21.15.73:0/1206689376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56981085c0 0x7f56981089a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 --2- 172.21.15.73:0/1206689376 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5698102520 0x7f56981029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 -- 172.21.15.73:0/1206689376 >> 172.21.15.73:0/1206689376 conn(0x7f5698079680 msgr2=0x7f5698079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 -- 172.21.15.73:0/1206689376 shutdown_connections 2024-06-06T06:59:54.066 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.064+0000 7f569d974700 1 -- 172.21.15.73:0/1206689376 wait complete. 2024-06-06T06:59:54.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.065+0000 7f569d974700 1 Processor -- start 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.065+0000 7f569d974700 1 -- start start 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 0x7f569819ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56981085c0 0x7f5698198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5698112730 con 0x7f5698102520 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56981125b0 con 0x7f5698102ee0 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f569d974700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56981128b0 con 0x7f56981085c0 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45506/0 (socket says 172.21.15.73:45506) 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 -- 172.21.15.73:0/1020028167 learned_addr learned my addr 172.21.15.73:0/1020028167 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:54.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f56967fc700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 0x7f569819ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f56977fe700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56981085c0 0x7f5698198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 -- 172.21.15.73:0/1020028167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56981085c0 msgr2=0x7f5698198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56981085c0 0x7f5698198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 -- 172.21.15.73:0/1020028167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 msgr2=0x7f569819ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 0x7f569819ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 -- 172.21.15.73:0/1020028167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56981991b0 con 0x7f5698102520 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.066+0000 7f5696ffd700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f5688009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.067+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688026070 con 0x7f5698102520 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.067+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5698199440 con 0x7f5698102520 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.067+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5688004ca0 con 0x7f5698102520 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.067+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5688003d20 con 0x7f5698102520 2024-06-06T06:59:54.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.067+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56980746c0 con 0x7f5698102520 2024-06-06T06:59:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.068+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f569810aa50 con 0x7f5698102520 2024-06-06T06:59:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.069+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5688029070 con 0x7f5698102520 2024-06-06T06:59:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.069+0000 7f569c972700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 0x7f56800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.069+0000 7f56967fc700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 0x7f56800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:54.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.069+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f56880cfcb0 con 0x7f5698102520 2024-06-06T06:59:54.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.070+0000 7f56967fc700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 0x7f56800a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5698079c80 tx=0x7f568c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:54.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.074+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5688013070 con 0x7f5698102520 2024-06-06T06:59:54.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.273+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f5698199f30 con 0x7f5698102520 2024-06-06T06:59:54.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.273+0000 7f569c972700 1 -- 172.21.15.73:0/1020028167 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f568809c990 con 0x7f5698102520 2024-06-06T06:59:54.274 INFO:teuthology.orchestra.run.smithi073.stdout:34359738472 2024-06-06T06:59:54.276 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.275+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 msgr2=0x7f56800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.275+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 0x7f56800a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f5698079c80 tx=0x7f568c009190 comp rx=0 tx=0).stop 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.275+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 msgr2=0x7f569819e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.275+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5688006010 tx=0x7f5688009f80 comp rx=0 tx=0).stop 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 shutdown_connections 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f56800a2c20 0x7f56800a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56981085c0 0x7f5698198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5698102520 0x7f569819e840 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.277 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 --2- 172.21.15.73:0/1020028167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5698102ee0 0x7f569819ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:54.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 >> 172.21.15.73:0/1020028167 conn(0x7f5698079680 msgr2=0x7f5698101230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:54.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 shutdown_connections 2024-06-06T06:59:54.278 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:54.276+0000 7f569d974700 1 -- 172.21.15.73:0/1020028167 wait complete. 2024-06-06T06:59:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:54 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/740679468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T06:59:54.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:54 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1020028167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T06:59:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:54 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/740679468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T06:59:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:54 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1020028167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T06:59:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:54 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/740679468' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-06T06:59:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:54 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1020028167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-06T06:59:54.947 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299456 got 223338299462 for osd.7 2024-06-06T06:59:54.947 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 -- 172.21.15.73:0/443763641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc854102030 msgr2=0x7fc854105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 --2- 172.21.15.73:0/443763641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc854102030 0x7fc854105fd0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc8400099c0 tx=0x7fc840009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 -- 172.21.15.73:0/443763641 shutdown_connections 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 --2- 172.21.15.73:0/443763641 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8541066b0 0x7fc85410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 --2- 172.21.15.73:0/443763641 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854101680 0x7fc854101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 --2- 172.21.15.73:0/443763641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc854102030 0x7fc854105fd0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 -- 172.21.15.73:0/443763641 >> 172.21.15.73:0/443763641 conn(0x7fc8540773e0 msgr2=0x7fc8540777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 -- 172.21.15.73:0/443763641 shutdown_connections 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.084+0000 7fc85b17a700 1 -- 172.21.15.73:0/443763641 wait complete. 2024-06-06T06:59:55.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 Processor -- start 2024-06-06T06:59:55.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 -- start start 2024-06-06T06:59:55.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc854101680 0x7fc8541999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:55.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854102030 0x7fc854199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:55.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:55.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc854111540 con 0x7fc8541066b0 2024-06-06T06:59:55.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541113c0 con 0x7fc854102030 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc85b17a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8541116c0 con 0x7fc854101680 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc859717700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc859717700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45782/0 (socket says 172.21.15.73:45782) 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.085+0000 7fc859717700 1 -- 172.21.15.73:0/2915792199 learned_addr learned my addr 172.21.15.73:0/2915792199 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc853fff700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854102030 0x7fc854199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc858f16700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc854101680 0x7fc8541999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 -- 172.21.15.73:0/2915792199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc854101680 msgr2=0x7fc8541999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc854101680 0x7fc8541999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 -- 172.21.15.73:0/2915792199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854102030 msgr2=0x7fc854199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854102030 0x7fc854199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 -- 172.21.15.73:0/2915792199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8541a4f60 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc859717700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007ee0 tx=0x7fc84c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c004bb0 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8541a5190 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc84c004d50 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc84c0058e0 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.086+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8541a56a0 con 0x7fc8541066b0 2024-06-06T06:59:55.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.087+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc854069180 con 0x7fc8541066b0 2024-06-06T06:59:55.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.088+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc84c01a020 con 0x7fc8541066b0 2024-06-06T06:59:55.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.089+0000 7fc851ffb700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 0x7fc8440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:55.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.089+0000 7fc858f16700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 0x7fc8440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:55.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.089+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc84c0cea80 con 0x7fc8541066b0 2024-06-06T06:59:55.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.090+0000 7fc858f16700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 0x7fc8440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc848005950 tx=0x7fc84800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.094+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc84c010070 con 0x7fc8541066b0 2024-06-06T06:59:55.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.324+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fc85402de60 con 0x7fc8541066b0 2024-06-06T06:59:55.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.324+0000 7fc851ffb700 1 -- 172.21.15.73:0/2915792199 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc84c09b610 con 0x7fc8541066b0 2024-06-06T06:59:55.325 INFO:teuthology.orchestra.run.smithi073.stdout:201863462987 2024-06-06T06:59:55.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 msgr2=0x7fc8440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:55.327 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 0x7fc8440a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fc848005950 tx=0x7fc84800b410 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 msgr2=0x7fc8541a4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc84c007ee0 tx=0x7fc84c00d3b0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 shutdown_connections 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc8440a2c20 0x7fc8440a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc854101680 0x7fc8541999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8541066b0 0x7fc8541a4a20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 --2- 172.21.15.73:0/2915792199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc854102030 0x7fc854199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.326+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 >> 172.21.15.73:0/2915792199 conn(0x7fc8540773e0 msgr2=0x7fc8540ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.327+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 shutdown_connections 2024-06-06T06:59:55.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:55.327+0000 7fc85b17a700 1 -- 172.21.15.73:0/2915792199 wait complete. 2024-06-06T06:59:55.592 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738467 got 34359738472 for osd.0 2024-06-06T06:59:55.592 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:56.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462982 got 201863462987 for osd.6 2024-06-06T06:59:56.269 DEBUG:teuthology.parallel:result is None 2024-06-06T06:59:56.269 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-06T06:59:56.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T06:59:56.515 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T06:59:56.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2915792199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:59:56.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2915792199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:59:56.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2915792199' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-06T06:59:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: pgmap v392: 129 pgs: 129 active+clean; 645 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-06T06:59:56.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T06:59:56.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T06:59:56.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 -- 172.21.15.73:0/4078096881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 msgr2=0x7f4b20107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 --2- 172.21.15.73:0/4078096881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20107110 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4b140099c0 tx=0x7f4b14009cd0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 -- 172.21.15.73:0/4078096881 shutdown_connections 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 --2- 172.21.15.73:0/4078096881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 0x7f4b2010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 --2- 172.21.15.73:0/4078096881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 --2- 172.21.15.73:0/4078096881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20107110 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 -- 172.21.15.73:0/4078096881 >> 172.21.15.73:0/4078096881 conn(0x7f4b200795f0 msgr2=0x7f4b20079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 -- 172.21.15.73:0/4078096881 shutdown_connections 2024-06-06T06:59:58.318 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.316+0000 7f4b25039700 1 -- 172.21.15.73:0/4078096881 wait complete. 2024-06-06T06:59:58.319 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 Processor -- start 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 -- start start 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 0x7f4b20075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20112600 con 0x7f4b201030f0 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20112480 con 0x7f4b20102740 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b25039700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4b20112780 con 0x7f4b20107840 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45798/0 (socket says 172.21.15.73:45798) 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 -- 172.21.15.73:0/808495916 learned_addr learned my addr 172.21.15.73:0/808495916 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1ed9d700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:58.320 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1f59e700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 0x7f4b20075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 -- 172.21.15.73:0/808495916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 msgr2=0x7f4b20075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 0x7f4b20075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 -- 172.21.15.73:0/808495916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 msgr2=0x7f4b20074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.317+0000 7f4b1e59c700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b1e59c700 1 -- 172.21.15.73:0/808495916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b201a7020 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b1ed9d700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b1e59c700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006010 tx=0x7f4b14009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14027070 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b201a72b0 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b14003fd0 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b14022d70 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.318+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b201a7760 con 0x7f4b201030f0 2024-06-06T06:59:58.321 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.319+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b20069180 con 0x7f4b201030f0 2024-06-06T06:59:58.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.320+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b1402a070 con 0x7f4b201030f0 2024-06-06T06:59:58.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.321+0000 7f4b07fff700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 0x7f4b080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T06:59:58.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.321+0000 7f4b1ed9d700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 0x7f4b080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T06:59:58.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.321+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f4b140d0120 con 0x7f4b201030f0 2024-06-06T06:59:58.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.321+0000 7f4b1ed9d700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 0x7f4b080a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b10007fd0 tx=0x7f4b10007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T06:59:58.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.325+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b14013070 con 0x7f4b201030f0 2024-06-06T06:59:58.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.522+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b200628d0 con 0x7f4b080a2ca0 2024-06-06T06:59:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.532+0000 7f4b07fff700 1 -- 172.21.15.73:0/808495916 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341125 (secure 0 0 0) 0x7f4b200628d0 con 0x7f4b080a2ca0 2024-06-06T06:59:58.538 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 msgr2=0x7f4b080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 0x7f4b080a5160 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b10007fd0 tx=0x7f4b10007830 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 msgr2=0x7f4b20074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f4b14006010 tx=0x7f4b14009f80 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 shutdown_connections 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f4b080a2ca0 0x7f4b080a5160 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4b20107840 0x7f4b20075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4b20102740 0x7f4b20074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 --2- 172.21.15.73:0/808495916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b201030f0 0x7f4b20074cd0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T06:59:58.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.535+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 >> 172.21.15.73:0/808495916 conn(0x7f4b200795f0 msgr2=0x7f4b201007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T06:59:58.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.536+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 shutdown_connections 2024-06-06T06:59:58.542 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T06:59:58.536+0000 7f4b25039700 1 -- 172.21.15.73:0/808495916 wait complete. 2024-06-06T06:59:58.542 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T06:59:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 06:59:58 smithi073 ceph-mon[32303]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T06:59:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 06:59:58 smithi136 ceph-mon[35901]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T06:59:58.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 06:59:58 smithi170 ceph-mon[36108]: pgmap v393: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T06:59:59.304 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-06-06T06:59:57.340695+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661260,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1144,"num_read_kb":700,"num_write":442,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387668,"kb_used_data":13556,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554156,"statfs":{"total":1151940427776,"available":1151543455744,"internally_reserved":0,"allocated":13881344,"data_stored":8835974,"data_compressed":496149,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":123,"num_read_kb":58,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003441"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348177+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T06:59:51.348177+0000","last_peered":"2024-06-06T06:59:51.348177+0000","last_clean":"2024-06-06T06:59:51.348177+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T06:59:51.348177+0000","last_undegraded":"2024-06-06T06:59:51.348177+0000","last_fullsized":"2024-06-06T06:59:51.348177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337699+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T06:59:51.337699+0000","last_peered":"2024-06-06T06:59:51.337699+0000","last_clean":"2024-06-06T06:59:51.337699+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T06:59:51.337699+0000","last_undegraded":"2024-06-06T06:59:51.337699+0000","last_fullsized":"2024-06-06T06:59:51.337699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348384+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T06:59:51.348384+0000","last_peered":"2024-06-06T06:59:51.348384+0000","last_clean":"2024-06-06T06:59:51.348384+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T06:59:51.348384+0000","last_undegraded":"2024-06-06T06:59:51.348384+0000","last_fullsized":"2024-06-06T06:59:51.348384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.341182+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T06:59:51.341182+0000","last_peered":"2024-06-06T06:59:51.341182+0000","last_clean":"2024-06-06T06:59:51.341182+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T06:59:51.341182+0000","last_undegraded":"2024-06-06T06:59:51.341182+0000","last_fullsized":"2024-06-06T06:59:51.341182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.322738+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T06:59:51.322738+0000","last_peered":"2024-06-06T06:59:51.322738+0000","last_clean":"2024-06-06T06:59:51.322738+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T06:59:51.322738+0000","last_undegraded":"2024-06-06T06:59:51.322738+0000","last_fullsized":"2024-06-06T06:59:51.322738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334494+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T06:59:51.334494+0000","last_peered":"2024-06-06T06:59:51.334494+0000","last_clean":"2024-06-06T06:59:51.334494+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T06:59:51.334494+0000","last_undegraded":"2024-06-06T06:59:51.334494+0000","last_fullsized":"2024-06-06T06:59:51.334494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344587+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T06:59:51.344587+0000","last_peered":"2024-06-06T06:59:51.344587+0000","last_clean":"2024-06-06T06:59:51.344587+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T06:59:51.344587+0000","last_undegraded":"2024-06-06T06:59:51.344587+0000","last_fullsized":"2024-06-06T06:59:51.344587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339568+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T06:59:51.339568+0000","last_peered":"2024-06-06T06:59:51.339568+0000","last_clean":"2024-06-06T06:59:51.339568+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T06:59:51.339568+0000","last_undegraded":"2024-06-06T06:59:51.339568+0000","last_fullsized":"2024-06-06T06:59:51.339568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347714+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T06:59:51.347714+0000","last_peered":"2024-06-06T06:59:51.347714+0000","last_clean":"2024-06-06T06:59:51.347714+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T06:59:51.347714+0000","last_undegraded":"2024-06-06T06:59:51.347714+0000","last_fullsized":"2024-06-06T06:59:51.347714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.338139+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T06:59:51.338139+0000","last_peered":"2024-06-06T06:59:51.338139+0000","last_clean":"2024-06-06T06:59:51.338139+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T06:59:51.338139+0000","last_undegraded":"2024-06-06T06:59:51.338139+0000","last_fullsized":"2024-06-06T06:59:51.338139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336864+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T06:59:51.336864+0000","last_peered":"2024-06-06T06:59:51.336864+0000","last_clean":"2024-06-06T06:59:51.336864+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T06:59:51.336864+0000","last_undegraded":"2024-06-06T06:59:51.336864+0000","last_fullsized":"2024-06-06T06:59:51.336864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339755+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T06:59:51.339755+0000","last_peered":"2024-06-06T06:59:51.339755+0000","last_clean":"2024-06-06T06:59:51.339755+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T06:59:51.339755+0000","last_undegraded":"2024-06-06T06:59:51.339755+0000","last_fullsized":"2024-06-06T06:59:51.339755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.333054+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T06:59:51.333054+0000","last_peered":"2024-06-06T06:59:51.333054+0000","last_clean":"2024-06-06T06:59:51.333054+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T06:59:51.333054+0000","last_undegraded":"2024-06-06T06:59:51.333054+0000","last_fullsized":"2024-06-06T06:59:51.333054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.330974+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T06:59:51.330974+0000","last_peered":"2024-06-06T06:59:51.330974+0000","last_clean":"2024-06-06T06:59:51.330974+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T06:59:51.330974+0000","last_undegraded":"2024-06-06T06:59:51.330974+0000","last_fullsized":"2024-06-06T06:59:51.330974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340835+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T06:59:51.340835+0000","last_peered":"2024-06-06T06:59:51.340835+0000","last_clean":"2024-06-06T06:59:51.340835+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T06:59:51.340835+0000","last_undegraded":"2024-06-06T06:59:51.340835+0000","last_fullsized":"2024-06-06T06:59:51.340835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346915+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T06:59:51.346915+0000","last_peered":"2024-06-06T06:59:51.346915+0000","last_clean":"2024-06-06T06:59:51.346915+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T06:59:51.346915+0000","last_undegraded":"2024-06-06T06:59:51.346915+0000","last_fullsized":"2024-06-06T06:59:51.346915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348424+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T06:59:51.348424+0000","last_peered":"2024-06-06T06:59:51.348424+0000","last_clean":"2024-06-06T06:59:51.348424+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T06:59:51.348424+0000","last_undegraded":"2024-06-06T06:59:51.348424+0000","last_fullsized":"2024-06-06T06:59:51.348424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340935+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T06:59:51.340935+0000","last_peered":"2024-06-06T06:59:51.340935+0000","last_clean":"2024-06-06T06:59:51.340935+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T06:59:51.340935+0000","last_undegraded":"2024-06-06T06:59:51.340935+0000","last_fullsized":"2024-06-06T06:59:51.340935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.331034+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T06:59:51.331034+0000","last_peered":"2024-06-06T06:59:51.331034+0000","last_clean":"2024-06-06T06:59:51.331034+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T06:59:51.331034+0000","last_undegraded":"2024-06-06T06:59:51.331034+0000","last_fullsized":"2024-06-06T06:59:51.331034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388193+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T06:58:53.388193+0000","last_peered":"2024-06-06T06:58:53.388193+0000","last_clean":"2024-06-06T06:58:53.388193+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T06:58:53.388193+0000","last_undegraded":"2024-06-06T06:58:53.388193+0000","last_fullsized":"2024-06-06T06:58:53.388193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":154,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:52.557454+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T06:59:52.557454+0000","last_peered":"2024-06-06T06:59:52.557454+0000","last_clean":"2024-06-06T06:59:52.557454+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T06:59:52.557454+0000","last_undegraded":"2024-06-06T06:59:52.557454+0000","last_fullsized":"2024-06-06T06:59:52.557454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346198+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T06:59:51.346198+0000","last_peered":"2024-06-06T06:59:51.346198+0000","last_clean":"2024-06-06T06:59:51.346198+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T06:59:51.346198+0000","last_undegraded":"2024-06-06T06:59:51.346198+0000","last_fullsized":"2024-06-06T06:59:51.346198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476358+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T06:58:53.476358+0000","last_peered":"2024-06-06T06:58:53.476358+0000","last_clean":"2024-06-06T06:58:53.476358+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T06:58:53.476358+0000","last_undegraded":"2024-06-06T06:58:53.476358+0000","last_fullsized":"2024-06-06T06:58:53.476358+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.545391+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T06:58:53.545391+0000","last_peered":"2024-06-06T06:58:53.545391+0000","last_clean":"2024-06-06T06:58:53.545391+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T06:58:53.545391+0000","last_undegraded":"2024-06-06T06:58:53.545391+0000","last_fullsized":"2024-06-06T06:58:53.545391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348254+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T06:59:51.348254+0000","last_peered":"2024-06-06T06:59:51.348254+0000","last_clean":"2024-06-06T06:59:51.348254+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T06:59:51.348254+0000","last_undegraded":"2024-06-06T06:59:51.348254+0000","last_fullsized":"2024-06-06T06:59:51.348254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476601+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T06:58:53.476601+0000","last_peered":"2024-06-06T06:58:53.476601+0000","last_clean":"2024-06-06T06:58:53.476601+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T06:58:53.476601+0000","last_undegraded":"2024-06-06T06:58:53.476601+0000","last_fullsized":"2024-06-06T06:58:53.476601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957525+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T06:58:53.957525+0000","last_peered":"2024-06-06T06:58:53.957525+0000","last_clean":"2024-06-06T06:58:53.957525+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T06:58:53.957525+0000","last_undegraded":"2024-06-06T06:58:53.957525+0000","last_fullsized":"2024-06-06T06:58:53.957525+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347704+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T06:59:51.347704+0000","last_peered":"2024-06-06T06:59:51.347704+0000","last_clean":"2024-06-06T06:59:51.347704+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T06:59:51.347704+0000","last_undegraded":"2024-06-06T06:59:51.347704+0000","last_fullsized":"2024-06-06T06:59:51.347704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476093+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T06:58:53.476093+0000","last_peered":"2024-06-06T06:58:53.476093+0000","last_clean":"2024-06-06T06:58:53.476093+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T06:58:53.476093+0000","last_undegraded":"2024-06-06T06:58:53.476093+0000","last_fullsized":"2024-06-06T06:58:53.476093+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391829+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T06:58:53.391829+0000","last_peered":"2024-06-06T06:58:53.391829+0000","last_clean":"2024-06-06T06:58:53.391829+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T06:58:53.391829+0000","last_undegraded":"2024-06-06T06:58:53.391829+0000","last_fullsized":"2024-06-06T06:58:53.391829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347644+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T06:59:51.347644+0000","last_peered":"2024-06-06T06:59:51.347644+0000","last_clean":"2024-06-06T06:59:51.347644+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T06:59:51.347644+0000","last_undegraded":"2024-06-06T06:59:51.347644+0000","last_fullsized":"2024-06-06T06:59:51.347644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388646+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T06:58:53.388646+0000","last_peered":"2024-06-06T06:58:53.388646+0000","last_clean":"2024-06-06T06:58:53.388646+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T06:58:53.388646+0000","last_undegraded":"2024-06-06T06:58:53.388646+0000","last_fullsized":"2024-06-06T06:58:53.388646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":153,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:53.560324+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T06:59:53.560324+0000","last_peered":"2024-06-06T06:59:53.560324+0000","last_clean":"2024-06-06T06:59:53.560324+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T06:59:53.560324+0000","last_undegraded":"2024-06-06T06:59:53.560324+0000","last_fullsized":"2024-06-06T06:59:53.560324+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340224+0000","last_change":"2024-06-06T06:58:53.476855+0000","last_active":"2024-06-06T06:59:51.340224+0000","last_peered":"2024-06-06T06:59:51.340224+0000","last_clean":"2024-06-06T06:59:51.340224+0000","last_became_active":"2024-06-06T06:58:53.476098+0000","last_became_peered":"2024-06-06T06:58:53.476098+0000","last_unstale":"2024-06-06T06:59:51.340224+0000","last_undegraded":"2024-06-06T06:59:51.340224+0000","last_fullsized":"2024-06-06T06:59:51.340224+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:07:20.623344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388328+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T06:58:53.388328+0000","last_peered":"2024-06-06T06:58:53.388328+0000","last_clean":"2024-06-06T06:58:53.388328+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T06:58:53.388328+0000","last_undegraded":"2024-06-06T06:58:53.388328+0000","last_fullsized":"2024-06-06T06:58:53.388328+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.389088+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T06:58:53.389088+0000","last_peered":"2024-06-06T06:58:53.389088+0000","last_clean":"2024-06-06T06:58:53.389088+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T06:58:53.389088+0000","last_undegraded":"2024-06-06T06:58:53.389088+0000","last_fullsized":"2024-06-06T06:58:53.389088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346521+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T06:59:51.346521+0000","last_peered":"2024-06-06T06:59:51.346521+0000","last_clean":"2024-06-06T06:59:51.346521+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T06:59:51.346521+0000","last_undegraded":"2024-06-06T06:59:51.346521+0000","last_fullsized":"2024-06-06T06:59:51.346521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957650+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T06:58:53.957650+0000","last_peered":"2024-06-06T06:58:53.957650+0000","last_clean":"2024-06-06T06:58:53.957650+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T06:58:53.957650+0000","last_undegraded":"2024-06-06T06:58:53.957650+0000","last_fullsized":"2024-06-06T06:58:53.957650+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387515+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T06:58:53.387515+0000","last_peered":"2024-06-06T06:58:53.387515+0000","last_clean":"2024-06-06T06:58:53.387515+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T06:58:53.387515+0000","last_undegraded":"2024-06-06T06:58:53.387515+0000","last_fullsized":"2024-06-06T06:58:53.387515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334137+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T06:59:51.334137+0000","last_peered":"2024-06-06T06:59:51.334137+0000","last_clean":"2024-06-06T06:59:51.334137+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T06:59:51.334137+0000","last_undegraded":"2024-06-06T06:59:51.334137+0000","last_fullsized":"2024-06-06T06:59:51.334137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570872+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T06:58:53.570872+0000","last_peered":"2024-06-06T06:58:53.570872+0000","last_clean":"2024-06-06T06:58:53.570872+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T06:58:53.570872+0000","last_undegraded":"2024-06-06T06:58:53.570872+0000","last_fullsized":"2024-06-06T06:58:53.570872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475593+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T06:58:53.475593+0000","last_peered":"2024-06-06T06:58:53.475593+0000","last_clean":"2024-06-06T06:58:53.475593+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T06:58:53.475593+0000","last_undegraded":"2024-06-06T06:58:53.475593+0000","last_fullsized":"2024-06-06T06:58:53.475593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337309+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T06:59:51.337309+0000","last_peered":"2024-06-06T06:59:51.337309+0000","last_clean":"2024-06-06T06:59:51.337309+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T06:59:51.337309+0000","last_undegraded":"2024-06-06T06:59:51.337309+0000","last_fullsized":"2024-06-06T06:59:51.337309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544875+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T06:58:53.544875+0000","last_peered":"2024-06-06T06:58:53.544875+0000","last_clean":"2024-06-06T06:58:53.544875+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T06:58:53.544875+0000","last_undegraded":"2024-06-06T06:58:53.544875+0000","last_fullsized":"2024-06-06T06:58:53.544875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":242,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.561074+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T06:59:54.561074+0000","last_peered":"2024-06-06T06:59:54.561074+0000","last_clean":"2024-06-06T06:59:54.561074+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T06:59:54.561074+0000","last_undegraded":"2024-06-06T06:59:54.561074+0000","last_fullsized":"2024-06-06T06:59:54.561074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":192,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347332+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T06:59:51.347332+0000","last_peered":"2024-06-06T06:59:51.347332+0000","last_clean":"2024-06-06T06:59:51.347332+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:59:51.347332+0000","last_undegraded":"2024-06-06T06:59:51.347332+0000","last_fullsized":"2024-06-06T06:59:51.347332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.390637+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T06:58:53.390637+0000","last_peered":"2024-06-06T06:58:53.390637+0000","last_clean":"2024-06-06T06:58:53.390637+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T06:58:53.390637+0000","last_undegraded":"2024-06-06T06:58:53.390637+0000","last_fullsized":"2024-06-06T06:58:53.390637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475698+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T06:58:53.475698+0000","last_peered":"2024-06-06T06:58:53.475698+0000","last_clean":"2024-06-06T06:58:53.475698+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T06:58:53.475698+0000","last_undegraded":"2024-06-06T06:58:53.475698+0000","last_fullsized":"2024-06-06T06:58:53.475698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337693+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T06:59:51.337693+0000","last_peered":"2024-06-06T06:59:51.337693+0000","last_clean":"2024-06-06T06:59:51.337693+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T06:59:51.337693+0000","last_undegraded":"2024-06-06T06:59:51.337693+0000","last_fullsized":"2024-06-06T06:59:51.337693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476959+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T06:58:53.476959+0000","last_peered":"2024-06-06T06:58:53.476959+0000","last_clean":"2024-06-06T06:58:53.476959+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T06:58:53.476959+0000","last_undegraded":"2024-06-06T06:58:53.476959+0000","last_fullsized":"2024-06-06T06:58:53.476959+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476353+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T06:58:53.476353+0000","last_peered":"2024-06-06T06:58:53.476353+0000","last_clean":"2024-06-06T06:58:53.476353+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T06:58:53.476353+0000","last_undegraded":"2024-06-06T06:58:53.476353+0000","last_fullsized":"2024-06-06T06:58:53.476353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348339+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T06:59:51.348339+0000","last_peered":"2024-06-06T06:59:51.348339+0000","last_clean":"2024-06-06T06:59:51.348339+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T06:59:51.348339+0000","last_undegraded":"2024-06-06T06:59:51.348339+0000","last_fullsized":"2024-06-06T06:59:51.348339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391104+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T06:58:53.391104+0000","last_peered":"2024-06-06T06:58:53.391104+0000","last_clean":"2024-06-06T06:58:53.391104+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T06:58:53.391104+0000","last_undegraded":"2024-06-06T06:58:53.391104+0000","last_fullsized":"2024-06-06T06:58:53.391104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'38","reported_seq":285,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.561088+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T06:59:54.561088+0000","last_peered":"2024-06-06T06:59:54.561088+0000","last_clean":"2024-06-06T06:59:54.561088+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T06:59:54.561088+0000","last_undegraded":"2024-06-06T06:59:54.561088+0000","last_fullsized":"2024-06-06T06:59:54.561088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63174,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":202,"num_read_kb":163,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337677+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T06:59:51.337677+0000","last_peered":"2024-06-06T06:59:51.337677+0000","last_clean":"2024-06-06T06:59:51.337677+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T06:59:51.337677+0000","last_undegraded":"2024-06-06T06:59:51.337677+0000","last_fullsized":"2024-06-06T06:59:51.337677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476659+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T06:58:53.476659+0000","last_peered":"2024-06-06T06:58:53.476659+0000","last_clean":"2024-06-06T06:58:53.476659+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T06:58:53.476659+0000","last_undegraded":"2024-06-06T06:58:53.476659+0000","last_fullsized":"2024-06-06T06:58:53.476659+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388408+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T06:58:53.388408+0000","last_peered":"2024-06-06T06:58:53.388408+0000","last_clean":"2024-06-06T06:58:53.388408+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T06:58:53.388408+0000","last_undegraded":"2024-06-06T06:58:53.388408+0000","last_fullsized":"2024-06-06T06:58:53.388408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.323471+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T06:59:51.323471+0000","last_peered":"2024-06-06T06:59:51.323471+0000","last_clean":"2024-06-06T06:59:51.323471+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T06:59:51.323471+0000","last_undegraded":"2024-06-06T06:59:51.323471+0000","last_fullsized":"2024-06-06T06:59:51.323471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957376+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T06:58:53.957376+0000","last_peered":"2024-06-06T06:58:53.957376+0000","last_clean":"2024-06-06T06:58:53.957376+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T06:58:53.957376+0000","last_undegraded":"2024-06-06T06:58:53.957376+0000","last_fullsized":"2024-06-06T06:58:53.957376+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475971+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T06:58:53.475971+0000","last_peered":"2024-06-06T06:58:53.475971+0000","last_clean":"2024-06-06T06:58:53.475971+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T06:58:53.475971+0000","last_undegraded":"2024-06-06T06:58:53.475971+0000","last_fullsized":"2024-06-06T06:58:53.475971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337551+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T06:59:51.337551+0000","last_peered":"2024-06-06T06:59:51.337551+0000","last_clean":"2024-06-06T06:59:51.337551+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T06:59:51.337551+0000","last_undegraded":"2024-06-06T06:59:51.337551+0000","last_fullsized":"2024-06-06T06:59:51.337551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391531+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T06:58:53.391531+0000","last_peered":"2024-06-06T06:58:53.391531+0000","last_clean":"2024-06-06T06:58:53.391531+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T06:58:53.391531+0000","last_undegraded":"2024-06-06T06:58:53.391531+0000","last_fullsized":"2024-06-06T06:58:53.391531+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":152,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.560932+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T06:59:54.560932+0000","last_peered":"2024-06-06T06:59:54.560932+0000","last_clean":"2024-06-06T06:59:54.560932+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T06:59:54.560932+0000","last_undegraded":"2024-06-06T06:59:54.560932+0000","last_fullsized":"2024-06-06T06:59:54.560932+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344670+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T06:59:51.344670+0000","last_peered":"2024-06-06T06:59:51.344670+0000","last_clean":"2024-06-06T06:59:51.344670+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T06:59:51.344670+0000","last_undegraded":"2024-06-06T06:59:51.344670+0000","last_fullsized":"2024-06-06T06:59:51.344670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388088+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T06:58:53.388088+0000","last_peered":"2024-06-06T06:58:53.388088+0000","last_clean":"2024-06-06T06:58:53.388088+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T06:58:53.388088+0000","last_undegraded":"2024-06-06T06:58:53.388088+0000","last_fullsized":"2024-06-06T06:58:53.388088+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391207+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T06:58:53.391207+0000","last_peered":"2024-06-06T06:58:53.391207+0000","last_clean":"2024-06-06T06:58:53.391207+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T06:58:53.391207+0000","last_undegraded":"2024-06-06T06:58:53.391207+0000","last_fullsized":"2024-06-06T06:58:53.391207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347249+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T06:59:51.347249+0000","last_peered":"2024-06-06T06:59:51.347249+0000","last_clean":"2024-06-06T06:59:51.347249+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T06:59:51.347249+0000","last_undegraded":"2024-06-06T06:59:51.347249+0000","last_fullsized":"2024-06-06T06:59:51.347249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475495+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T06:58:53.475495+0000","last_peered":"2024-06-06T06:58:53.475495+0000","last_clean":"2024-06-06T06:58:53.475495+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T06:58:53.475495+0000","last_undegraded":"2024-06-06T06:58:53.475495+0000","last_fullsized":"2024-06-06T06:58:53.475495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.392084+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T06:58:53.392084+0000","last_peered":"2024-06-06T06:58:53.392084+0000","last_clean":"2024-06-06T06:58:53.392084+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T06:58:53.392084+0000","last_undegraded":"2024-06-06T06:58:53.392084+0000","last_fullsized":"2024-06-06T06:58:53.392084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346527+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T06:59:51.346527+0000","last_peered":"2024-06-06T06:59:51.346527+0000","last_clean":"2024-06-06T06:59:51.346527+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T06:59:51.346527+0000","last_undegraded":"2024-06-06T06:59:51.346527+0000","last_fullsized":"2024-06-06T06:59:51.346527+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391563+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T06:58:53.391563+0000","last_peered":"2024-06-06T06:58:53.391563+0000","last_clean":"2024-06-06T06:58:53.391563+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T06:58:53.391563+0000","last_undegraded":"2024-06-06T06:58:53.391563+0000","last_fullsized":"2024-06-06T06:58:53.391563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.390993+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T06:58:53.390993+0000","last_peered":"2024-06-06T06:58:53.390993+0000","last_clean":"2024-06-06T06:58:53.390993+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T06:58:53.390993+0000","last_undegraded":"2024-06-06T06:58:53.390993+0000","last_fullsized":"2024-06-06T06:58:53.390993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.324048+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:59:51.324048+0000","last_peered":"2024-06-06T06:59:51.324048+0000","last_clean":"2024-06-06T06:59:51.324048+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T06:59:51.324048+0000","last_undegraded":"2024-06-06T06:59:51.324048+0000","last_fullsized":"2024-06-06T06:59:51.324048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387755+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T06:58:53.387755+0000","last_peered":"2024-06-06T06:58:53.387755+0000","last_clean":"2024-06-06T06:58:53.387755+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T06:58:53.387755+0000","last_undegraded":"2024-06-06T06:58:53.387755+0000","last_fullsized":"2024-06-06T06:58:53.387755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'5","reported_seq":148,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:55.562046+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T06:59:55.562046+0000","last_peered":"2024-06-06T06:59:55.562046+0000","last_clean":"2024-06-06T06:59:55.562046+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T06:59:55.562046+0000","last_undegraded":"2024-06-06T06:59:55.562046+0000","last_fullsized":"2024-06-06T06:59:55.562046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334535+0000","last_change":"2024-06-06T06:58:53.571939+0000","last_active":"2024-06-06T06:59:51.334535+0000","last_peered":"2024-06-06T06:59:51.334535+0000","last_clean":"2024-06-06T06:59:51.334535+0000","last_became_active":"2024-06-06T06:58:53.571590+0000","last_became_peered":"2024-06-06T06:58:53.571590+0000","last_unstale":"2024-06-06T06:59:51.334535+0000","last_undegraded":"2024-06-06T06:59:51.334535+0000","last_fullsized":"2024-06-06T06:59:51.334535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:10.858649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387951+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T06:58:53.387951+0000","last_peered":"2024-06-06T06:58:53.387951+0000","last_clean":"2024-06-06T06:58:53.387951+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T06:58:53.387951+0000","last_undegraded":"2024-06-06T06:58:53.387951+0000","last_fullsized":"2024-06-06T06:58:53.387951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342886+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T06:59:51.342886+0000","last_peered":"2024-06-06T06:59:51.342886+0000","last_clean":"2024-06-06T06:59:51.342886+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T06:59:51.342886+0000","last_undegraded":"2024-06-06T06:59:51.342886+0000","last_fullsized":"2024-06-06T06:59:51.342886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.560270+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T06:59:54.560270+0000","last_peered":"2024-06-06T06:59:54.560270+0000","last_clean":"2024-06-06T06:59:54.560270+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T06:59:54.560270+0000","last_undegraded":"2024-06-06T06:59:54.560270+0000","last_fullsized":"2024-06-06T06:59:54.560270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475368+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T06:58:53.475368+0000","last_peered":"2024-06-06T06:58:53.475368+0000","last_clean":"2024-06-06T06:58:53.475368+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T06:58:53.475368+0000","last_undegraded":"2024-06-06T06:58:53.475368+0000","last_fullsized":"2024-06-06T06:58:53.475368+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339323+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T06:59:51.339323+0000","last_peered":"2024-06-06T06:59:51.339323+0000","last_clean":"2024-06-06T06:59:51.339323+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T06:59:51.339323+0000","last_undegraded":"2024-06-06T06:59:51.339323+0000","last_fullsized":"2024-06-06T06:59:51.339323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476637+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T06:58:53.476637+0000","last_peered":"2024-06-06T06:58:53.476637+0000","last_clean":"2024-06-06T06:58:53.476637+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T06:58:53.476637+0000","last_undegraded":"2024-06-06T06:58:53.476637+0000","last_fullsized":"2024-06-06T06:58:53.476637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":376,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.474652+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T06:58:53.474652+0000","last_peered":"2024-06-06T06:58:53.474652+0000","last_clean":"2024-06-06T06:58:53.474652+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T06:58:53.474652+0000","last_undegraded":"2024-06-06T06:58:53.474652+0000","last_fullsized":"2024-06-06T06:58:53.474652+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476129+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T06:58:53.476129+0000","last_peered":"2024-06-06T06:58:53.476129+0000","last_clean":"2024-06-06T06:58:53.476129+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T06:58:53.476129+0000","last_undegraded":"2024-06-06T06:58:53.476129+0000","last_fullsized":"2024-06-06T06:58:53.476129+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.327623+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T06:59:51.327623+0000","last_peered":"2024-06-06T06:59:51.327623+0000","last_clean":"2024-06-06T06:59:51.327623+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T06:59:51.327623+0000","last_undegraded":"2024-06-06T06:59:51.327623+0000","last_fullsized":"2024-06-06T06:59:51.327623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.474874+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T06:58:53.474874+0000","last_peered":"2024-06-06T06:58:53.474874+0000","last_clean":"2024-06-06T06:58:53.474874+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T06:58:53.474874+0000","last_undegraded":"2024-06-06T06:58:53.474874+0000","last_fullsized":"2024-06-06T06:58:53.474874+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957368+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T06:58:53.957368+0000","last_peered":"2024-06-06T06:58:53.957368+0000","last_clean":"2024-06-06T06:58:53.957368+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T06:58:53.957368+0000","last_undegraded":"2024-06-06T06:58:53.957368+0000","last_fullsized":"2024-06-06T06:58:53.957368+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391881+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T06:58:53.391881+0000","last_peered":"2024-06-06T06:58:53.391881+0000","last_clean":"2024-06-06T06:58:53.391881+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T06:58:53.391881+0000","last_undegraded":"2024-06-06T06:58:53.391881+0000","last_fullsized":"2024-06-06T06:58:53.391881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.329338+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T06:59:51.329338+0000","last_peered":"2024-06-06T06:59:51.329338+0000","last_clean":"2024-06-06T06:59:51.329338+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T06:59:51.329338+0000","last_undegraded":"2024-06-06T06:59:51.329338+0000","last_fullsized":"2024-06-06T06:59:51.329338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544951+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T06:58:53.544951+0000","last_peered":"2024-06-06T06:58:53.544951+0000","last_clean":"2024-06-06T06:58:53.544951+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T06:58:53.544951+0000","last_undegraded":"2024-06-06T06:58:53.544951+0000","last_fullsized":"2024-06-06T06:58:53.544951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'17","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:14.377279+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T06:59:14.377279+0000","last_peered":"2024-06-06T06:59:14.377279+0000","last_clean":"2024-06-06T06:59:14.377279+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T06:59:14.377279+0000","last_undegraded":"2024-06-06T06:59:14.377279+0000","last_fullsized":"2024-06-06T06:59:14.377279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336522+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T06:59:51.336522+0000","last_peered":"2024-06-06T06:59:51.336522+0000","last_clean":"2024-06-06T06:59:51.336522+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T06:59:51.336522+0000","last_undegraded":"2024-06-06T06:59:51.336522+0000","last_fullsized":"2024-06-06T06:59:51.336522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388108+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T06:58:53.388108+0000","last_peered":"2024-06-06T06:58:53.388108+0000","last_clean":"2024-06-06T06:58:53.388108+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T06:58:53.388108+0000","last_undegraded":"2024-06-06T06:58:53.388108+0000","last_fullsized":"2024-06-06T06:58:53.388108+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476799+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T06:58:53.476799+0000","last_peered":"2024-06-06T06:58:53.476799+0000","last_clean":"2024-06-06T06:58:53.476799+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T06:58:53.476799+0000","last_undegraded":"2024-06-06T06:58:53.476799+0000","last_fullsized":"2024-06-06T06:58:53.476799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337370+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T06:59:51.337370+0000","last_peered":"2024-06-06T06:59:51.337370+0000","last_clean":"2024-06-06T06:59:51.337370+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T06:59:51.337370+0000","last_undegraded":"2024-06-06T06:59:51.337370+0000","last_fullsized":"2024-06-06T06:59:51.337370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476257+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T06:58:53.476257+0000","last_peered":"2024-06-06T06:58:53.476257+0000","last_clean":"2024-06-06T06:58:53.476257+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T06:58:53.476257+0000","last_undegraded":"2024-06-06T06:58:53.476257+0000","last_fullsized":"2024-06-06T06:58:53.476257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570702+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T06:58:53.570702+0000","last_peered":"2024-06-06T06:58:53.570702+0000","last_clean":"2024-06-06T06:58:53.570702+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T06:58:53.570702+0000","last_undegraded":"2024-06-06T06:58:53.570702+0000","last_fullsized":"2024-06-06T06:58:53.570702+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336782+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T06:59:51.336782+0000","last_peered":"2024-06-06T06:59:51.336782+0000","last_clean":"2024-06-06T06:59:51.336782+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T06:59:51.336782+0000","last_undegraded":"2024-06-06T06:59:51.336782+0000","last_fullsized":"2024-06-06T06:59:51.336782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391470+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T06:58:53.391470+0000","last_peered":"2024-06-06T06:58:53.391470+0000","last_clean":"2024-06-06T06:58:53.391470+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T06:58:53.391470+0000","last_undegraded":"2024-06-06T06:58:53.391470+0000","last_fullsized":"2024-06-06T06:58:53.391470+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544952+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T06:58:53.544952+0000","last_peered":"2024-06-06T06:58:53.544952+0000","last_clean":"2024-06-06T06:58:53.544952+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T06:58:53.544952+0000","last_undegraded":"2024-06-06T06:58:53.544952+0000","last_fullsized":"2024-06-06T06:58:53.544952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337845+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T06:59:51.337845+0000","last_peered":"2024-06-06T06:59:51.337845+0000","last_clean":"2024-06-06T06:59:51.337845+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T06:59:51.337845+0000","last_undegraded":"2024-06-06T06:59:51.337845+0000","last_fullsized":"2024-06-06T06:59:51.337845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476251+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T06:58:53.476251+0000","last_peered":"2024-06-06T06:58:53.476251+0000","last_clean":"2024-06-06T06:58:53.476251+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T06:58:53.476251+0000","last_undegraded":"2024-06-06T06:58:53.476251+0000","last_fullsized":"2024-06-06T06:58:53.476251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388434+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T06:58:53.388434+0000","last_peered":"2024-06-06T06:58:53.388434+0000","last_clean":"2024-06-06T06:58:53.388434+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T06:58:53.388434+0000","last_undegraded":"2024-06-06T06:58:53.388434+0000","last_fullsized":"2024-06-06T06:58:53.388434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.332688+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T06:59:51.332688+0000","last_peered":"2024-06-06T06:59:51.332688+0000","last_clean":"2024-06-06T06:59:51.332688+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T06:59:51.332688+0000","last_undegraded":"2024-06-06T06:59:51.332688+0000","last_fullsized":"2024-06-06T06:59:51.332688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476177+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T06:58:53.476177+0000","last_peered":"2024-06-06T06:58:53.476177+0000","last_clean":"2024-06-06T06:58:53.476177+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T06:58:53.476177+0000","last_undegraded":"2024-06-06T06:58:53.476177+0000","last_fullsized":"2024-06-06T06:58:53.476177+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391359+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T06:58:53.391359+0000","last_peered":"2024-06-06T06:58:53.391359+0000","last_clean":"2024-06-06T06:58:53.391359+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T06:58:53.391359+0000","last_undegraded":"2024-06-06T06:58:53.391359+0000","last_fullsized":"2024-06-06T06:58:53.391359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344930+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T06:59:51.344930+0000","last_peered":"2024-06-06T06:59:51.344930+0000","last_clean":"2024-06-06T06:59:51.344930+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T06:59:51.344930+0000","last_undegraded":"2024-06-06T06:59:51.344930+0000","last_fullsized":"2024-06-06T06:59:51.344930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570750+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T06:58:53.570750+0000","last_peered":"2024-06-06T06:58:53.570750+0000","last_clean":"2024-06-06T06:58:53.570750+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T06:58:53.570750+0000","last_undegraded":"2024-06-06T06:58:53.570750+0000","last_fullsized":"2024-06-06T06:58:53.570750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":152,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.561950+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T06:59:54.561950+0000","last_peered":"2024-06-06T06:59:54.561950+0000","last_clean":"2024-06-06T06:59:54.561950+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T06:59:54.561950+0000","last_undegraded":"2024-06-06T06:59:54.561950+0000","last_fullsized":"2024-06-06T06:59:54.561950+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.345155+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T06:59:51.345155+0000","last_peered":"2024-06-06T06:59:51.345155+0000","last_clean":"2024-06-06T06:59:51.345155+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T06:59:51.345155+0000","last_undegraded":"2024-06-06T06:59:51.345155+0000","last_fullsized":"2024-06-06T06:59:51.345155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570771+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T06:58:53.570771+0000","last_peered":"2024-06-06T06:58:53.570771+0000","last_clean":"2024-06-06T06:58:53.570771+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T06:58:53.570771+0000","last_undegraded":"2024-06-06T06:58:53.570771+0000","last_fullsized":"2024-06-06T06:58:53.570771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476328+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T06:58:53.476328+0000","last_peered":"2024-06-06T06:58:53.476328+0000","last_clean":"2024-06-06T06:58:53.476328+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T06:58:53.476328+0000","last_undegraded":"2024-06-06T06:58:53.476328+0000","last_fullsized":"2024-06-06T06:58:53.476328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344405+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:59:51.344405+0000","last_peered":"2024-06-06T06:59:51.344405+0000","last_clean":"2024-06-06T06:59:51.344405+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T06:59:51.344405+0000","last_undegraded":"2024-06-06T06:59:51.344405+0000","last_fullsized":"2024-06-06T06:59:51.344405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475379+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T06:58:53.475379+0000","last_peered":"2024-06-06T06:58:53.475379+0000","last_clean":"2024-06-06T06:58:53.475379+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T06:58:53.475379+0000","last_undegraded":"2024-06-06T06:58:53.475379+0000","last_fullsized":"2024-06-06T06:58:53.475379+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387985+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T06:58:53.387985+0000","last_peered":"2024-06-06T06:58:53.387985+0000","last_clean":"2024-06-06T06:58:53.387985+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T06:58:53.387985+0000","last_undegraded":"2024-06-06T06:58:53.387985+0000","last_fullsized":"2024-06-06T06:58:53.387985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.345005+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T06:59:51.345005+0000","last_peered":"2024-06-06T06:59:51.345005+0000","last_clean":"2024-06-06T06:59:51.345005+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T06:59:51.345005+0000","last_undegraded":"2024-06-06T06:59:51.345005+0000","last_fullsized":"2024-06-06T06:59:51.345005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347846+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T06:59:51.347846+0000","last_peered":"2024-06-06T06:59:51.347846+0000","last_clean":"2024-06-06T06:59:51.347846+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T06:59:51.347846+0000","last_undegraded":"2024-06-06T06:59:51.347846+0000","last_fullsized":"2024-06-06T06:59:51.347846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344430+0000","last_change":"2024-06-06T06:58:53.478114+0000","last_active":"2024-06-06T06:59:51.344430+0000","last_peered":"2024-06-06T06:59:51.344430+0000","last_clean":"2024-06-06T06:59:51.344430+0000","last_became_active":"2024-06-06T06:58:53.477664+0000","last_became_peered":"2024-06-06T06:58:53.477664+0000","last_unstale":"2024-06-06T06:59:51.344430+0000","last_undegraded":"2024-06-06T06:59:51.344430+0000","last_fullsized":"2024-06-06T06:59:51.344430+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:01.541804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339964+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T06:59:51.339964+0000","last_peered":"2024-06-06T06:59:51.339964+0000","last_clean":"2024-06-06T06:59:51.339964+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T06:59:51.339964+0000","last_undegraded":"2024-06-06T06:59:51.339964+0000","last_fullsized":"2024-06-06T06:59:51.339964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337362+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T06:59:51.337362+0000","last_peered":"2024-06-06T06:59:51.337362+0000","last_clean":"2024-06-06T06:59:51.337362+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T06:59:51.337362+0000","last_undegraded":"2024-06-06T06:59:51.337362+0000","last_fullsized":"2024-06-06T06:59:51.337362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337810+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T06:59:51.337810+0000","last_peered":"2024-06-06T06:59:51.337810+0000","last_clean":"2024-06-06T06:59:51.337810+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T06:59:51.337810+0000","last_undegraded":"2024-06-06T06:59:51.337810+0000","last_fullsized":"2024-06-06T06:59:51.337810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346547+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T06:59:51.346547+0000","last_peered":"2024-06-06T06:59:51.346547+0000","last_clean":"2024-06-06T06:59:51.346547+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T06:59:51.346547+0000","last_undegraded":"2024-06-06T06:59:51.346547+0000","last_fullsized":"2024-06-06T06:59:51.346547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342784+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T06:59:51.342784+0000","last_peered":"2024-06-06T06:59:51.342784+0000","last_clean":"2024-06-06T06:59:51.342784+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T06:59:51.342784+0000","last_undegraded":"2024-06-06T06:59:51.342784+0000","last_fullsized":"2024-06-06T06:59:51.342784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.331626+0000","last_change":"2024-06-06T06:58:53.478046+0000","last_active":"2024-06-06T06:59:51.331626+0000","last_peered":"2024-06-06T06:59:51.331626+0000","last_clean":"2024-06-06T06:59:51.331626+0000","last_became_active":"2024-06-06T06:58:53.477722+0000","last_became_peered":"2024-06-06T06:58:53.477722+0000","last_unstale":"2024-06-06T06:59:51.331626+0000","last_undegraded":"2024-06-06T06:59:51.331626+0000","last_fullsized":"2024-06-06T06:59:51.331626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:22.491510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346457+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T06:59:51.346457+0000","last_peered":"2024-06-06T06:59:51.346457+0000","last_clean":"2024-06-06T06:59:51.346457+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T06:59:51.346457+0000","last_undegraded":"2024-06-06T06:59:51.346457+0000","last_fullsized":"2024-06-06T06:59:51.346457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.333531+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T06:59:51.333531+0000","last_peered":"2024-06-06T06:59:51.333531+0000","last_clean":"2024-06-06T06:59:51.333531+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T06:59:51.333531+0000","last_undegraded":"2024-06-06T06:59:51.333531+0000","last_fullsized":"2024-06-06T06:59:51.333531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340143+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T06:59:51.340143+0000","last_peered":"2024-06-06T06:59:51.340143+0000","last_clean":"2024-06-06T06:59:51.340143+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:59:51.340143+0000","last_undegraded":"2024-06-06T06:59:51.340143+0000","last_fullsized":"2024-06-06T06:59:51.340143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337935+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T06:59:51.337935+0000","last_peered":"2024-06-06T06:59:51.337935+0000","last_clean":"2024-06-06T06:59:51.337935+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T06:59:51.337935+0000","last_undegraded":"2024-06-06T06:59:51.337935+0000","last_fullsized":"2024-06-06T06:59:51.337935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342252+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T06:59:51.342252+0000","last_peered":"2024-06-06T06:59:51.342252+0000","last_clean":"2024-06-06T06:59:51.342252+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T06:59:51.342252+0000","last_undegraded":"2024-06-06T06:59:51.342252+0000","last_fullsized":"2024-06-06T06:59:51.342252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285685,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70658,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1008,"num_read_kb":581,"num_write":259,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351026,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":574163,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612663,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1224153,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776187,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1138688,"data_stored":577658,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037826,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":573940,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299463,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1170577,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":569066,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":633270,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":571053,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411424,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1191936,"data_stored":633399,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149786,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":568774,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574293,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1191936,"data_stored":1165628,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T06:59:59.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-06T06:59:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:00 smithi073 ceph-mon[32303]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:00 smithi073 ceph-mon[32303]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T07:00:00.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:00 smithi073 ceph-mon[32303]: overall HEALTH_OK 2024-06-06T07:00:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:00 smithi136 ceph-mon[35901]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:00 smithi136 ceph-mon[35901]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T07:00:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:00 smithi136 ceph-mon[35901]: overall HEALTH_OK 2024-06-06T07:00:00.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:00 smithi170 ceph-mon[36108]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:00.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:00 smithi170 ceph-mon[36108]: pgmap v394: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 597 B/s wr, 9 op/s 2024-06-06T07:00:00.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:00 smithi170 ceph-mon[36108]: overall HEALTH_OK 2024-06-06T07:00:01.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- 172.21.15.73:0/849972108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300107570 msgr2=0x7fc300107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 --2- 172.21.15.73:0/849972108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300107570 0x7fc300107950 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0009990 tx=0x7fc2f0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- 172.21.15.73:0/849972108 shutdown_connections 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 --2- 172.21.15.73:0/849972108 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300101810 0x7fc30010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 --2- 172.21.15.73:0/849972108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300107570 0x7fc300107950 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 --2- 172.21.15.73:0/849972108 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc300100e50 0x7fc3001012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- 172.21.15.73:0/849972108 >> 172.21.15.73:0/849972108 conn(0x7fc3000766b0 msgr2=0x7fc300076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- 172.21.15.73:0/849972108 shutdown_connections 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- 172.21.15.73:0/849972108 wait complete. 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 Processor -- start 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.454+0000 7fc30808b700 1 -- start start 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:01.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc300101810 0x7fc30019ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300107570 0x7fc300198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc3001126e0 con 0x7fc300100e50 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc300112560 con 0x7fc300107570 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc300112860 con 0x7fc300101810 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45818/0 (socket says 172.21.15.73:45818) 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 -- 172.21.15.73:0/4198874125 learned_addr learned my addr 172.21.15.73:0/4198874125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc306628700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300107570 0x7fc300198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:01.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 -- 172.21.15.73:0/4198874125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc300101810 msgr2=0x7fc30019ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc300101810 0x7fc30019ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 -- 172.21.15.73:0/4198874125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300107570 msgr2=0x7fc300198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300107570 0x7fc300198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 -- 172.21.15.73:0/4198874125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3001991c0 con 0x7fc300100e50 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc305e27700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0027070 con 0x7fc300100e50 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc300199450 con 0x7fc300100e50 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3001a5720 con 0x7fc300100e50 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.455+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2f000bdc0 con 0x7fc300100e50 2024-06-06T07:00:01.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.456+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0031060 con 0x7fc300100e50 2024-06-06T07:00:01.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.456+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc300069180 con 0x7fc300100e50 2024-06-06T07:00:01.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.458+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc2f002a070 con 0x7fc300100e50 2024-06-06T07:00:01.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.458+0000 7fc2f6ffd700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 0x7fc2ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:01.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.459+0000 7fc305626700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 0x7fc2ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:01.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.459+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc2f00d04f0 con 0x7fc300100e50 2024-06-06T07:00:01.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.459+0000 7fc305626700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 0x7fc2ec0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc300076d20 tx=0x7fc2fc006d20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:01.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.462+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc2f0013070 con 0x7fc300100e50 2024-06-06T07:00:01.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.653+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc300199f40 con 0x7fc2ec0a2c20 2024-06-06T07:00:01.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.662+0000 7fc2f6ffd700 1 -- 172.21.15.73:0/4198874125 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+341004 (secure 0 0 0) 0x7fc300199f40 con 0x7fc2ec0a2c20 2024-06-06T07:00:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 msgr2=0x7fc2ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 0x7fc2ec0a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc300076d20 tx=0x7fc2fc006d20 comp rx=0 tx=0).stop 2024-06-06T07:00:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 msgr2=0x7fc30019e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:01.670 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fc2f0000c00 tx=0x7fc2f0003a80 comp rx=0 tx=0).stop 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 shutdown_connections 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc300107570 0x7fc300198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc300100e50 0x7fc30019e850 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc2ec0a2c20 0x7fc2ec0a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 --2- 172.21.15.73:0/4198874125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc300101810 0x7fc30019ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 >> 172.21.15.73:0/4198874125 conn(0x7fc3000766b0 msgr2=0x7fc3000ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 shutdown_connections 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:01.667+0000 7fc30808b700 1 -- 172.21.15.73:0/4198874125 wait complete. 2024-06-06T07:00:01.671 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T07:00:02.523 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-06-06T07:00:01.341689+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":661260,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1184,"num_read_kb":720,"num_write":442,"num_write_kb":3089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":862,"ondisk_log_size":862,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387668,"kb_used_data":13556,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124554156,"statfs":{"total":1151940427776,"available":1151543455744,"internally_reserved":0,"allocated":13881344,"data_stored":8835974,"data_compressed":496149,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":531,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":143,"num_read_kb":73,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348177+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T06:59:51.348177+0000","last_peered":"2024-06-06T06:59:51.348177+0000","last_clean":"2024-06-06T06:59:51.348177+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T06:59:51.348177+0000","last_undegraded":"2024-06-06T06:59:51.348177+0000","last_fullsized":"2024-06-06T06:59:51.348177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":40,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337699+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T06:59:51.337699+0000","last_peered":"2024-06-06T06:59:51.337699+0000","last_clean":"2024-06-06T06:59:51.337699+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T06:59:51.337699+0000","last_undegraded":"2024-06-06T06:59:51.337699+0000","last_fullsized":"2024-06-06T06:59:51.337699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348384+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T06:59:51.348384+0000","last_peered":"2024-06-06T06:59:51.348384+0000","last_clean":"2024-06-06T06:59:51.348384+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T06:59:51.348384+0000","last_undegraded":"2024-06-06T06:59:51.348384+0000","last_fullsized":"2024-06-06T06:59:51.348384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.341182+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T06:59:51.341182+0000","last_peered":"2024-06-06T06:59:51.341182+0000","last_clean":"2024-06-06T06:59:51.341182+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T06:59:51.341182+0000","last_undegraded":"2024-06-06T06:59:51.341182+0000","last_fullsized":"2024-06-06T06:59:51.341182+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.322738+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T06:59:51.322738+0000","last_peered":"2024-06-06T06:59:51.322738+0000","last_clean":"2024-06-06T06:59:51.322738+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T06:59:51.322738+0000","last_undegraded":"2024-06-06T06:59:51.322738+0000","last_fullsized":"2024-06-06T06:59:51.322738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334494+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T06:59:51.334494+0000","last_peered":"2024-06-06T06:59:51.334494+0000","last_clean":"2024-06-06T06:59:51.334494+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T06:59:51.334494+0000","last_undegraded":"2024-06-06T06:59:51.334494+0000","last_fullsized":"2024-06-06T06:59:51.334494+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344587+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T06:59:51.344587+0000","last_peered":"2024-06-06T06:59:51.344587+0000","last_clean":"2024-06-06T06:59:51.344587+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T06:59:51.344587+0000","last_undegraded":"2024-06-06T06:59:51.344587+0000","last_fullsized":"2024-06-06T06:59:51.344587+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339568+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T06:59:51.339568+0000","last_peered":"2024-06-06T06:59:51.339568+0000","last_clean":"2024-06-06T06:59:51.339568+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T06:59:51.339568+0000","last_undegraded":"2024-06-06T06:59:51.339568+0000","last_fullsized":"2024-06-06T06:59:51.339568+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347714+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T06:59:51.347714+0000","last_peered":"2024-06-06T06:59:51.347714+0000","last_clean":"2024-06-06T06:59:51.347714+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T06:59:51.347714+0000","last_undegraded":"2024-06-06T06:59:51.347714+0000","last_fullsized":"2024-06-06T06:59:51.347714+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.338139+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T06:59:51.338139+0000","last_peered":"2024-06-06T06:59:51.338139+0000","last_clean":"2024-06-06T06:59:51.338139+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T06:59:51.338139+0000","last_undegraded":"2024-06-06T06:59:51.338139+0000","last_fullsized":"2024-06-06T06:59:51.338139+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336864+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T06:59:51.336864+0000","last_peered":"2024-06-06T06:59:51.336864+0000","last_clean":"2024-06-06T06:59:51.336864+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T06:59:51.336864+0000","last_undegraded":"2024-06-06T06:59:51.336864+0000","last_fullsized":"2024-06-06T06:59:51.336864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339755+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T06:59:51.339755+0000","last_peered":"2024-06-06T06:59:51.339755+0000","last_clean":"2024-06-06T06:59:51.339755+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T06:59:51.339755+0000","last_undegraded":"2024-06-06T06:59:51.339755+0000","last_fullsized":"2024-06-06T06:59:51.339755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.333054+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T06:59:51.333054+0000","last_peered":"2024-06-06T06:59:51.333054+0000","last_clean":"2024-06-06T06:59:51.333054+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T06:59:51.333054+0000","last_undegraded":"2024-06-06T06:59:51.333054+0000","last_fullsized":"2024-06-06T06:59:51.333054+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.330974+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T06:59:51.330974+0000","last_peered":"2024-06-06T06:59:51.330974+0000","last_clean":"2024-06-06T06:59:51.330974+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T06:59:51.330974+0000","last_undegraded":"2024-06-06T06:59:51.330974+0000","last_fullsized":"2024-06-06T06:59:51.330974+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340835+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T06:59:51.340835+0000","last_peered":"2024-06-06T06:59:51.340835+0000","last_clean":"2024-06-06T06:59:51.340835+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T06:59:51.340835+0000","last_undegraded":"2024-06-06T06:59:51.340835+0000","last_fullsized":"2024-06-06T06:59:51.340835+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346915+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T06:59:51.346915+0000","last_peered":"2024-06-06T06:59:51.346915+0000","last_clean":"2024-06-06T06:59:51.346915+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T06:59:51.346915+0000","last_undegraded":"2024-06-06T06:59:51.346915+0000","last_fullsized":"2024-06-06T06:59:51.346915+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348424+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T06:59:51.348424+0000","last_peered":"2024-06-06T06:59:51.348424+0000","last_clean":"2024-06-06T06:59:51.348424+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T06:59:51.348424+0000","last_undegraded":"2024-06-06T06:59:51.348424+0000","last_fullsized":"2024-06-06T06:59:51.348424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340935+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T06:59:51.340935+0000","last_peered":"2024-06-06T06:59:51.340935+0000","last_clean":"2024-06-06T06:59:51.340935+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T06:59:51.340935+0000","last_undegraded":"2024-06-06T06:59:51.340935+0000","last_fullsized":"2024-06-06T06:59:51.340935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.331034+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T06:59:51.331034+0000","last_peered":"2024-06-06T06:59:51.331034+0000","last_clean":"2024-06-06T06:59:51.331034+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T06:59:51.331034+0000","last_undegraded":"2024-06-06T06:59:51.331034+0000","last_fullsized":"2024-06-06T06:59:51.331034+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388193+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T06:58:53.388193+0000","last_peered":"2024-06-06T06:58:53.388193+0000","last_clean":"2024-06-06T06:58:53.388193+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T06:58:53.388193+0000","last_undegraded":"2024-06-06T06:58:53.388193+0000","last_fullsized":"2024-06-06T06:58:53.388193+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"98'13","reported_seq":159,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:57.564323+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T06:59:57.564323+0000","last_peered":"2024-06-06T06:59:57.564323+0000","last_clean":"2024-06-06T06:59:57.564323+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T06:59:57.564323+0000","last_undegraded":"2024-06-06T06:59:57.564323+0000","last_fullsized":"2024-06-06T06:59:57.564323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346198+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T06:59:51.346198+0000","last_peered":"2024-06-06T06:59:51.346198+0000","last_clean":"2024-06-06T06:59:51.346198+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T06:59:51.346198+0000","last_undegraded":"2024-06-06T06:59:51.346198+0000","last_fullsized":"2024-06-06T06:59:51.346198+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476358+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T06:58:53.476358+0000","last_peered":"2024-06-06T06:58:53.476358+0000","last_clean":"2024-06-06T06:58:53.476358+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T06:58:53.476358+0000","last_undegraded":"2024-06-06T06:58:53.476358+0000","last_fullsized":"2024-06-06T06:58:53.476358+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.545391+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T06:58:53.545391+0000","last_peered":"2024-06-06T06:58:53.545391+0000","last_clean":"2024-06-06T06:58:53.545391+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T06:58:53.545391+0000","last_undegraded":"2024-06-06T06:58:53.545391+0000","last_fullsized":"2024-06-06T06:58:53.545391+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348254+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T06:59:51.348254+0000","last_peered":"2024-06-06T06:59:51.348254+0000","last_clean":"2024-06-06T06:59:51.348254+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T06:59:51.348254+0000","last_undegraded":"2024-06-06T06:59:51.348254+0000","last_fullsized":"2024-06-06T06:59:51.348254+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476601+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T06:58:53.476601+0000","last_peered":"2024-06-06T06:58:53.476601+0000","last_clean":"2024-06-06T06:58:53.476601+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T06:58:53.476601+0000","last_undegraded":"2024-06-06T06:58:53.476601+0000","last_fullsized":"2024-06-06T06:58:53.476601+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957525+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T06:58:53.957525+0000","last_peered":"2024-06-06T06:58:53.957525+0000","last_clean":"2024-06-06T06:58:53.957525+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T06:58:53.957525+0000","last_undegraded":"2024-06-06T06:58:53.957525+0000","last_fullsized":"2024-06-06T06:58:53.957525+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347704+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T06:59:51.347704+0000","last_peered":"2024-06-06T06:59:51.347704+0000","last_clean":"2024-06-06T06:59:51.347704+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T06:59:51.347704+0000","last_undegraded":"2024-06-06T06:59:51.347704+0000","last_fullsized":"2024-06-06T06:59:51.347704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476093+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T06:58:53.476093+0000","last_peered":"2024-06-06T06:58:53.476093+0000","last_clean":"2024-06-06T06:58:53.476093+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T06:58:53.476093+0000","last_undegraded":"2024-06-06T06:58:53.476093+0000","last_fullsized":"2024-06-06T06:58:53.476093+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":51,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391829+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T06:58:53.391829+0000","last_peered":"2024-06-06T06:58:53.391829+0000","last_clean":"2024-06-06T06:58:53.391829+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T06:58:53.391829+0000","last_undegraded":"2024-06-06T06:58:53.391829+0000","last_fullsized":"2024-06-06T06:58:53.391829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347644+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T06:59:51.347644+0000","last_peered":"2024-06-06T06:59:51.347644+0000","last_clean":"2024-06-06T06:59:51.347644+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T06:59:51.347644+0000","last_undegraded":"2024-06-06T06:59:51.347644+0000","last_fullsized":"2024-06-06T06:59:51.347644+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388646+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T06:58:53.388646+0000","last_peered":"2024-06-06T06:58:53.388646+0000","last_clean":"2024-06-06T06:58:53.388646+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T06:58:53.388646+0000","last_undegraded":"2024-06-06T06:58:53.388646+0000","last_fullsized":"2024-06-06T06:58:53.388646+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"98'5","reported_seq":158,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:58.567843+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T06:59:58.567843+0000","last_peered":"2024-06-06T06:59:58.567843+0000","last_clean":"2024-06-06T06:59:58.567843+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T06:59:58.567843+0000","last_undegraded":"2024-06-06T06:59:58.567843+0000","last_fullsized":"2024-06-06T06:59:58.567843+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340224+0000","last_change":"2024-06-06T06:58:53.476855+0000","last_active":"2024-06-06T06:59:51.340224+0000","last_peered":"2024-06-06T06:59:51.340224+0000","last_clean":"2024-06-06T06:59:51.340224+0000","last_became_active":"2024-06-06T06:58:53.476098+0000","last_became_peered":"2024-06-06T06:58:53.476098+0000","last_unstale":"2024-06-06T06:59:51.340224+0000","last_undegraded":"2024-06-06T06:59:51.340224+0000","last_fullsized":"2024-06-06T06:59:51.340224+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:07:20.623344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388328+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T06:58:53.388328+0000","last_peered":"2024-06-06T06:58:53.388328+0000","last_clean":"2024-06-06T06:58:53.388328+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T06:58:53.388328+0000","last_undegraded":"2024-06-06T06:58:53.388328+0000","last_fullsized":"2024-06-06T06:58:53.388328+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.389088+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T06:58:53.389088+0000","last_peered":"2024-06-06T06:58:53.389088+0000","last_clean":"2024-06-06T06:58:53.389088+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T06:58:53.389088+0000","last_undegraded":"2024-06-06T06:58:53.389088+0000","last_fullsized":"2024-06-06T06:58:53.389088+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346521+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T06:59:51.346521+0000","last_peered":"2024-06-06T06:59:51.346521+0000","last_clean":"2024-06-06T06:59:51.346521+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T06:59:51.346521+0000","last_undegraded":"2024-06-06T06:59:51.346521+0000","last_fullsized":"2024-06-06T06:59:51.346521+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957650+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T06:58:53.957650+0000","last_peered":"2024-06-06T06:58:53.957650+0000","last_clean":"2024-06-06T06:58:53.957650+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T06:58:53.957650+0000","last_undegraded":"2024-06-06T06:58:53.957650+0000","last_fullsized":"2024-06-06T06:58:53.957650+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387515+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T06:58:53.387515+0000","last_peered":"2024-06-06T06:58:53.387515+0000","last_clean":"2024-06-06T06:58:53.387515+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T06:58:53.387515+0000","last_undegraded":"2024-06-06T06:58:53.387515+0000","last_fullsized":"2024-06-06T06:58:53.387515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334137+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T06:59:51.334137+0000","last_peered":"2024-06-06T06:59:51.334137+0000","last_clean":"2024-06-06T06:59:51.334137+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T06:59:51.334137+0000","last_undegraded":"2024-06-06T06:59:51.334137+0000","last_fullsized":"2024-06-06T06:59:51.334137+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570872+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T06:58:53.570872+0000","last_peered":"2024-06-06T06:58:53.570872+0000","last_clean":"2024-06-06T06:58:53.570872+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T06:58:53.570872+0000","last_undegraded":"2024-06-06T06:58:53.570872+0000","last_fullsized":"2024-06-06T06:58:53.570872+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475593+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T06:58:53.475593+0000","last_peered":"2024-06-06T06:58:53.475593+0000","last_clean":"2024-06-06T06:58:53.475593+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T06:58:53.475593+0000","last_undegraded":"2024-06-06T06:58:53.475593+0000","last_fullsized":"2024-06-06T06:58:53.475593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337309+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T06:59:51.337309+0000","last_peered":"2024-06-06T06:59:51.337309+0000","last_clean":"2024-06-06T06:59:51.337309+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T06:59:51.337309+0000","last_undegraded":"2024-06-06T06:59:51.337309+0000","last_fullsized":"2024-06-06T06:59:51.337309+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544875+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T06:58:53.544875+0000","last_peered":"2024-06-06T06:58:53.544875+0000","last_clean":"2024-06-06T06:58:53.544875+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T06:58:53.544875+0000","last_undegraded":"2024-06-06T06:58:53.544875+0000","last_fullsized":"2024-06-06T06:58:53.544875+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'5","reported_seq":252,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:59.568181+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T06:59:59.568181+0000","last_peered":"2024-06-06T06:59:59.568181+0000","last_clean":"2024-06-06T06:59:59.568181+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T06:59:59.568181+0000","last_undegraded":"2024-06-06T06:59:59.568181+0000","last_fullsized":"2024-06-06T06:59:59.568181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347332+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T06:59:51.347332+0000","last_peered":"2024-06-06T06:59:51.347332+0000","last_clean":"2024-06-06T06:59:51.347332+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:59:51.347332+0000","last_undegraded":"2024-06-06T06:59:51.347332+0000","last_fullsized":"2024-06-06T06:59:51.347332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.390637+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T06:58:53.390637+0000","last_peered":"2024-06-06T06:58:53.390637+0000","last_clean":"2024-06-06T06:58:53.390637+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T06:58:53.390637+0000","last_undegraded":"2024-06-06T06:58:53.390637+0000","last_fullsized":"2024-06-06T06:58:53.390637+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475698+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T06:58:53.475698+0000","last_peered":"2024-06-06T06:58:53.475698+0000","last_clean":"2024-06-06T06:58:53.475698+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T06:58:53.475698+0000","last_undegraded":"2024-06-06T06:58:53.475698+0000","last_fullsized":"2024-06-06T06:58:53.475698+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337693+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T06:59:51.337693+0000","last_peered":"2024-06-06T06:59:51.337693+0000","last_clean":"2024-06-06T06:59:51.337693+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T06:59:51.337693+0000","last_undegraded":"2024-06-06T06:59:51.337693+0000","last_fullsized":"2024-06-06T06:59:51.337693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476959+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T06:58:53.476959+0000","last_peered":"2024-06-06T06:58:53.476959+0000","last_clean":"2024-06-06T06:58:53.476959+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T06:58:53.476959+0000","last_undegraded":"2024-06-06T06:58:53.476959+0000","last_fullsized":"2024-06-06T06:58:53.476959+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476353+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T06:58:53.476353+0000","last_peered":"2024-06-06T06:58:53.476353+0000","last_clean":"2024-06-06T06:58:53.476353+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T06:58:53.476353+0000","last_undegraded":"2024-06-06T06:58:53.476353+0000","last_fullsized":"2024-06-06T06:58:53.476353+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.348339+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T06:59:51.348339+0000","last_peered":"2024-06-06T06:59:51.348339+0000","last_clean":"2024-06-06T06:59:51.348339+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T06:59:51.348339+0000","last_undegraded":"2024-06-06T06:59:51.348339+0000","last_fullsized":"2024-06-06T06:59:51.348339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391104+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T06:58:53.391104+0000","last_peered":"2024-06-06T06:58:53.391104+0000","last_clean":"2024-06-06T06:58:53.391104+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T06:58:53.391104+0000","last_undegraded":"2024-06-06T06:58:53.391104+0000","last_fullsized":"2024-06-06T06:58:53.391104+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"100'38","reported_seq":295,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:59.568315+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T06:59:59.568315+0000","last_peered":"2024-06-06T06:59:59.568315+0000","last_clean":"2024-06-06T06:59:59.568315+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T06:59:59.568315+0000","last_undegraded":"2024-06-06T06:59:59.568315+0000","last_fullsized":"2024-06-06T06:59:59.568315+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63174,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":212,"num_read_kb":168,"num_write":40,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337677+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T06:59:51.337677+0000","last_peered":"2024-06-06T06:59:51.337677+0000","last_clean":"2024-06-06T06:59:51.337677+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T06:59:51.337677+0000","last_undegraded":"2024-06-06T06:59:51.337677+0000","last_fullsized":"2024-06-06T06:59:51.337677+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476659+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T06:58:53.476659+0000","last_peered":"2024-06-06T06:58:53.476659+0000","last_clean":"2024-06-06T06:58:53.476659+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T06:58:53.476659+0000","last_undegraded":"2024-06-06T06:58:53.476659+0000","last_fullsized":"2024-06-06T06:58:53.476659+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388408+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T06:58:53.388408+0000","last_peered":"2024-06-06T06:58:53.388408+0000","last_clean":"2024-06-06T06:58:53.388408+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T06:58:53.388408+0000","last_undegraded":"2024-06-06T06:58:53.388408+0000","last_fullsized":"2024-06-06T06:58:53.388408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.323471+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T06:59:51.323471+0000","last_peered":"2024-06-06T06:59:51.323471+0000","last_clean":"2024-06-06T06:59:51.323471+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T06:59:51.323471+0000","last_undegraded":"2024-06-06T06:59:51.323471+0000","last_fullsized":"2024-06-06T06:59:51.323471+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957376+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T06:58:53.957376+0000","last_peered":"2024-06-06T06:58:53.957376+0000","last_clean":"2024-06-06T06:58:53.957376+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T06:58:53.957376+0000","last_undegraded":"2024-06-06T06:58:53.957376+0000","last_fullsized":"2024-06-06T06:58:53.957376+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475971+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T06:58:53.475971+0000","last_peered":"2024-06-06T06:58:53.475971+0000","last_clean":"2024-06-06T06:58:53.475971+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T06:58:53.475971+0000","last_undegraded":"2024-06-06T06:58:53.475971+0000","last_fullsized":"2024-06-06T06:58:53.475971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337551+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T06:59:51.337551+0000","last_peered":"2024-06-06T06:59:51.337551+0000","last_clean":"2024-06-06T06:59:51.337551+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T06:59:51.337551+0000","last_undegraded":"2024-06-06T06:59:51.337551+0000","last_fullsized":"2024-06-06T06:59:51.337551+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391531+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T06:58:53.391531+0000","last_peered":"2024-06-06T06:58:53.391531+0000","last_clean":"2024-06-06T06:58:53.391531+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T06:58:53.391531+0000","last_undegraded":"2024-06-06T06:58:53.391531+0000","last_fullsized":"2024-06-06T06:58:53.391531+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"100'7","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:59.567363+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T06:59:59.567363+0000","last_peered":"2024-06-06T06:59:59.567363+0000","last_clean":"2024-06-06T06:59:59.567363+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T06:59:59.567363+0000","last_undegraded":"2024-06-06T06:59:59.567363+0000","last_fullsized":"2024-06-06T06:59:59.567363+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1119,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344670+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T06:59:51.344670+0000","last_peered":"2024-06-06T06:59:51.344670+0000","last_clean":"2024-06-06T06:59:51.344670+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T06:59:51.344670+0000","last_undegraded":"2024-06-06T06:59:51.344670+0000","last_fullsized":"2024-06-06T06:59:51.344670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388088+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T06:58:53.388088+0000","last_peered":"2024-06-06T06:58:53.388088+0000","last_clean":"2024-06-06T06:58:53.388088+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T06:58:53.388088+0000","last_undegraded":"2024-06-06T06:58:53.388088+0000","last_fullsized":"2024-06-06T06:58:53.388088+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":63,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391207+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T06:58:53.391207+0000","last_peered":"2024-06-06T06:58:53.391207+0000","last_clean":"2024-06-06T06:58:53.391207+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T06:58:53.391207+0000","last_undegraded":"2024-06-06T06:58:53.391207+0000","last_fullsized":"2024-06-06T06:58:53.391207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347249+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T06:59:51.347249+0000","last_peered":"2024-06-06T06:59:51.347249+0000","last_clean":"2024-06-06T06:59:51.347249+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T06:59:51.347249+0000","last_undegraded":"2024-06-06T06:59:51.347249+0000","last_fullsized":"2024-06-06T06:59:51.347249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475495+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T06:58:53.475495+0000","last_peered":"2024-06-06T06:58:53.475495+0000","last_clean":"2024-06-06T06:58:53.475495+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T06:58:53.475495+0000","last_undegraded":"2024-06-06T06:58:53.475495+0000","last_fullsized":"2024-06-06T06:58:53.475495+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"98'2","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.392084+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T06:58:53.392084+0000","last_peered":"2024-06-06T06:58:53.392084+0000","last_clean":"2024-06-06T06:58:53.392084+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T06:58:53.392084+0000","last_undegraded":"2024-06-06T06:58:53.392084+0000","last_fullsized":"2024-06-06T06:58:53.392084+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346527+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T06:59:51.346527+0000","last_peered":"2024-06-06T06:59:51.346527+0000","last_clean":"2024-06-06T06:59:51.346527+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T06:59:51.346527+0000","last_undegraded":"2024-06-06T06:59:51.346527+0000","last_fullsized":"2024-06-06T06:59:51.346527+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391563+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T06:58:53.391563+0000","last_peered":"2024-06-06T06:58:53.391563+0000","last_clean":"2024-06-06T06:58:53.391563+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T06:58:53.391563+0000","last_undegraded":"2024-06-06T06:58:53.391563+0000","last_fullsized":"2024-06-06T06:58:53.391563+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.390993+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T06:58:53.390993+0000","last_peered":"2024-06-06T06:58:53.390993+0000","last_clean":"2024-06-06T06:58:53.390993+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T06:58:53.390993+0000","last_undegraded":"2024-06-06T06:58:53.390993+0000","last_fullsized":"2024-06-06T06:58:53.390993+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.324048+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:59:51.324048+0000","last_peered":"2024-06-06T06:59:51.324048+0000","last_clean":"2024-06-06T06:59:51.324048+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T06:59:51.324048+0000","last_undegraded":"2024-06-06T06:59:51.324048+0000","last_fullsized":"2024-06-06T06:59:51.324048+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387755+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T06:58:53.387755+0000","last_peered":"2024-06-06T06:58:53.387755+0000","last_clean":"2024-06-06T06:58:53.387755+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T06:58:53.387755+0000","last_undegraded":"2024-06-06T06:58:53.387755+0000","last_fullsized":"2024-06-06T06:58:53.387755+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"98'5","reported_seq":148,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:55.562046+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T06:59:55.562046+0000","last_peered":"2024-06-06T06:59:55.562046+0000","last_clean":"2024-06-06T06:59:55.562046+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T06:59:55.562046+0000","last_undegraded":"2024-06-06T06:59:55.562046+0000","last_fullsized":"2024-06-06T06:59:55.562046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":99,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.334535+0000","last_change":"2024-06-06T06:58:53.571939+0000","last_active":"2024-06-06T06:59:51.334535+0000","last_peered":"2024-06-06T06:59:51.334535+0000","last_clean":"2024-06-06T06:59:51.334535+0000","last_became_active":"2024-06-06T06:58:53.571590+0000","last_became_peered":"2024-06-06T06:58:53.571590+0000","last_unstale":"2024-06-06T06:59:51.334535+0000","last_undegraded":"2024-06-06T06:59:51.334535+0000","last_fullsized":"2024-06-06T06:59:51.334535+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:10.858649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387951+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T06:58:53.387951+0000","last_peered":"2024-06-06T06:58:53.387951+0000","last_clean":"2024-06-06T06:58:53.387951+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T06:58:53.387951+0000","last_undegraded":"2024-06-06T06:58:53.387951+0000","last_fullsized":"2024-06-06T06:58:53.387951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342886+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T06:59:51.342886+0000","last_peered":"2024-06-06T06:59:51.342886+0000","last_clean":"2024-06-06T06:59:51.342886+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T06:59:51.342886+0000","last_undegraded":"2024-06-06T06:59:51.342886+0000","last_fullsized":"2024-06-06T06:59:51.342886+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'6","reported_seq":149,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:54.560270+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T06:59:54.560270+0000","last_peered":"2024-06-06T06:59:54.560270+0000","last_clean":"2024-06-06T06:59:54.560270+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T06:59:54.560270+0000","last_undegraded":"2024-06-06T06:59:54.560270+0000","last_fullsized":"2024-06-06T06:59:54.560270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475368+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T06:58:53.475368+0000","last_peered":"2024-06-06T06:58:53.475368+0000","last_clean":"2024-06-06T06:58:53.475368+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T06:58:53.475368+0000","last_undegraded":"2024-06-06T06:58:53.475368+0000","last_fullsized":"2024-06-06T06:58:53.475368+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339323+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T06:59:51.339323+0000","last_peered":"2024-06-06T06:59:51.339323+0000","last_clean":"2024-06-06T06:59:51.339323+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T06:59:51.339323+0000","last_undegraded":"2024-06-06T06:59:51.339323+0000","last_fullsized":"2024-06-06T06:59:51.339323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476637+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T06:58:53.476637+0000","last_peered":"2024-06-06T06:58:53.476637+0000","last_clean":"2024-06-06T06:58:53.476637+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T06:58:53.476637+0000","last_undegraded":"2024-06-06T06:58:53.476637+0000","last_fullsized":"2024-06-06T06:58:53.476637+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":376,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.474652+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T06:58:53.474652+0000","last_peered":"2024-06-06T06:58:53.474652+0000","last_clean":"2024-06-06T06:58:53.474652+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T06:58:53.474652+0000","last_undegraded":"2024-06-06T06:58:53.474652+0000","last_fullsized":"2024-06-06T06:58:53.474652+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476129+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T06:58:53.476129+0000","last_peered":"2024-06-06T06:58:53.476129+0000","last_clean":"2024-06-06T06:58:53.476129+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T06:58:53.476129+0000","last_undegraded":"2024-06-06T06:58:53.476129+0000","last_fullsized":"2024-06-06T06:58:53.476129+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.327623+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T06:59:51.327623+0000","last_peered":"2024-06-06T06:59:51.327623+0000","last_clean":"2024-06-06T06:59:51.327623+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T06:59:51.327623+0000","last_undegraded":"2024-06-06T06:59:51.327623+0000","last_fullsized":"2024-06-06T06:59:51.327623+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":54,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.474874+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T06:58:53.474874+0000","last_peered":"2024-06-06T06:58:53.474874+0000","last_clean":"2024-06-06T06:58:53.474874+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T06:58:53.474874+0000","last_undegraded":"2024-06-06T06:58:53.474874+0000","last_fullsized":"2024-06-06T06:58:53.474874+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.957368+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T06:58:53.957368+0000","last_peered":"2024-06-06T06:58:53.957368+0000","last_clean":"2024-06-06T06:58:53.957368+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T06:58:53.957368+0000","last_undegraded":"2024-06-06T06:58:53.957368+0000","last_fullsized":"2024-06-06T06:58:53.957368+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391881+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T06:58:53.391881+0000","last_peered":"2024-06-06T06:58:53.391881+0000","last_clean":"2024-06-06T06:58:53.391881+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T06:58:53.391881+0000","last_undegraded":"2024-06-06T06:58:53.391881+0000","last_fullsized":"2024-06-06T06:58:53.391881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.329338+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T06:59:51.329338+0000","last_peered":"2024-06-06T06:59:51.329338+0000","last_clean":"2024-06-06T06:59:51.329338+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T06:59:51.329338+0000","last_undegraded":"2024-06-06T06:59:51.329338+0000","last_fullsized":"2024-06-06T06:59:51.329338+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544951+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T06:58:53.544951+0000","last_peered":"2024-06-06T06:58:53.544951+0000","last_clean":"2024-06-06T06:58:53.544951+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T06:58:53.544951+0000","last_undegraded":"2024-06-06T06:58:53.544951+0000","last_fullsized":"2024-06-06T06:58:53.544951+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"100'17","reported_seq":62,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:14.377279+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T06:59:14.377279+0000","last_peered":"2024-06-06T06:59:14.377279+0000","last_clean":"2024-06-06T06:59:14.377279+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T06:59:14.377279+0000","last_undegraded":"2024-06-06T06:59:14.377279+0000","last_fullsized":"2024-06-06T06:59:14.377279+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336522+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T06:59:51.336522+0000","last_peered":"2024-06-06T06:59:51.336522+0000","last_clean":"2024-06-06T06:59:51.336522+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T06:59:51.336522+0000","last_undegraded":"2024-06-06T06:59:51.336522+0000","last_fullsized":"2024-06-06T06:59:51.336522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388108+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T06:58:53.388108+0000","last_peered":"2024-06-06T06:58:53.388108+0000","last_clean":"2024-06-06T06:58:53.388108+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T06:58:53.388108+0000","last_undegraded":"2024-06-06T06:58:53.388108+0000","last_fullsized":"2024-06-06T06:58:53.388108+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476799+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T06:58:53.476799+0000","last_peered":"2024-06-06T06:58:53.476799+0000","last_clean":"2024-06-06T06:58:53.476799+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T06:58:53.476799+0000","last_undegraded":"2024-06-06T06:58:53.476799+0000","last_fullsized":"2024-06-06T06:58:53.476799+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337370+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T06:59:51.337370+0000","last_peered":"2024-06-06T06:59:51.337370+0000","last_clean":"2024-06-06T06:59:51.337370+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T06:59:51.337370+0000","last_undegraded":"2024-06-06T06:59:51.337370+0000","last_fullsized":"2024-06-06T06:59:51.337370+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476257+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T06:58:53.476257+0000","last_peered":"2024-06-06T06:58:53.476257+0000","last_clean":"2024-06-06T06:58:53.476257+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T06:58:53.476257+0000","last_undegraded":"2024-06-06T06:58:53.476257+0000","last_fullsized":"2024-06-06T06:58:53.476257+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":53,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570702+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T06:58:53.570702+0000","last_peered":"2024-06-06T06:58:53.570702+0000","last_clean":"2024-06-06T06:58:53.570702+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T06:58:53.570702+0000","last_undegraded":"2024-06-06T06:58:53.570702+0000","last_fullsized":"2024-06-06T06:58:53.570702+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":41,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.336782+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T06:59:51.336782+0000","last_peered":"2024-06-06T06:59:51.336782+0000","last_clean":"2024-06-06T06:59:51.336782+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T06:59:51.336782+0000","last_undegraded":"2024-06-06T06:59:51.336782+0000","last_fullsized":"2024-06-06T06:59:51.336782+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":36,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391470+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T06:58:53.391470+0000","last_peered":"2024-06-06T06:58:53.391470+0000","last_clean":"2024-06-06T06:58:53.391470+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T06:58:53.391470+0000","last_undegraded":"2024-06-06T06:58:53.391470+0000","last_fullsized":"2024-06-06T06:58:53.391470+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.544952+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T06:58:53.544952+0000","last_peered":"2024-06-06T06:58:53.544952+0000","last_clean":"2024-06-06T06:58:53.544952+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T06:58:53.544952+0000","last_undegraded":"2024-06-06T06:58:53.544952+0000","last_fullsized":"2024-06-06T06:58:53.544952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337845+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T06:59:51.337845+0000","last_peered":"2024-06-06T06:59:51.337845+0000","last_clean":"2024-06-06T06:59:51.337845+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T06:59:51.337845+0000","last_undegraded":"2024-06-06T06:59:51.337845+0000","last_fullsized":"2024-06-06T06:59:51.337845+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":30,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476251+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T06:58:53.476251+0000","last_peered":"2024-06-06T06:58:53.476251+0000","last_clean":"2024-06-06T06:58:53.476251+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T06:58:53.476251+0000","last_undegraded":"2024-06-06T06:58:53.476251+0000","last_fullsized":"2024-06-06T06:58:53.476251+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"98'3","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.388434+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T06:58:53.388434+0000","last_peered":"2024-06-06T06:58:53.388434+0000","last_clean":"2024-06-06T06:58:53.388434+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T06:58:53.388434+0000","last_undegraded":"2024-06-06T06:58:53.388434+0000","last_fullsized":"2024-06-06T06:58:53.388434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.332688+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T06:59:51.332688+0000","last_peered":"2024-06-06T06:59:51.332688+0000","last_clean":"2024-06-06T06:59:51.332688+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T06:59:51.332688+0000","last_undegraded":"2024-06-06T06:59:51.332688+0000","last_fullsized":"2024-06-06T06:59:51.332688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476177+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T06:58:53.476177+0000","last_peered":"2024-06-06T06:58:53.476177+0000","last_clean":"2024-06-06T06:58:53.476177+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T06:58:53.476177+0000","last_undegraded":"2024-06-06T06:58:53.476177+0000","last_fullsized":"2024-06-06T06:58:53.476177+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.391359+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T06:58:53.391359+0000","last_peered":"2024-06-06T06:58:53.391359+0000","last_clean":"2024-06-06T06:58:53.391359+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T06:58:53.391359+0000","last_undegraded":"2024-06-06T06:58:53.391359+0000","last_fullsized":"2024-06-06T06:58:53.391359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":42,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344930+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T06:59:51.344930+0000","last_peered":"2024-06-06T06:59:51.344930+0000","last_clean":"2024-06-06T06:59:51.344930+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T06:59:51.344930+0000","last_undegraded":"2024-06-06T06:59:51.344930+0000","last_fullsized":"2024-06-06T06:59:51.344930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570750+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T06:58:53.570750+0000","last_peered":"2024-06-06T06:58:53.570750+0000","last_clean":"2024-06-06T06:58:53.570750+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T06:58:53.570750+0000","last_undegraded":"2024-06-06T06:58:53.570750+0000","last_fullsized":"2024-06-06T06:58:53.570750+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"100'7","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:59.568973+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T06:59:59.568973+0000","last_peered":"2024-06-06T06:59:59.568973+0000","last_clean":"2024-06-06T06:59:59.568973+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T06:59:59.568973+0000","last_undegraded":"2024-06-06T06:59:59.568973+0000","last_fullsized":"2024-06-06T06:59:59.568973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1701,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":44,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.345155+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T06:59:51.345155+0000","last_peered":"2024-06-06T06:59:51.345155+0000","last_clean":"2024-06-06T06:59:51.345155+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T06:59:51.345155+0000","last_undegraded":"2024-06-06T06:59:51.345155+0000","last_fullsized":"2024-06-06T06:59:51.345155+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.570771+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T06:58:53.570771+0000","last_peered":"2024-06-06T06:58:53.570771+0000","last_clean":"2024-06-06T06:58:53.570771+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T06:58:53.570771+0000","last_undegraded":"2024-06-06T06:58:53.570771+0000","last_fullsized":"2024-06-06T06:58:53.570771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"98'3","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.476328+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T06:58:53.476328+0000","last_peered":"2024-06-06T06:58:53.476328+0000","last_clean":"2024-06-06T06:58:53.476328+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T06:58:53.476328+0000","last_undegraded":"2024-06-06T06:58:53.476328+0000","last_fullsized":"2024-06-06T06:58:53.476328+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344405+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T06:59:51.344405+0000","last_peered":"2024-06-06T06:59:51.344405+0000","last_clean":"2024-06-06T06:59:51.344405+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T06:59:51.344405+0000","last_undegraded":"2024-06-06T06:59:51.344405+0000","last_fullsized":"2024-06-06T06:59:51.344405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.475379+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T06:58:53.475379+0000","last_peered":"2024-06-06T06:58:53.475379+0000","last_clean":"2024-06-06T06:58:53.475379+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T06:58:53.475379+0000","last_undegraded":"2024-06-06T06:58:53.475379+0000","last_fullsized":"2024-06-06T06:58:53.475379+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:58:53.387985+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T06:58:53.387985+0000","last_peered":"2024-06-06T06:58:53.387985+0000","last_clean":"2024-06-06T06:58:53.387985+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T06:58:53.387985+0000","last_undegraded":"2024-06-06T06:58:53.387985+0000","last_fullsized":"2024-06-06T06:58:53.387985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":46,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.345005+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T06:59:51.345005+0000","last_peered":"2024-06-06T06:59:51.345005+0000","last_clean":"2024-06-06T06:59:51.345005+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T06:59:51.345005+0000","last_undegraded":"2024-06-06T06:59:51.345005+0000","last_fullsized":"2024-06-06T06:59:51.345005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":48,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.347846+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T06:59:51.347846+0000","last_peered":"2024-06-06T06:59:51.347846+0000","last_clean":"2024-06-06T06:59:51.347846+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T06:59:51.347846+0000","last_undegraded":"2024-06-06T06:59:51.347846+0000","last_fullsized":"2024-06-06T06:59:51.347846+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":55,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.344430+0000","last_change":"2024-06-06T06:58:53.478114+0000","last_active":"2024-06-06T06:59:51.344430+0000","last_peered":"2024-06-06T06:59:51.344430+0000","last_clean":"2024-06-06T06:59:51.344430+0000","last_became_active":"2024-06-06T06:58:53.477664+0000","last_became_peered":"2024-06-06T06:58:53.477664+0000","last_unstale":"2024-06-06T06:59:51.344430+0000","last_undegraded":"2024-06-06T06:59:51.344430+0000","last_fullsized":"2024-06-06T06:59:51.344430+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:01.541804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.339964+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T06:59:51.339964+0000","last_peered":"2024-06-06T06:59:51.339964+0000","last_clean":"2024-06-06T06:59:51.339964+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T06:59:51.339964+0000","last_undegraded":"2024-06-06T06:59:51.339964+0000","last_fullsized":"2024-06-06T06:59:51.339964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337362+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T06:59:51.337362+0000","last_peered":"2024-06-06T06:59:51.337362+0000","last_clean":"2024-06-06T06:59:51.337362+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T06:59:51.337362+0000","last_undegraded":"2024-06-06T06:59:51.337362+0000","last_fullsized":"2024-06-06T06:59:51.337362+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337810+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T06:59:51.337810+0000","last_peered":"2024-06-06T06:59:51.337810+0000","last_clean":"2024-06-06T06:59:51.337810+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T06:59:51.337810+0000","last_undegraded":"2024-06-06T06:59:51.337810+0000","last_fullsized":"2024-06-06T06:59:51.337810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":45,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346547+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T06:59:51.346547+0000","last_peered":"2024-06-06T06:59:51.346547+0000","last_clean":"2024-06-06T06:59:51.346547+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T06:59:51.346547+0000","last_undegraded":"2024-06-06T06:59:51.346547+0000","last_fullsized":"2024-06-06T06:59:51.346547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":52,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342784+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T06:59:51.342784+0000","last_peered":"2024-06-06T06:59:51.342784+0000","last_clean":"2024-06-06T06:59:51.342784+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T06:59:51.342784+0000","last_undegraded":"2024-06-06T06:59:51.342784+0000","last_fullsized":"2024-06-06T06:59:51.342784+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":13,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.331626+0000","last_change":"2024-06-06T06:58:53.478046+0000","last_active":"2024-06-06T06:59:51.331626+0000","last_peered":"2024-06-06T06:59:51.331626+0000","last_clean":"2024-06-06T06:59:51.331626+0000","last_became_active":"2024-06-06T06:58:53.477722+0000","last_became_peered":"2024-06-06T06:58:53.477722+0000","last_unstale":"2024-06-06T06:59:51.331626+0000","last_undegraded":"2024-06-06T06:59:51.331626+0000","last_fullsized":"2024-06-06T06:59:51.331626+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:22.491510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":43,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.346457+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T06:59:51.346457+0000","last_peered":"2024-06-06T06:59:51.346457+0000","last_clean":"2024-06-06T06:59:51.346457+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T06:59:51.346457+0000","last_undegraded":"2024-06-06T06:59:51.346457+0000","last_fullsized":"2024-06-06T06:59:51.346457+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.333531+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T06:59:51.333531+0000","last_peered":"2024-06-06T06:59:51.333531+0000","last_clean":"2024-06-06T06:59:51.333531+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T06:59:51.333531+0000","last_undegraded":"2024-06-06T06:59:51.333531+0000","last_fullsized":"2024-06-06T06:59:51.333531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":49,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.340143+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T06:59:51.340143+0000","last_peered":"2024-06-06T06:59:51.340143+0000","last_clean":"2024-06-06T06:59:51.340143+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T06:59:51.340143+0000","last_undegraded":"2024-06-06T06:59:51.340143+0000","last_fullsized":"2024-06-06T06:59:51.340143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":50,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.337935+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T06:59:51.337935+0000","last_peered":"2024-06-06T06:59:51.337935+0000","last_clean":"2024-06-06T06:59:51.337935+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T06:59:51.337935+0000","last_undegraded":"2024-06-06T06:59:51.337935+0000","last_fullsized":"2024-06-06T06:59:51.337935+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":47,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-06-06T06:59:51.342252+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T06:59:51.342252+0000","last_peered":"2024-06-06T06:59:51.342252+0000","last_clean":"2024-06-06T06:59:51.342252+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T06:59:51.342252+0000","last_undegraded":"2024-06-06T06:59:51.342252+0000","last_fullsized":"2024-06-06T06:59:51.342252+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":285685,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70658,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1048,"num_read_kb":601,"num_write":259,"num_write_kb":149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":211974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":223,"ondisk_log_size":223,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351027,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":574163,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612664,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28248,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716904,"statfs":{"total":95995035648,"available":95966109696,"internally_reserved":0,"allocated":1261568,"data_stored":1224153,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776188,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32288,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712864,"statfs":{"total":95995035648,"available":95961972736,"internally_reserved":0,"allocated":1138688,"data_stored":577658,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037826,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1130496,"data_stored":573940,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299463,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28172,"kb_used_data":1156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716980,"statfs":{"total":95995035648,"available":95966187520,"internally_reserved":0,"allocated":1183744,"data_stored":1170577,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462988,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":569066,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574949,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36488,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708664,"statfs":{"total":95995035648,"available":95957671936,"internally_reserved":0,"allocated":1179648,"data_stored":633270,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738474,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":571053,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1191936,"data_stored":633399,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149787,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":568774,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888151,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":574293,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724625,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36500,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708652,"statfs":{"total":95995035648,"available":95957659648,"internally_reserved":0,"allocated":1191936,"data_stored":1165628,"data_compressed":68588,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34690,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2709,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":64926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":65055,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:00:02.528 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-06T07:00:02.528 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-06T07:00:02.528 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-06T07:00:02.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-06T07:00:02.771 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:02.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:02 smithi073 ceph-mon[32303]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-06T07:00:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:02 smithi136 ceph-mon[35901]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-06T07:00:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:02 smithi170 ceph-mon[36108]: pgmap v395: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 597 B/s wr, 12 op/s 2024-06-06T07:00:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:03 smithi073 ceph-mon[32303]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:03 smithi136 ceph-mon[35901]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:03.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:03 smithi170 ceph-mon[36108]: from='client.14988 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:00:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 -- 172.21.15.73:0/174462705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0107400 msgr2=0x7fd0f01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 --2- 172.21.15.73:0/174462705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0107400 0x7fd0f01077e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8009990 tx=0x7fd0e8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 -- 172.21.15.73:0/174462705 shutdown_connections 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 --2- 172.21.15.73:0/174462705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 --2- 172.21.15.73:0/174462705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0107400 0x7fd0f01077e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 --2- 172.21.15.73:0/174462705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0101390 0x7fd0f0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 -- 172.21.15.73:0/174462705 >> 172.21.15.73:0/174462705 conn(0x7fd0f0078340 msgr2=0x7fd0f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 -- 172.21.15.73:0/174462705 shutdown_connections 2024-06-06T07:00:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.389+0000 7fd0f4933700 1 -- 172.21.15.73:0/174462705 wait complete. 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 Processor -- start 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 -- start start 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0107400 0x7fd0f010f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f0112640 con 0x7fd0f0101390 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f01124c0 con 0x7fd0f0107400 2024-06-06T07:00:04.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0f4933700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f01127c0 con 0x7fd0f0101d50 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0ee59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0ee59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45844/0 (socket says 172.21.15.73:45844) 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.390+0000 7fd0ee59c700 1 -- 172.21.15.73:0/3960104867 learned_addr learned my addr 172.21.15.73:0/3960104867 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0eed9d700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0107400 0x7fd0f010f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0edd9b700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 -- 172.21.15.73:0/3960104867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 msgr2=0x7fd0f010f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 -- 172.21.15.73:0/3960104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0107400 msgr2=0x7fd0f010f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0107400 0x7fd0f010f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 -- 172.21.15.73:0/3960104867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f01a7100 con 0x7fd0f0101390 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0edd9b700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0ee59c700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8000c00 tx=0x7fd0e8003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:04.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8027070 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e800bdc0 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f01a7390 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e8022d30 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.391+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f01a77e0 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.392+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f0069180 con 0x7fd0f0101390 2024-06-06T07:00:04.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.393+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0e802a070 con 0x7fd0f0101390 2024-06-06T07:00:04.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.394+0000 7fd0e77fe700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 0x7fd0dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:04.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.394+0000 7fd0edd9b700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 0x7fd0dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:04.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.394+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd0e80d0510 con 0x7fd0f0101390 2024-06-06T07:00:04.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.394+0000 7fd0edd9b700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 0x7fd0dc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0078af0 tx=0x7fd0d8005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:04.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.397+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0e8013070 con 0x7fd0f0101390 2024-06-06T07:00:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:04 smithi073 ceph-mon[32303]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:00:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.643+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fd0f002de60 con 0x7fd0f0101390 2024-06-06T07:00:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.644+0000 7fd0e77fe700 1 -- 172.21.15.73:0/3960104867 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fd0e809d130 con 0x7fd0f0101390 2024-06-06T07:00:04.645 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:04.645 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 msgr2=0x7fd0dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 0x7fd0dc0a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0078af0 tx=0x7fd0d8005f50 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 msgr2=0x7fd0f010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0e8000c00 tx=0x7fd0e8003a80 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 shutdown_connections 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd0f0107400 0x7fd0f010f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0f0101390 0x7fd0f010ecd0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd0dc0a2c20 0x7fd0dc0a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 --2- 172.21.15.73:0/3960104867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd0f0101d50 0x7fd0f010f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:04.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 >> 172.21.15.73:0/3960104867 conn(0x7fd0f0078340 msgr2=0x7fd0f0100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 shutdown_connections 2024-06-06T07:00:04.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:04.646+0000 7fd0f4933700 1 -- 172.21.15.73:0/3960104867 wait complete. 2024-06-06T07:00:04.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:04 smithi136 ceph-mon[35901]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:00:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:04 smithi170 ceph-mon[36108]: pgmap v396: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:00:05.539 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-06T07:00:05.540 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:00:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:05 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3960104867' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:00:05.782 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:05 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3960104867' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:00:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:05 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3960104867' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-06T07:00:06.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:06 smithi073 ceph-mon[32303]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-06T07:00:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:06 smithi136 ceph-mon[35901]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-06T07:00:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:06 smithi170 ceph-mon[36108]: pgmap v397: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 -- 172.21.15.73:0/1513991624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff738101fd0 msgr2=0x7ff738105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/1513991624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff738101fd0 0x7ff738105fd0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff7280099c0 tx=0x7ff728009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 -- 172.21.15.73:0/1513991624 shutdown_connections 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/1513991624 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff7381066b0 0x7ff73810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/1513991624 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101620 0x7ff738101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/1513991624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff738101fd0 0x7ff738105fd0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 -- 172.21.15.73:0/1513991624 >> 172.21.15.73:0/1513991624 conn(0x7ff738078360 msgr2=0x7ff738078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 -- 172.21.15.73:0/1513991624 shutdown_connections 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.483+0000 7ff73fbb2700 1 -- 172.21.15.73:0/1513991624 wait complete. 2024-06-06T07:00:07.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 Processor -- start 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 -- start start 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff738101620 0x7ff73819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101fd0 0x7ff73819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff738111620 con 0x7ff7381066b0 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381114a0 con 0x7ff738101fd0 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73fbb2700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff7381117a0 con 0x7ff738101620 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73e14f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73d94e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff738101620 0x7ff73819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73e14f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44670/0 (socket says 172.21.15.73:44670) 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73e14f700 1 -- 172.21.15.73:0/2881688891 learned_addr learned my addr 172.21.15.73:0/2881688891 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.484+0000 7ff73d14d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101fd0 0x7ff73819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 -- 172.21.15.73:0/2881688891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff738101620 msgr2=0x7ff73819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:07.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff738101620 0x7ff73819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 -- 172.21.15.73:0/2881688891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101fd0 msgr2=0x7ff73819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101fd0 0x7ff73819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 -- 172.21.15.73:0/2881688891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7381991f0 con 0x7ff7381066b0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73e14f700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff73000b550 tx=0x7ff73000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730015020 con 0x7ff7381066b0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff730011ce0 con 0x7ff7381066b0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7381994e0 con 0x7ff7381066b0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff730010cd0 con 0x7ff7381066b0 2024-06-06T07:00:07.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.485+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff738102fa0 con 0x7ff7381066b0 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.486+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff738069180 con 0x7ff7381066b0 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.487+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff730018020 con 0x7ff7381066b0 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.488+0000 7ff72effd700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 0x7ff7240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.488+0000 7ff73d94e700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 0x7ff7240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.488+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7ff7300cefb0 con 0x7ff7381066b0 2024-06-06T07:00:07.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.488+0000 7ff73d94e700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 0x7ff7240a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff738199bd0 tx=0x7ff73400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:07.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.491+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff73001f020 con 0x7ff7381066b0 2024-06-06T07:00:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:07 smithi073 ceph-mon[32303]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T07:00:07.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.732+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff738199fc0 con 0x7ff7381066b0 2024-06-06T07:00:07.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.733+0000 7ff72effd700 1 -- 172.21.15.73:0/2881688891 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7ff73009bb40 con 0x7ff7381066b0 2024-06-06T07:00:07.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:07.734 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T07:00:07.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 msgr2=0x7ff7240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 0x7ff7240a5160 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff738199bd0 tx=0x7ff73400d040 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 msgr2=0x7ff738198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7ff73000b550 tx=0x7ff73000b860 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 shutdown_connections 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff7240a2ca0 0x7ff7240a5160 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff738101620 0x7ff73819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7381066b0 0x7ff738198a90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 --2- 172.21.15.73:0/2881688891 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff738101fd0 0x7ff73819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 >> 172.21.15.73:0/2881688891 conn(0x7ff738078360 msgr2=0x7ff73806af00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 shutdown_connections 2024-06-06T07:00:07.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:07.735+0000 7ff73fbb2700 1 -- 172.21.15.73:0/2881688891 wait complete. 2024-06-06T07:00:07.738 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:07 smithi136 ceph-mon[35901]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T07:00:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:07 smithi170 ceph-mon[36108]: pgmap v398: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-06T07:00:08.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:00:08.735 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:08.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:08 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2881688891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:08.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:08 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2881688891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:08.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:08 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2881688891' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:09.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:09 smithi073 ceph-mon[32303]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:09 smithi136 ceph-mon[35901]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:09 smithi170 ceph-mon[36108]: pgmap v399: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:10.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 -- 172.21.15.73:0/1260607749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101fd0 msgr2=0x7efd64105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:10.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 --2- 172.21.15.73:0/1260607749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101fd0 0x7efd64105fd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd540099c0 tx=0x7efd54009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 -- 172.21.15.73:0/1260607749 shutdown_connections 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 --2- 172.21.15.73:0/1260607749 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd641066b0 0x7efd6410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 --2- 172.21.15.73:0/1260607749 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd64101620 0x7efd64101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 --2- 172.21.15.73:0/1260607749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101fd0 0x7efd64105fd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 -- 172.21.15.73:0/1260607749 >> 172.21.15.73:0/1260607749 conn(0x7efd64078360 msgr2=0x7efd64078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 -- 172.21.15.73:0/1260607749 shutdown_connections 2024-06-06T07:00:10.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.508+0000 7efd6b457700 1 -- 172.21.15.73:0/1260607749 wait complete. 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 Processor -- start 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 -- start start 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd64101fd0 0x7efd6419dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 0x7efd64197800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd64111520 con 0x7efd64101620 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7efd641113a0 con 0x7efd641066b0 2024-06-06T07:00:10.512 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd6b457700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7efd641116a0 con 0x7efd64101fd0 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd691f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd691f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44688/0 (socket says 172.21.15.73:44688) 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.509+0000 7efd691f3700 1 -- 172.21.15.73:0/2266590235 learned_addr learned my addr 172.21.15.73:0/2266590235 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd699f4700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 0x7efd64197800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd689f2700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd64101fd0 0x7efd6419dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 -- 172.21.15.73:0/2266590235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd64101fd0 msgr2=0x7efd6419dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd64101fd0 0x7efd6419dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 -- 172.21.15.73:0/2266590235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 msgr2=0x7efd64197800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 0x7efd64197800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 -- 172.21.15.73:0/2266590235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd64197fe0 con 0x7efd64101620 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd699f4700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 0x7efd64197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd691f3700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd60007f10 tx=0x7efd6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60003a00 con 0x7efd64101620 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd641982d0 con 0x7efd64101620 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efd60003ba0 con 0x7efd64101620 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd60015660 con 0x7efd64101620 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.510+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd641a44e0 con 0x7efd64101620 2024-06-06T07:00:10.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.511+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd64102ab0 con 0x7efd64101620 2024-06-06T07:00:10.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.512+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efd60004000 con 0x7efd64101620 2024-06-06T07:00:10.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.513+0000 7efd5a7fc700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 0x7efd500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:10.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.513+0000 7efd689f2700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 0x7efd500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:10.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.513+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7efd600cf8c0 con 0x7efd64101620 2024-06-06T07:00:10.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.513+0000 7efd689f2700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 0x7efd500a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.516+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efd60023070 con 0x7efd64101620 2024-06-06T07:00:10.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.759+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd6402dca0 con 0x7efd64101620 2024-06-06T07:00:10.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.759+0000 7efd5a7fc700 1 -- 172.21.15.73:0/2266590235 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7efd60010070 con 0x7efd64101620 2024-06-06T07:00:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:10.761 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 msgr2=0x7efd500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 0x7efd500a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd54005fd0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 msgr2=0x7efd6419d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7efd60007f10 tx=0x7efd6000d3b0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 shutdown_connections 2024-06-06T07:00:10.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7efd641066b0 0x7efd64197800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd64101620 0x7efd6419d680 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7efd500a2c20 0x7efd500a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 --2- 172.21.15.73:0/2266590235 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7efd64101fd0 0x7efd6419dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 >> 172.21.15.73:0/2266590235 conn(0x7efd64078360 msgr2=0x7efd640ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 shutdown_connections 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:10.762+0000 7efd6b457700 1 -- 172.21.15.73:0/2266590235 wait complete. 2024-06-06T07:00:10.764 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:11.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:10 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2266590235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:11.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:10 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2266590235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:10 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2266590235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:11.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:00:11.741 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:12.072 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:11 smithi073 ceph-mon[32303]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:12.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:11 smithi170 ceph-mon[36108]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:12.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:11 smithi136 ceph-mon[35901]: pgmap v400: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 -- 172.21.15.73:0/2024362277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20041084f0 msgr2=0x7f20041088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 --2- 172.21.15.73:0/2024362277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20041084f0 0x7f20041088d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1ff4009990 tx=0x7f1ff4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 -- 172.21.15.73:0/2024362277 shutdown_connections 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 --2- 172.21.15.73:0/2024362277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2004102f10 0x7f200410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 --2- 172.21.15.73:0/2024362277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20041084f0 0x7f20041088d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 --2- 172.21.15.73:0/2024362277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 0x7f20041029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 -- 172.21.15.73:0/2024362277 >> 172.21.15.73:0/2024362277 conn(0x7f2004079650 msgr2=0x7f2004079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 -- 172.21.15.73:0/2024362277 shutdown_connections 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 -- 172.21.15.73:0/2024362277 wait complete. 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.415+0000 7f2009262700 1 Processor -- start 2024-06-06T07:00:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 -- start start 2024-06-06T07:00:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 0x7f200419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20041084f0 0x7f2004198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2004112620 con 0x7f2004102f10 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20041124a0 con 0x7f2004102550 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2009262700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20041127a0 con 0x7f20041084f0 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f20027fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f20027fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44708/0 (socket says 172.21.15.73:44708) 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f20027fc700 1 -- 172.21.15.73:0/4238234209 learned_addr learned my addr 172.21.15.73:0/4238234209 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f2002ffd700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 0x7f200419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.416+0000 7f20037fe700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20041084f0 0x7f2004198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 -- 172.21.15.73:0/4238234209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20041084f0 msgr2=0x7f2004198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20041084f0 0x7f2004198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 -- 172.21.15.73:0/4238234209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 msgr2=0x7f200419e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 0x7f200419e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 -- 172.21.15.73:0/4238234209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20041990a0 con 0x7f2004102f10 2024-06-06T07:00:13.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f20027fc700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d950 tx=0x7f1ff800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff8016070 con 0x7f2004102f10 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2004199390 con 0x7f2004102f10 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ff800fd40 con 0x7f2004102f10 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800e3b0 con 0x7f2004102f10 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.417+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20041a5650 con 0x7f2004102f10 2024-06-06T07:00:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.418+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2004103ba0 con 0x7f2004102f10 2024-06-06T07:00:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.419+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ff8019070 con 0x7f2004102f10 2024-06-06T07:00:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.420+0000 7f1febfff700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 0x7f1fec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.420+0000 7f2002ffd700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 0x7f1fec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.420+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f1ff80cee70 con 0x7f2004102f10 2024-06-06T07:00:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.420+0000 7f2002ffd700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 0x7f1fec0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2004199a80 tx=0x7f1ff4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:13.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.423+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1ff809ba00 con 0x7f2004102f10 2024-06-06T07:00:13.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.620+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f200402dca0 con 0x7f2004102f10 2024-06-06T07:00:13.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.622+0000 7f1febfff700 1 -- 172.21.15.73:0/4238234209 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24816 (secure 0 0 0) 0x7f1ff809b300 con 0x7f2004102f10 2024-06-06T07:00:13.623 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:13.623 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:58:53.383839+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 msgr2=0x7f1fec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 0x7f1fec0a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2004199a80 tx=0x7f1ff4005960 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 msgr2=0x7f200419ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1ff800d950 tx=0x7f1ff800dc60 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 shutdown_connections 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f1fec0a2c20 0x7f1fec0a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f20041084f0 0x7f2004198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2004102550 0x7f200419e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 --2- 172.21.15.73:0/4238234209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2004102f10 0x7f200419ecc0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:13.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 >> 172.21.15.73:0/4238234209 conn(0x7f2004079650 msgr2=0x7f2004100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:13.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 shutdown_connections 2024-06-06T07:00:13.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:13.624+0000 7f2009262700 1 -- 172.21.15.73:0/4238234209 wait complete. 2024-06-06T07:00:14.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:00:14.580 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:14.702 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:14 smithi073 ceph-mon[32303]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:14.702 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:14 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/4238234209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:14.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:14 smithi136 ceph-mon[35901]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:14.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:14 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/4238234209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:14.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:14 smithi170 ceph-mon[36108]: pgmap v401: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:14.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:14 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/4238234209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.184+0000 7f32b585f700 1 -- 172.21.15.73:0/1334439952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 msgr2=0x7f32b0107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.184+0000 7f32b585f700 1 --2- 172.21.15.73:0/1334439952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0107830 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f32a4009990 tx=0x7f32a4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.184+0000 7f32b585f700 1 -- 172.21.15.73:0/1334439952 shutdown_connections 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.184+0000 7f32b585f700 1 --2- 172.21.15.73:0/1334439952 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102ee0 0x7f32b010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 --2- 172.21.15.73:0/1334439952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0107830 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 --2- 172.21.15.73:0/1334439952 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102540 0x7f32b01029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 -- 172.21.15.73:0/1334439952 >> 172.21.15.73:0/1334439952 conn(0x7f32b00795f0 msgr2=0x7f32b0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 -- 172.21.15.73:0/1334439952 shutdown_connections 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 -- 172.21.15.73:0/1334439952 wait complete. 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.185+0000 7f32b585f700 1 Processor -- start 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 -- start start 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102540 0x7f32b019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 0x7f32b019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01127d0 con 0x7f32b0107450 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0112650 con 0x7f32b0102540 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32b585f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0112950 con 0x7f32b0102ee0 2024-06-06T07:00:16.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43786/0 (socket says 172.21.15.73:43786) 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 -- 172.21.15.73:0/269035369 learned_addr learned my addr 172.21.15.73:0/269035369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32aeffd700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102540 0x7f32b019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32ae7fc700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 0x7f32b019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 -- 172.21.15.73:0/269035369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 msgr2=0x7f32b019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 0x7f32b019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 -- 172.21.15.73:0/269035369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102540 msgr2=0x7f32b019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102540 0x7f32b019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32af7fe700 1 -- 172.21.15.73:0/269035369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b01991f0 con 0x7f32b0107450 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.186+0000 7f32ae7fc700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 0x7f32b019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.187+0000 7f32af7fe700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b9e0 tx=0x7f32a800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:16.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.187+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800d8e0 con 0x7f32b0107450 2024-06-06T07:00:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.187+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b01994e0 con 0x7f32b0107450 2024-06-06T07:00:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.187+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32a8015420 con 0x7f32b0107450 2024-06-06T07:00:16.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.187+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b01a57a0 con 0x7f32b0107450 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.188+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8023060 con 0x7f32b0107450 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.188+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f329c005330 con 0x7f32b0107450 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.189+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32a80155d0 con 0x7f32b0107450 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.190+0000 7f32b485d700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 0x7f32980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.190+0000 7f32aeffd700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 0x7f32980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.190+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f32a80cec20 con 0x7f32b0107450 2024-06-06T07:00:16.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.191+0000 7f32aeffd700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 0x7f32980a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32b0199bd0 tx=0x7f32a400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:16.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.195+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f32a809b7b0 con 0x7f32b0107450 2024-06-06T07:00:16.449 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:16 smithi073 ceph-mon[32303]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.447+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f329c005620 con 0x7f32b0107450 2024-06-06T07:00:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.448+0000 7f32b485d700 1 -- 172.21.15.73:0/269035369 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7f32a809b0b0 con 0x7f32b0107450 2024-06-06T07:00:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:16.450 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 msgr2=0x7f32980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 0x7f32980a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f32b0199bd0 tx=0x7f32a400b540 comp rx=0 tx=0).stop 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 msgr2=0x7f32b0198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32a800b9e0 tx=0x7f32a800bcf0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 shutdown_connections 2024-06-06T07:00:16.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b0107450 0x7f32b0198a90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f32b0102540 0x7f32b019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32980a2c20 0x7f32980a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 --2- 172.21.15.73:0/269035369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f32b0102ee0 0x7f32b019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 >> 172.21.15.73:0/269035369 conn(0x7f32b00795f0 msgr2=0x7f32b0100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 shutdown_connections 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:16.450+0000 7f32b585f700 1 -- 172.21.15.73:0/269035369 wait complete. 2024-06-06T07:00:16.453 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:16.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:16 smithi136 ceph-mon[35901]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:16.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:16 smithi170 ceph-mon[36108]: pgmap v402: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:17.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T06:57:09.432568+0000', 'modified': '2024-06-06T06:58:21.359605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24433, 'mds_1': 24421, 'mds_2': 24419, 'mds_3': 24415, 'mds_4': 24404}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14529': {'gid': 14529, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6837/38418017', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 38418017}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 38418017}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6839/2070842628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2070842628}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2070842628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6841/552212635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6840', 'nonce': 552212635}, {'type': 'v1', 'addr': '172.21.15.73:6841', 'nonce': 552212635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24365': {'gid': 24365, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/464039391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 464039391}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 464039391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24380': {'gid': 24380, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/687158910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 687158910}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 687158910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.136:6837/450719145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 450719145}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 450719145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.170:6835/3477948531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 3477948531}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 3477948531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.136:6839/2417852679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2417852679}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2417852679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.170:6837/1210322990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 1210322990}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1210322990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.170:6839/3230829204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3230829204}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3230829204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24433, 'qdb_cluster': [24404, 24415, 24421, 24433, 24419]} 2024-06-06T07:00:17.137 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-06T07:00:17.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T07:00:17.383 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:17.407 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:17 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/269035369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:17 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/269035369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:17 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/269035369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:18.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:18 smithi073 ceph-mon[32303]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:18 smithi136 ceph-mon[35901]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:18 smithi170 ceph-mon[36108]: pgmap v403: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:19.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 -- 172.21.15.73:0/3007048073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 msgr2=0x7fc9941088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.064 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 --2- 172.21.15.73:0/3007048073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc9941088f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc984009990 tx=0x7fc984009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 -- 172.21.15.73:0/3007048073 shutdown_connections 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 --2- 172.21.15.73:0/3007048073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102ed0 0x7fc99410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 --2- 172.21.15.73:0/3007048073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc9941088f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 --2- 172.21.15.73:0/3007048073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102510 0x7fc994102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 -- 172.21.15.73:0/3007048073 >> 172.21.15.73:0/3007048073 conn(0x7fc994079560 msgr2=0x7fc994079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 -- 172.21.15.73:0/3007048073 shutdown_connections 2024-06-06T07:00:19.065 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.063+0000 7fc999026700 1 -- 172.21.15.73:0/3007048073 wait complete. 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.064+0000 7fc999026700 1 Processor -- start 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.064+0000 7fc999026700 1 -- start start 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102510 0x7fc99419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 0x7fc99419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc9941126a0 con 0x7fc994108510 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc994112520 con 0x7fc994102510 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc994112820 con 0x7fc994102ed0 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43804/0 (socket says 172.21.15.73:43804) 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 -- 172.21.15.73:0/1071735165 learned_addr learned my addr 172.21.15.73:0/1071735165 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102510 0x7fc99419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99259c700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 0x7fc99419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 -- 172.21.15.73:0/1071735165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 msgr2=0x7fc99419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 0x7fc99419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 -- 172.21.15.73:0/1071735165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102510 msgr2=0x7fc99419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102510 0x7fc99419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 -- 172.21.15.73:0/1071735165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9941991a0 con 0x7fc994108510 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99259c700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 0x7fc99419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc99359e700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ada0 tx=0x7fc98c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc97bfff700 1 -- 172.21.15.73:0/1071735165 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c017020 con 0x7fc994108510 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc994199490 con 0x7fc994108510 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc97bfff700 1 -- 172.21.15.73:0/1071735165 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc98c00ce30 con 0x7fc994108510 2024-06-06T07:00:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.065+0000 7fc97bfff700 1 -- 172.21.15.73:0/1071735165 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98c005350 con 0x7fc994108510 2024-06-06T07:00:19.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.066+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9941a5750 con 0x7fc994108510 2024-06-06T07:00:19.069 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.068+0000 7fc97bfff700 1 -- 172.21.15.73:0/1071735165 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc98c00c980 con 0x7fc994108510 2024-06-06T07:00:19.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.068+0000 7fc97bfff700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 0x7fc97c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.068+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 0x7fc97c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.068+0000 7fc97bfff700 1 -- 172.21.15.73:0/1071735165 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc98c0ceb60 con 0x7fc994108510 2024-06-06T07:00:19.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 0x7fc97c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b5c0 tx=0x7fc984005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 Processor -- start 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 -- start start 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719a8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719a8750 0x55a0719a8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a07189af10 con 0x55a0719adf10 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0718f9800 con 0x55a0719ae830 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.069+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a071861410 con 0x55a0719a8750 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43808/0 (socket says 172.21.15.73:43808) 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 -- 172.21.15.73:0/2581844060 learned_addr learned my addr 172.21.15.73:0/2581844060 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc99359e700 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719a8750 0x55a0719a8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc99259c700 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 -- 172.21.15.73:0/2581844060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719a8750 msgr2=0x55a0719a8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719a8750 0x55a0719a8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 -- 172.21.15.73:0/2581844060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 msgr2=0x55a0719a8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719a8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 -- 172.21.15.73:0/2581844060 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a071848880 con 0x55a0719adf10 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc984015000 tx=0x7fc984020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7e4d48ea0f60fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc9797fa700 1 -- 172.21.15.73:0/2581844060 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98400f870 con 0x55a0719adf10 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc9797fa700 1 -- 172.21.15.73:0/2581844060 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc984024410 con 0x55a0719adf10 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.070+0000 7fc9797fa700 1 -- 172.21.15.73:0/2581844060 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98402f660 con 0x55a0719adf10 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2581844060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 msgr2=0x55a0719ae2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fc984015000 tx=0x7fc984020000 comp rx=0 tx=0).stop 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2581844060 shutdown_connections 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719a8750 0x55a0719a8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719ae2f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2581844060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2581844060 >> 172.21.15.73:0/2581844060 conn(0x55a0718d5030 msgr2=0x55a0719a31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2581844060 shutdown_connections 2024-06-06T07:00:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2581844060 wait complete. 2024-06-06T07:00:19.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.071+0000 7fc9a38a9b80 1 -- start start 2024-06-06T07:00:19.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 Processor -- start 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 0x55a0719b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a071899f70 con 0x55a0719adf10 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0718f57c0 con 0x55a0719a8750 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a0719b5150 con 0x55a0719ae830 2024-06-06T07:00:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc9a38a9b80 10 client.0 _ll_register_callbacks cb 0x55a07188f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43816/0 (socket says 172.21.15.73:43816) 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 -- 172.21.15.73:0/2652482018 learned_addr learned my addr 172.21.15.73:0/2652482018 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99359e700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 0x55a0719b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 -- 172.21.15.73:0/2652482018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 msgr2=0x55a0719b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719b1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 -- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 msgr2=0x55a0719b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 0x55a0719b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 -- 172.21.15.73:0/2652482018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0718bf700 con 0x55a0719adf10 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 0x55a0719b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:19.079 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc99259c700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9880104f0 tx=0x7fc988010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78f90a9668ad08 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.072+0000 7fc976ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc988018070 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc988014640 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc984024410 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc988024050 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 10 client.15030 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0719bf960 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc984024410 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9880138e0 con 0x55a0719adf10 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 10 client.15030 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 20 client.15030 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:19.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a0719c1fd0 0x55a0719c3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 4 client.15030 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.073+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a071906d00 con 0x55a0719c1fd0 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.074+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a0719c1fd0 0x55a0719c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.074+0000 7fc976ff5700 10 client.15030 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.074+0000 7fc992d9d700 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a0719c1fd0 0x55a0719c3440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80feefa1be53390b server_cookie=7b9df1c58e61bab in_seq=0 out_seq=0 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc976ff5700 1 -- 172.21.15.73:0/2652482018 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a071906d00 con 0x55a0719c1fd0 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc976ff5700 10 client.15030 handle_command_reply: tid=0 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 client.15030 shutdown 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 msgr2=0x55a0719b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc9880104f0 tx=0x7fc988010800 comp rx=0 tx=0).stop 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 shutdown_connections 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a0719c1fd0 0x55a0719c3440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719a8750 0x55a0719b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/2652482018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719adf10 0x55a0719b1880 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 >> 172.21.15.73:0/2652482018 conn(0x55a0718d5030 msgr2=0x55a0719a26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc976ff5700 0 client.15030 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:00:19.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 shutdown_connections 2024-06-06T07:00:19.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/2652482018 wait complete. 2024-06-06T07:00:19.082 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.075+0000 7fc9a38a9b80 20 client.15030 trim_cache size 0 max 16384 2024-06-06T07:00:19.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.096+0000 7fc9a38a9b80 1 Processor -- start 2024-06-06T07:00:19.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.096+0000 7fc9a38a9b80 1 -- start start 2024-06-06T07:00:19.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.098 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719adf10 0x55a0719a3370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c1780 con 0x55a0719a8750 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c1e50 con 0x55a0719ae830 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c2520 con 0x55a0719adf10 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43818/0 (socket says 172.21.15.73:43818) 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 -- 172.21.15.73:0/1155742785 learned_addr learned my addr 172.21.15.73:0/1155742785 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc99259c700 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719adf10 0x55a0719a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 -- 172.21.15.73:0/1155742785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719adf10 msgr2=0x55a0719a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719adf10 0x55a0719a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 -- 172.21.15.73:0/1155742785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 msgr2=0x55a0719aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 -- 172.21.15.73:0/1155742785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0719afdf0 con 0x55a0719a8750 2024-06-06T07:00:19.099 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc992d9d700 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9840098e0 tx=0x7fc984020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5789ade141c80813 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.097+0000 7fc978ff9700 1 -- 172.21.15.73:0/1155742785 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984024740 con 0x55a0719a8750 2024-06-06T07:00:19.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc978ff9700 1 -- 172.21.15.73:0/1155742785 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc984041070 con 0x55a0719a8750 2024-06-06T07:00:19.100 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc978ff9700 1 -- 172.21.15.73:0/1155742785 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc984038440 con 0x55a0719a8750 2024-06-06T07:00:19.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/1155742785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 msgr2=0x55a0719a2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc9840098e0 tx=0x7fc984020000 comp rx=0 tx=0).stop 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/1155742785 shutdown_connections 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719ae830 0x55a0719aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719a2e30 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/1155742785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719adf10 0x55a0719a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/1155742785 >> 172.21.15.73:0/1155742785 conn(0x55a0718d5030 msgr2=0x55a0719b0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/1155742785 shutdown_connections 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.098+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/1155742785 wait complete. 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 -- start start 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 Processor -- start 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 0x55a0719a2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719a2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c32c0 con 0x55a0719a8750 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c2bf0 con 0x55a0719adf10 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a0719c3990 con 0x55a0719ae830 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc9a38a9b80 10 client.0 _ll_register_callbacks cb 0x55a07188f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43824/0 (socket says 172.21.15.73:43824) 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 -- 172.21.15.73:0/850026160 learned_addr learned my addr 172.21.15.73:0/850026160 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc99259c700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 0x55a0719a2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 -- 172.21.15.73:0/850026160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 msgr2=0x55a0719a2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719a2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 -- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 msgr2=0x55a0719a2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:19.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 0x55a0719a2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc992d9d700 1 -- 172.21.15.73:0/850026160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0719cac90 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.099+0000 7fc99259c700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 0x55a0719a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc992d9d700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc98400bb70 tx=0x7fc9840244c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3310667e276dda7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc98404a060 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc984024ac0 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc984041070 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc98400f740 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 10 client.15036 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a071a2da40 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc984041070 con 0x55a0719a8750 2024-06-06T07:00:19.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9840389a0 con 0x55a0719a8750 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 10 client.15036 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 20 client.15036 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a071a31fb0 0x55a071a30cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 4 client.15036 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.100+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "status"}) -- 0x55a071901280 con 0x55a071a31fb0 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc99259c700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a071a31fb0 0x55a071a30cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc976ff5700 10 client.15036 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc99259c700 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a071a31fb0 0x55a071a30cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fdaafd92d3335bd4 server_cookie=e1ab42a2de9e2416 in_seq=0 out_seq=0 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc976ff5700 1 -- 172.21.15.73:0/850026160 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55a071901280 con 0x55a071a31fb0 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc976ff5700 10 client.15036 handle_command_reply: tid=0 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 client.15036 shutdown 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 msgr2=0x55a0719cd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.107 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc98400bb70 tx=0x7fc9840244c0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 shutdown_connections 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a071a31fb0 0x55a071a30cc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0719ae830 0x55a0719a2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0719a8750 0x55a0719cd1a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 --2- 172.21.15.73:0/850026160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0719adf10 0x55a0719a2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 >> 172.21.15.73:0/850026160 conn(0x55a0718d5030 msgr2=0x55a0719aec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.101+0000 7fc976ff5700 0 client.15036 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 shutdown_connections 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc9a38a9b80 1 -- 172.21.15.73:0/850026160 wait complete. 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc9a38a9b80 20 client.15036 trim_cache size 0 max 16384 2024-06-06T07:00:19.108 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 4, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24404, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 120.691508599, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 218.17499431300001 2024-06-06T07:00:19.109 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 msgr2=0x7fc97c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 0x7fc97c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc98400b5c0 tx=0x7fc984005960 comp rx=0 tx=0).stop 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 msgr2=0x7fc994198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00ada0 tx=0x7fc98c00c330 comp rx=0 tx=0).stop 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 shutdown_connections 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc994108510 0x7fc994198a40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc994102510 0x7fc99419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc97c0a2c20 0x7fc97c0a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 --2- 172.21.15.73:0/1071735165 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc994102ed0 0x7fc99419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:19.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.102+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 >> 172.21.15.73:0/1071735165 conn(0x7fc994079560 msgr2=0x7fc9941011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:19.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.103+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 shutdown_connections 2024-06-06T07:00:19.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:19.103+0000 7fc999026700 1 -- 172.21.15.73:0/1071735165 wait complete. 2024-06-06T07:00:19.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-06T07:00:20.046 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:20.418 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:20 smithi073 ceph-mon[32303]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:20 smithi136 ceph-mon[35901]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:20.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:20 smithi170 ceph-mon[36108]: pgmap v404: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 -- 172.21.15.73:0/1498513462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9641030f0 msgr2=0x7fd964107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/1498513462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9641030f0 0x7fd964107110 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fd9540099c0 tx=0x7fd954009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 -- 172.21.15.73:0/1498513462 shutdown_connections 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/1498513462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964102740 0x7fd964102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/1498513462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd964107840 0x7fd96410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/1498513462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9641030f0 0x7fd964107110 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 -- 172.21.15.73:0/1498513462 >> 172.21.15.73:0/1498513462 conn(0x7fd9640795f0 msgr2=0x7fd964079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 -- 172.21.15.73:0/1498513462 shutdown_connections 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.749+0000 7fd96bbf5700 1 -- 172.21.15.73:0/1498513462 wait complete. 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 Processor -- start 2024-06-06T07:00:21.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 -- start start 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9641030f0 0x7fd96419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964107840 0x7fd9641988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd964112620 con 0x7fd964102740 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641124a0 con 0x7fd9641030f0 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96bbf5700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9641127a0 con 0x7fd964107840 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43840/0 (socket says 172.21.15.73:43840) 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 -- 172.21.15.73:0/3199918301 learned_addr learned my addr 172.21.15.73:0/3199918301 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969190700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9641030f0 0x7fd96419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd96a192700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964107840 0x7fd9641988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 -- 172.21.15.73:0/3199918301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964107840 msgr2=0x7fd9641988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964107840 0x7fd9641988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 -- 172.21.15.73:0/3199918301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9641030f0 msgr2=0x7fd96419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9641030f0 0x7fd96419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.750+0000 7fd969991700 1 -- 172.21.15.73:0/3199918301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd964199020 con 0x7fd964102740 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd969991700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d980 tx=0x7fd96000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd95affd700 1 -- 172.21.15.73:0/3199918301 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960017070 con 0x7fd964102740 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd964199310 con 0x7fd964102740 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd95affd700 1 -- 172.21.15.73:0/3199918301 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd960004830 con 0x7fd964102740 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd95affd700 1 -- 172.21.15.73:0/3199918301 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd960005440 con 0x7fd964102740 2024-06-06T07:00:21.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.751+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9641a5580 con 0x7fd964102740 2024-06-06T07:00:21.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.753+0000 7fd95affd700 1 -- 172.21.15.73:0/3199918301 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd9600055e0 con 0x7fd964102740 2024-06-06T07:00:21.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.754+0000 7fd95affd700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 0x7fd9500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.754+0000 7fd969190700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 0x7fd9500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.754+0000 7fd95affd700 1 -- 172.21.15.73:0/3199918301 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd9600cf370 con 0x7fd964102740 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.754+0000 7fd969190700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 0x7fd9500a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd964102060 tx=0x7fd954005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.754+0000 7fd976478b80 1 Processor -- start 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 -- start start 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fca9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fc4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577e8f167c0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577e8ebe6d0 con 0x5577e8fca5f0 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd976478b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577e8f12780 con 0x5577e8fcaf10 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43842/0 (socket says 172.21.15.73:43842) 2024-06-06T07:00:21.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 -- 172.21.15.73:0/3515743466 learned_addr learned my addr 172.21.15.73:0/3515743466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd969190700 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fc4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd969991700 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fca9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 -- 172.21.15.73:0/3515743466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 msgr2=0x5577e8fc4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fc4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 -- 172.21.15.73:0/3515743466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 msgr2=0x5577e8fca9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fca9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 -- 172.21.15.73:0/3515743466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e8eb4f00 con 0x5577e8fc4ed0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.755+0000 7fd96a192700 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e190 tx=0x7fd95c00e4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b731345d220782fb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd94ffff700 1 -- 172.21.15.73:0/3515743466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00ed30 con 0x5577e8fc4ed0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd94ffff700 1 -- 172.21.15.73:0/3515743466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd95c004bb0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd94ffff700 1 -- 172.21.15.73:0/3515743466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c014dd0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd976478b80 1 -- 172.21.15.73:0/3515743466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 msgr2=0x5577e8fc52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd976478b80 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e190 tx=0x7fd95c00e4a0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd976478b80 1 -- 172.21.15.73:0/3515743466 shutdown_connections 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.756+0000 7fd976478b80 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc52f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fca9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 --2- 172.21.15.73:0/3515743466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fc4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 -- 172.21.15.73:0/3515743466 >> 172.21.15.73:0/3515743466 conn(0x5577e8eefc70 msgr2=0x5577e8fbf800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 -- 172.21.15.73:0/3515743466 shutdown_connections 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 -- 172.21.15.73:0/3515743466 wait complete. 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 -- start start 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.757+0000 7fd976478b80 1 Processor -- start 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fcdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fd1980 con 0x5577e8fc4ed0 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577e8eb6ba0 con 0x5577e8fca5f0 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fd1b00 con 0x5577e8fcaf10 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 10 client.0 _ll_register_callbacks cb 0x5577e8eb08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43858/0 (socket says 172.21.15.73:43858) 2024-06-06T07:00:21.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 -- 172.21.15.73:0/2365096817 learned_addr learned my addr 172.21.15.73:0/2365096817 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd96a192700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969190700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fcdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 -- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 msgr2=0x5577e8fce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 -- 172.21.15.73:0/2365096817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 msgr2=0x5577e8fcdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fcdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 -- 172.21.15.73:0/2365096817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e8e65880 con 0x5577e8fc4ed0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd969991700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd96009b500 tx=0x7fd96000b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e58548830ca5bc6f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd94d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd96009fbb0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd9600049e0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.758+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd95c004bb0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9600a4470 con 0x5577e8fc4ed0 2024-06-06T07:00:21.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 10 client.15048 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577e8fdc330 con 0x5577e8fc4ed0 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd95c004bb0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd96009ba60 con 0x5577e8fc4ed0 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 10 client.15048 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 20 client.15048 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e8fde940 0x5577e8fdfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 4 client.15048 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.759+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577e8f23cc0 con 0x5577e8fde940 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.760+0000 7fd969190700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e8fde940 0x5577e8fdfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.760+0000 7fd969190700 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e8fde940 0x5577e8fdfdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=527ad2113100c71d server_cookie=dbfd7f664f7bb946 in_seq=0 out_seq=0 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.760+0000 7fd94d7fa700 10 client.15048 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd94d7fa700 1 -- 172.21.15.73:0/2365096817 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5577e8f23cc0 con 0x5577e8fde940 2024-06-06T07:00:21.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd94d7fa700 10 client.15048 handle_command_reply: tid=0 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 client.15048 shutdown 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 msgr2=0x5577e8fd0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fd96009b500 tx=0x7fd96000b510 comp rx=0 tx=0).stop 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 shutdown_connections 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e8fde940 0x5577e8fdfdb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fcaf10 0x5577e8fce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fc4ed0 0x5577e8fd0ce0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd94d7fa700 0 client.15048 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 --2- 172.21.15.73:0/2365096817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fcdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 >> 172.21.15.73:0/2365096817 conn(0x5577e8eefc70 msgr2=0x5577e8fc3980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 shutdown_connections 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 1 -- 172.21.15.73:0/2365096817 wait complete. 2024-06-06T07:00:21.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.761+0000 7fd976478b80 20 client.15048 trim_cache size 0 max 16384 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 Processor -- start 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 -- start start 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fca5f0 0x5577e8fb94c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fde0f0 con 0x5577e8fcaf10 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fde7c0 con 0x5577e8fc4ed0 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd976478b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fdee90 con 0x5577e8fca5f0 2024-06-06T07:00:21.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43874/0 (socket says 172.21.15.73:43874) 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 -- 172.21.15.73:0/1880507016 learned_addr learned my addr 172.21.15.73:0/1880507016 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd969190700 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fca5f0 0x5577e8fb94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 -- 172.21.15.73:0/1880507016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fca5f0 msgr2=0x5577e8fb94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fca5f0 0x5577e8fb94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 -- 172.21.15.73:0/1880507016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fc4ed0 msgr2=0x5577e8fc4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.783+0000 7fd96a192700 1 -- 172.21.15.73:0/1880507016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577e8fc04b0 con 0x5577e8fcaf10 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd96a192700 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd95c000f80 tx=0x7fd95c007690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5029e9c49ef07aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd94f7fe700 1 -- 172.21.15.73:0/1880507016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c00ed30 con 0x5577e8fcaf10 2024-06-06T07:00:21.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd94f7fe700 1 -- 172.21.15.73:0/1880507016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd95c003a00 con 0x5577e8fcaf10 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd94f7fe700 1 -- 172.21.15.73:0/1880507016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c004320 con 0x5577e8fcaf10 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd976478b80 1 -- 172.21.15.73:0/1880507016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 msgr2=0x5577e8fcb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.784+0000 7fd976478b80 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd95c000f80 tx=0x7fd95c007690 comp rx=0 tx=0).stop 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- 172.21.15.73:0/1880507016 shutdown_connections 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fcb2f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fc4ed0 0x5577e8fc4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- 172.21.15.73:0/1880507016 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fca5f0 0x5577e8fb94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- 172.21.15.73:0/1880507016 >> 172.21.15.73:0/1880507016 conn(0x5577e8eefc70 msgr2=0x5577e8fb8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- 172.21.15.73:0/1880507016 shutdown_connections 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- 172.21.15.73:0/1880507016 wait complete. 2024-06-06T07:00:21.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- start start 2024-06-06T07:00:21.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 Processor -- start 2024-06-06T07:00:21.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fc4ed0 0x5577e8fe9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fe4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fdfc30 con 0x5577e8fcaf10 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fdf560 con 0x5577e8fca5f0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5577e8fe0300 con 0x5577e8fc4ed0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.785+0000 7fd976478b80 10 client.0 _ll_register_callbacks cb 0x5577e8eb08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43886/0 (socket says 172.21.15.73:43886) 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 -- 172.21.15.73:0/961824729 learned_addr learned my addr 172.21.15.73:0/961824729 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd969991700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fc4ed0 0x5577e8fe9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd969190700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fe4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 -- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fc4ed0 msgr2=0x5577e8fe9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fc4ed0 0x5577e8fe9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 -- 172.21.15.73:0/961824729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 msgr2=0x5577e8fe4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:21.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fe4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 -- 172.21.15.73:0/961824729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577e8fe5ab0 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd96a192700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e4e0 tx=0x7fd95c003e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f05561ab2aed54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd94d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd95c02e070 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd95c010040 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd95c003a00 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd95c005440 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd976478b80 10 client.15054 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.786+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577e904a260 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd95c003a00 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd95c014b90 con 0x5577e8fcaf10 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:21.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 10 client.15054 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 20 client.15054 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e904e730 0x5577e904d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 4 client.15054 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "status"}) -- 0x5577e8f1e240 con 0x5577e904e730 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.787+0000 7fd969991700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e904e730 0x5577e904d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd969991700 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e904e730 0x5577e904d440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3595a0870e9d9a6 server_cookie=978f892a42b0cccb in_seq=0 out_seq=0 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd94d7fa700 10 client.15054 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd94d7fa700 1 -- 172.21.15.73:0/961824729 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5577e8f1e240 con 0x5577e904e730 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd94d7fa700 10 client.15054 handle_command_reply: tid=0 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 client.15054 shutdown 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 msgr2=0x5577e8fe5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd95c00e4e0 tx=0x7fd95c003e50 comp rx=0 tx=0).stop 2024-06-06T07:00:21.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 shutdown_connections 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5577e904e730 0x5577e904d440 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5577e8fc4ed0 0x5577e8fe9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5577e8fcaf10 0x5577e8fe5270 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd94d7fa700 0 client.15054 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 --2- 172.21.15.73:0/961824729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5577e8fca5f0 0x5577e8fe4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 >> 172.21.15.73:0/961824729 conn(0x5577e8eefc70 msgr2=0x5577e8fcb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 shutdown_connections 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 1 -- 172.21.15.73:0/961824729 wait complete. 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.788+0000 7fd976478b80 20 client.15054 trim_cache size 0 max 16384 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:00:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 3, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24415, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 124.383895721, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 226.412737163 2024-06-06T07:00:21.796 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 msgr2=0x7fd9500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 0x7fd9500a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd964102060 tx=0x7fd954005fd0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 msgr2=0x7fd96419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fd96000d980 tx=0x7fd96000dd40 comp rx=0 tx=0).stop 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 shutdown_connections 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd9500a2c20 0x7fd9500a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd964107840 0x7fd9641988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd964102740 0x7fd96419e6b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 --2- 172.21.15.73:0/3199918301 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9641030f0 0x7fd96419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:21.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 >> 172.21.15.73:0/3199918301 conn(0x7fd9640795f0 msgr2=0x7fd9641007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:21.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 shutdown_connections 2024-06-06T07:00:21.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:21.789+0000 7fd96bbf5700 1 -- 172.21.15.73:0/3199918301 wait complete. 2024-06-06T07:00:22.467 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:22 smithi073 ceph-mon[32303]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:22.494 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:00:22.733 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:22 smithi136 ceph-mon[35901]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:22.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:22 smithi170 ceph-mon[36108]: pgmap v405: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:24.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 -- 172.21.15.73:0/1115556456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806ab30 msgr2=0x7fd58810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 --2- 172.21.15.73:0/1115556456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806ab30 0x7fd58810f320 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd57800b0a0 tx=0x7fd57800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 -- 172.21.15.73:0/1115556456 shutdown_connections 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 --2- 172.21.15.73:0/1115556456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806ab30 0x7fd58810f320 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 --2- 172.21.15.73:0/1115556456 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 0x7fd5881088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 --2- 172.21.15.73:0/1115556456 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806a170 0x7fd58806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 -- 172.21.15.73:0/1115556456 >> 172.21.15.73:0/1115556456 conn(0x7fd588078340 msgr2=0x7fd588078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 -- 172.21.15.73:0/1115556456 shutdown_connections 2024-06-06T07:00:24.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 -- 172.21.15.73:0/1115556456 wait complete. 2024-06-06T07:00:24.445 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:24 smithi073 ceph-mon[32303]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.443+0000 7fd58efe3700 1 Processor -- start 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 -- start start 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 0x7fd5881019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 0x7fd588101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.447 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd588112630 con 0x7fd58806a170 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881124b0 con 0x7fd588108500 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58efe3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5881127b0 con 0x7fd58806ab30 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43908/0 (socket says 172.21.15.73:43908) 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 -- 172.21.15.73:0/3831443472 learned_addr learned my addr 172.21.15.73:0/3831443472 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58d580700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 0x7fd588101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd587fff700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 0x7fd5881019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 -- 172.21.15.73:0/3831443472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 msgr2=0x7fd5881019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 0x7fd5881019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 -- 172.21.15.73:0/3831443472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 msgr2=0x7fd588101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 0x7fd588101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 -- 172.21.15.73:0/3831443472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5881a7030 con 0x7fd58806a170 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd587fff700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 0x7fd5881019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.444+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd58000d950 tx=0x7fd58000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.445+0000 7fd585ffb700 1 -- 172.21.15.73:0/3831443472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd580016070 con 0x7fd58806a170 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.445+0000 7fd585ffb700 1 -- 172.21.15.73:0/3831443472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd580009df0 con 0x7fd58806a170 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.445+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5881a7320 con 0x7fd58806a170 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.445+0000 7fd585ffb700 1 -- 172.21.15.73:0/3831443472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd58000f3c0 con 0x7fd58806a170 2024-06-06T07:00:24.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.445+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5881a7860 con 0x7fd58806a170 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.447+0000 7fd585ffb700 1 -- 172.21.15.73:0/3831443472 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd580019070 con 0x7fd58806a170 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.447+0000 7fd585ffb700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 0x7fd5740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.448+0000 7fd587fff700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 0x7fd5740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.448+0000 7fd585ffb700 1 -- 172.21.15.73:0/3831443472 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd5800cef90 con 0x7fd58806a170 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.448+0000 7fd587fff700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 0x7fd5740a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5880789b0 tx=0x7fd570009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 Processor -- start 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 -- start start 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbcf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56169bb1b7c0 con 0x56169bbcff10 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56169bac36d0 con 0x56169bbc9ed0 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd599866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56169bb17780 con 0x56169bbcf5f0 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43910/0 (socket says 172.21.15.73:43910) 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 -- 172.21.15.73:0/3072224490 learned_addr learned my addr 172.21.15.73:0/3072224490 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbcf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 -- 172.21.15.73:0/3072224490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 msgr2=0x56169bbcf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbcf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 -- 172.21.15.73:0/3072224490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 msgr2=0x56169bbca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 -- 172.21.15.73:0/3072224490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169bab9f00 con 0x56169bbcff10 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbcf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.449+0000 7fd587fff700 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd570011f80 tx=0x7fd570014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ff779bc00d6e47 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd57effd700 1 -- 172.21.15.73:0/3072224490 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57001f620 con 0x56169bbcff10 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd57effd700 1 -- 172.21.15.73:0/3072224490 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57001fc70 con 0x56169bbcff10 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd57effd700 1 -- 172.21.15.73:0/3072224490 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57001db40 con 0x56169bbcff10 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 -- 172.21.15.73:0/3072224490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 msgr2=0x56169bbc9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd570011f80 tx=0x7fd570014000 comp rx=0 tx=0).stop 2024-06-06T07:00:24.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 -- 172.21.15.73:0/3072224490 shutdown_connections 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbcf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 --2- 172.21.15.73:0/3072224490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcff10 0x56169bbc9990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 -- 172.21.15.73:0/3072224490 >> 172.21.15.73:0/3072224490 conn(0x56169baf4c70 msgr2=0x56169bbc4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 -- 172.21.15.73:0/3072224490 shutdown_connections 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.450+0000 7fd599866b80 1 -- 172.21.15.73:0/3072224490 wait complete. 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 -- start start 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 Processor -- start 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbd5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbd3290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56169bbd6980 con 0x56169bbd3930 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56169babbba0 con 0x56169bbc9ed0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56169bbd6b00 con 0x56169bbcf5f0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.451+0000 7fd599866b80 10 client.0 _ll_register_callbacks cb 0x56169bab58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43922/0 (socket says 172.21.15.73:43922) 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 -- 172.21.15.73:0/237750052 learned_addr learned my addr 172.21.15.73:0/237750052 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbd5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd587fff700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbd3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 -- 172.21.15.73:0/237750052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 msgr2=0x56169bbd3290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbd3290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 -- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 msgr2=0x56169bbd5f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbd5f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 -- 172.21.15.73:0/237750052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169ba6a880 con 0x56169bbd3930 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbd5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd58d580700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd578015000 tx=0x7fd57800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c72c12c92064d41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd567fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd578003e10 con 0x56169bbd3930 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd5780074e0 con 0x56169bbd3930 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd57001fc70 con 0x56169bbd3930 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd57802f050 con 0x56169bbd3930 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd599866b80 10 client.15066 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:24.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.452+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56169bbe1990 con 0x56169bbd3930 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd57001fc70 con 0x56169bbd3930 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd578011070 con 0x56169bbd3930 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 20 client.15066 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bbe3fa0 0x56169bbe5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 4 client.15066 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56169bb28cc0 con 0x56169bbe3fa0 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.453+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bbe3fa0 0x56169bbe5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.454+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bbe3fa0 0x56169bbe5410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b2d39155748987d6 server_cookie=7d795ec0e093efbd in_seq=0 out_seq=0 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.454+0000 7fd567fff700 10 client.15066 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:00:24.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd567fff700 1 -- 172.21.15.73:0/237750052 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56169bb28cc0 con 0x56169bbe3fa0 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd567fff700 10 client.15066 handle_command_reply: tid=0 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 client.15066 shutdown 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 msgr2=0x56169bbdfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fd578015000 tx=0x7fd57800bce0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 shutdown_connections 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bbe3fa0 0x56169bbe5410 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbd3930 0x56169bbdfe60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbd5f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd567fff700 0 client.15066 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 --2- 172.21.15.73:0/237750052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbcf5f0 0x56169bbd3290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 >> 172.21.15.73:0/237750052 conn(0x56169baf4c70 msgr2=0x56169bbc8980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 shutdown_connections 2024-06-06T07:00:24.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 1 -- 172.21.15.73:0/237750052 wait complete. 2024-06-06T07:00:24.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.455+0000 7fd599866b80 20 client.15066 trim_cache size 0 max 16384 2024-06-06T07:00:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.476+0000 7fd599866b80 1 Processor -- start 2024-06-06T07:00:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.476+0000 7fd599866b80 1 -- start start 2024-06-06T07:00:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbc9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 0x56169bbdfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe3750 con 0x56169bbcf5f0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe3e20 con 0x56169bbc9ed0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd599866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe44f0 con 0x56169bbd3930 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43938/0 (socket says 172.21.15.73:43938) 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 -- 172.21.15.73:0/235608959 learned_addr learned my addr 172.21.15.73:0/235608959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbc9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd58d580700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 0x56169bbdfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 -- 172.21.15.73:0/235608959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 msgr2=0x56169bbdfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 0x56169bbdfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 -- 172.21.15.73:0/235608959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 msgr2=0x56169bbc9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbc9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 -- 172.21.15.73:0/235608959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56169bbcfc20 con 0x56169bbcf5f0 2024-06-06T07:00:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd58d580700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 0x56169bbdfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd587fff700 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56169bbc55c0 tx=0x7fd570005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f952b80368789e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.477+0000 7fd57e7fc700 1 -- 172.21.15.73:0/235608959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5700053c0 con 0x56169bbcf5f0 2024-06-06T07:00:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd57e7fc700 1 -- 172.21.15.73:0/235608959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd570005560 con 0x56169bbcf5f0 2024-06-06T07:00:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd57e7fc700 1 -- 172.21.15.73:0/235608959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57001f800 con 0x56169bbcf5f0 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 -- 172.21.15.73:0/235608959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 msgr2=0x56169bbd3250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x56169bbc55c0 tx=0x7fd570005b00 comp rx=0 tx=0).stop 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 -- 172.21.15.73:0/235608959 shutdown_connections 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbd3930 0x56169bbdfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbc9ed0 0x56169bbc9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 --2- 172.21.15.73:0/235608959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbd3250 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 -- 172.21.15.73:0/235608959 >> 172.21.15.73:0/235608959 conn(0x56169baf4c70 msgr2=0x56169bbd2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 -- 172.21.15.73:0/235608959 shutdown_connections 2024-06-06T07:00:24.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.478+0000 7fd599866b80 1 -- 172.21.15.73:0/235608959 wait complete. 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 -- start start 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 Processor -- start 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbc9ed0 0x56169bbef010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbd3930 0x56169bbdf960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe5290 con 0x56169bbcf5f0 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe4bc0 con 0x56169bbd3930 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56169bbe5960 con 0x56169bbc9ed0 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd599866b80 10 client.0 _ll_register_callbacks cb 0x56169bab58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:24.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd587fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43942/0 (socket says 172.21.15.73:43942) 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd587fff700 1 -- 172.21.15.73:0/3482415430 learned_addr learned my addr 172.21.15.73:0/3482415430 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd58d580700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbd3930 0x56169bbdf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.479+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbc9ed0 0x56169bbef010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 -- 172.21.15.73:0/3482415430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbc9ed0 msgr2=0x56169bbef010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbc9ed0 0x56169bbef010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 -- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbd3930 msgr2=0x56169bbdf960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbd3930 0x56169bbdf960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 -- 172.21.15.73:0/3482415430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56169bbe01c0 con 0x56169bbcf5f0 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd587fff700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd570009850 tx=0x7fd5700053f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee7fb02248344c4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:24.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57003f020 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd57001fb60 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd570005560 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd5700046f0 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd599866b80 10 client.15072 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56169bc4f890 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd570005560 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.480+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd5700329e0 con 0x56169bbcf5f0 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 10 client.15072 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 10 client.15072 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 20 client.15072 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bc53d70 0x56169bc52a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 4 client.15072 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:00:24.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "status"}) -- 0x56169bb23240 con 0x56169bc53d70 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bc53d70 0x56169bc52a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd58cd7f700 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bc53d70 0x56169bc52a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f6e3f41b03d23030 server_cookie=941c41bb45d0562b in_seq=0 out_seq=0 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.481+0000 7fd567fff700 10 client.15072 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd567fff700 1 -- 172.21.15.73:0/3482415430 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x56169bb23240 con 0x56169bc53d70 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd567fff700 10 client.15072 handle_command_reply: tid=0 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 client.15072 shutdown 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 msgr2=0x56169bbdf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd570009850 tx=0x7fd5700053f0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 shutdown_connections 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56169bc53d70 0x56169bc52a80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56169bbc9ed0 0x56169bbef010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56169bbd3930 0x56169bbdf960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd567fff700 0 client.15072 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 --2- 172.21.15.73:0/3482415430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56169bbcf5f0 0x56169bbdf420 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 >> 172.21.15.73:0/3482415430 conn(0x56169baf4c70 msgr2=0x56169bbcd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 shutdown_connections 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 1 -- 172.21.15.73:0/3482415430 wait complete. 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.482+0000 7fd599866b80 20 client.15072 trim_cache size 0 max 16384 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 2, 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24419, 2024-06-06T07:00:24.489 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 128.08385032699999, 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:00:24.490 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 209.77604492699999 2024-06-06T07:00:24.491 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:00:24.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 msgr2=0x7fd5740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 0x7fd5740a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd5880789b0 tx=0x7fd570009260 comp rx=0 tx=0).stop 2024-06-06T07:00:24.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 msgr2=0x7fd588101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:24.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fd58000d950 tx=0x7fd58000dc60 comp rx=0 tx=0).stop 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 shutdown_connections 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd588108500 0x7fd588101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd58806a170 0x7fd588101460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd5740a2c20 0x7fd5740a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 --2- 172.21.15.73:0/3831443472 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd58806ab30 0x7fd5881019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 >> 172.21.15.73:0/3831443472 conn(0x7fd588078340 msgr2=0x7fd588100030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 shutdown_connections 2024-06-06T07:00:24.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:24.483+0000 7fd58efe3700 1 -- 172.21.15.73:0/3831443472 wait complete. 2024-06-06T07:00:24.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:24 smithi136 ceph-mon[35901]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:24.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:24 smithi170 ceph-mon[36108]: pgmap v406: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:25.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-06T07:00:25.466 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:25 smithi073 ceph-mon[32303]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:25 smithi136 ceph-mon[35901]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:25 smithi170 ceph-mon[36108]: pgmap v407: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 -- 172.21.15.73:0/1018949561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806b3c0 msgr2=0x7f712810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1018949561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806b3c0 0x7f712810f310 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f712000b0a0 tx=0x7f712000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 -- 172.21.15.73:0/1018949561 shutdown_connections 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1018949561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71281084f0 0x7f71281088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1018949561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806b3c0 0x7f712810f310 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1018949561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f712806aa00 0x7f712806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 -- 172.21.15.73:0/1018949561 >> 172.21.15.73:0/1018949561 conn(0x7f7128078300 msgr2=0x7f7128078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 -- 172.21.15.73:0/1018949561 shutdown_connections 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.194+0000 7f712d4bc700 1 -- 172.21.15.73:0/1018949561 wait complete. 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.195+0000 7f712d4bc700 1 Processor -- start 2024-06-06T07:00:27.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.195+0000 7f712d4bc700 1 -- start start 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.195+0000 7f712d4bc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806aa00 0x7f7128101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f712d4bc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f712d4bc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71281084f0 0x7f7128101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f712d4bc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7128112620 con 0x7f712806aa00 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f712d4bc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71281124a0 con 0x7f71281084f0 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f712d4bc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71281127a0 con 0x7f712806b3c0 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806aa00 0x7f7128101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56964/0 (socket says 172.21.15.73:56964) 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 -- 172.21.15.73:0/1683282392 learned_addr learned my addr 172.21.15.73:0/1683282392 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71277fe700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71281084f0 0x7f7128101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 -- 172.21.15.73:0/1683282392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71281084f0 msgr2=0x7f7128101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71281084f0 0x7f7128101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 -- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806aa00 msgr2=0x7f7128101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806aa00 0x7f7128101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 -- 172.21.15.73:0/1683282392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a70e0 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.196+0000 7f71267fc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d950 tx=0x7f711c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.197+0000 7f710ffff700 1 -- 172.21.15.73:0/1683282392 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c016070 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.197+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a73d0 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.197+0000 7f710ffff700 1 -- 172.21.15.73:0/1683282392 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f711c00fd40 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.197+0000 7f710ffff700 1 -- 172.21.15.73:0/1683282392 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c00e3b0 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.197+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281a7910 con 0x7f712806b3c0 2024-06-06T07:00:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.199+0000 7f710ffff700 1 -- 172.21.15.73:0/1683282392 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f711c019070 con 0x7f712806b3c0 2024-06-06T07:00:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.200+0000 7f710ffff700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 0x7f71100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.201+0000 7f7126ffd700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 0x7f71100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.201+0000 7f710ffff700 1 -- 172.21.15.73:0/1683282392 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f711c0cee90 con 0x7f712806b3c0 2024-06-06T07:00:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.201+0000 7f7126ffd700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 0x7f71100a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71180045b0 tx=0x7f711800b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.202+0000 7f7137d3fb80 1 Processor -- start 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.202+0000 7f7137d3fb80 1 -- start start 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b06990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559967a587c0 con 0x559967b0c5f0 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559967a006d0 con 0x559967b0cf10 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559967a54780 con 0x559967b06ed0 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58564/0 (socket says 172.21.15.73:58564) 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 -- 172.21.15.73:0/1119934925 learned_addr learned my addr 172.21.15.73:0/1119934925 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f71277fe700 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f71267fc700 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b06990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 -- 172.21.15.73:0/1119934925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 msgr2=0x559967b072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 -- 172.21.15.73:0/1119934925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 msgr2=0x559967b06990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b06990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 -- 172.21.15.73:0/1119934925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599679f6f00 con 0x559967b0c5f0 2024-06-06T07:00:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.203+0000 7f7126ffd700 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7118003b20 tx=0x7f7118013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2459a21bdfdd9708 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f710d7fa700 1 -- 172.21.15.73:0/1119934925 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71180118d0 con 0x559967b0c5f0 2024-06-06T07:00:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f710d7fa700 1 -- 172.21.15.73:0/1119934925 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7118004830 con 0x559967b0c5f0 2024-06-06T07:00:27.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f710d7fa700 1 -- 172.21.15.73:0/1119934925 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118005520 con 0x559967b0c5f0 2024-06-06T07:00:27.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f7137d3fb80 1 -- 172.21.15.73:0/1119934925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 msgr2=0x559967b0c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7118003b20 tx=0x7f7118013000 comp rx=0 tx=0).stop 2024-06-06T07:00:27.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f7137d3fb80 1 -- 172.21.15.73:0/1119934925 shutdown_connections 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.204+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0c9d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/1119934925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b06990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- 172.21.15.73:0/1119934925 >> 172.21.15.73:0/1119934925 conn(0x559967a31c70 msgr2=0x559967b01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- 172.21.15.73:0/1119934925 shutdown_connections 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- 172.21.15.73:0/1119934925 wait complete. 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- start start 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 Processor -- start 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b12ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b10520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559967b13980 con 0x559967b0c5f0 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5599679f8ba0 con 0x559967b0cf10 2024-06-06T07:00:27.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559967b13b00 con 0x559967b06ed0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f7137d3fb80 10 client.0 _ll_register_callbacks cb 0x5599679f28d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.205+0000 7f71267fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58572/0 (socket says 172.21.15.73:58572) 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 -- 172.21.15.73:0/137586627 learned_addr learned my addr 172.21.15.73:0/137586627 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f7126ffd700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b12ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71277fe700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b10520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 -- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 msgr2=0x559967b12ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b12ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 -- 172.21.15.73:0/137586627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 msgr2=0x559967b10520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b10520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 -- 172.21.15.73:0/137586627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599679a7880 con 0x559967b0c5f0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f71267fc700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f711c0cc930 tx=0x7f711c09baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51d3ea74983cbdb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f710aff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c020040 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f711c010710 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7118004830 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711c09f6d0 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f7137d3fb80 10 client.15084 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.206+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559967b1e2e0 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7118004830 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f711c0a3b90 con 0x559967b0c5f0 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 10 client.15084 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 10 client.15084 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 20 client.15084 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b208b0 0x559967b21d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 4 client.15084 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:00:27.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.207+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559967a65cc0 con 0x559967b208b0 2024-06-06T07:00:27.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.208+0000 7f7126ffd700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b208b0 0x559967b21d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.208+0000 7f710aff5700 10 client.15084 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.208+0000 7f7126ffd700 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b208b0 0x559967b21d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=113a1d5937d15a0c server_cookie=7237d41d95b0266e in_seq=0 out_seq=0 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f710aff5700 1 -- 172.21.15.73:0/137586627 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559967a65cc0 con 0x559967b208b0 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f710aff5700 10 client.15084 handle_command_reply: tid=0 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 client.15084 shutdown 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 msgr2=0x559967b0ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f711c0cc930 tx=0x7f711c09baf0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 shutdown_connections 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b208b0 0x559967b21d20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b06ed0 0x559967b12ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b10520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.213 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/137586627 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b0c5f0 0x559967b0ffe0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 >> 172.21.15.73:0/137586627 conn(0x559967a31c70 msgr2=0x559967b015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f710aff5700 0 client.15084 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:00:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 shutdown_connections 2024-06-06T07:00:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.209+0000 7f7137d3fb80 1 -- 172.21.15.73:0/137586627 wait complete. 2024-06-06T07:00:27.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.210+0000 7f7137d3fb80 20 client.15084 trim_cache size 0 max 16384 2024-06-06T07:00:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.232+0000 7f7137d3fb80 1 Processor -- start 2024-06-06T07:00:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.232+0000 7f7137d3fb80 1 -- start start 2024-06-06T07:00:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0c5f0 0x559967b0d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b0d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559967b20060 con 0x559967b06ed0 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559967b20730 con 0x559967b0cf10 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559967b20e00 con 0x559967b0c5f0 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58582/0 (socket says 172.21.15.73:58582) 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 -- 172.21.15.73:0/3916183313 learned_addr learned my addr 172.21.15.73:0/3916183313 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f71277fe700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b0d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f71267fc700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0c5f0 0x559967b0d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 -- 172.21.15.73:0/3916183313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0c5f0 msgr2=0x559967b0d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0c5f0 0x559967b0d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 -- 172.21.15.73:0/3916183313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 msgr2=0x559967b0d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b0d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 -- 172.21.15.73:0/3916183313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559967b12920 con 0x559967b06ed0 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f71277fe700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b0d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.233+0000 7f7126ffd700 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7118003b20 tx=0x7f7118016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c072a34fcab8395f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.234+0000 7f710cff9700 1 -- 172.21.15.73:0/3916183313 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118037020 con 0x559967b06ed0 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.234+0000 7f710cff9700 1 -- 172.21.15.73:0/3916183313 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7118012e40 con 0x559967b06ed0 2024-06-06T07:00:27.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.234+0000 7f710cff9700 1 -- 172.21.15.73:0/3916183313 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118004bb0 con 0x559967b06ed0 2024-06-06T07:00:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.234+0000 7f7137d3fb80 1 -- 172.21.15.73:0/3916183313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 msgr2=0x559967b06990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.234+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7118003b20 tx=0x7f7118016000 comp rx=0 tx=0).stop 2024-06-06T07:00:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- 172.21.15.73:0/3916183313 shutdown_connections 2024-06-06T07:00:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0cf10 0x559967b0d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.239 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b06990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/3916183313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0c5f0 0x559967b0d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- 172.21.15.73:0/3916183313 >> 172.21.15.73:0/3916183313 conn(0x559967a31c70 msgr2=0x559967b0a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- 172.21.15.73:0/3916183313 shutdown_connections 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- 172.21.15.73:0/3916183313 wait complete. 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- start start 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 Processor -- start 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 0x559967b26bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0cf10 0x559967b27130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559967b21ba0 con 0x559967b06ed0 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559967b214d0 con 0x559967b0c5f0 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559967b22270 con 0x559967b0cf10 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7137d3fb80 10 client.0 _ll_register_callbacks cb 0x5599679f28d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:27.240 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.235+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58594/0 (socket says 172.21.15.73:58594) 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 -- 172.21.15.73:0/4138015863 learned_addr learned my addr 172.21.15.73:0/4138015863 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f71267fc700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 0x559967b26bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 -- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0cf10 msgr2=0x559967b27130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f71277fe700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0cf10 0x559967b27130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0cf10 0x559967b27130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 -- 172.21.15.73:0/4138015863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 msgr2=0x559967b26bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 0x559967b26bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 -- 172.21.15.73:0/4138015863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559967b27950 con 0x559967b06ed0 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f71267fc700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 0x559967b26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:27.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7126ffd700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71180046d0 tx=0x7f7118004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34c3f10d04fe0e2e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f710aff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7118037030 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f7118011a60 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559967a60240 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711804c050 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7137d3fb80 10 client.15090 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.236+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559967b8c130 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x559967a60240 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71180334e0 con 0x559967b06ed0 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 10 client.15090 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 10 client.15090 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 20 client.15090 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b90600 0x559967b8f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:27.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 4 client.15090 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "status"}) -- 0x5599678da200 con 0x559967b90600 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f71267fc700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b90600 0x559967b8f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f71267fc700 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b90600 0x559967b8f310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=907ea2a791d91aad server_cookie=4d061443e021fd43 in_seq=0 out_seq=0 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.237+0000 7f710aff5700 10 client.15090 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f710aff5700 1 -- 172.21.15.73:0/4138015863 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5599678da200 con 0x559967b90600 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f710aff5700 10 client.15090 handle_command_reply: tid=0 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 client.15090 shutdown 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 msgr2=0x559967b2b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f71180046d0 tx=0x7f7118004360 comp rx=0 tx=0).stop 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 shutdown_connections 2024-06-06T07:00:27.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x559967b90600 0x559967b8f310 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559967b0cf10 0x559967b27130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f710aff5700 0 client.15090 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559967b06ed0 0x559967b2b870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 --2- 172.21.15.73:0/4138015863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559967b0c5f0 0x559967b26bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 >> 172.21.15.73:0/4138015863 conn(0x559967a31c70 msgr2=0x559967b0d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 shutdown_connections 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 1 -- 172.21.15.73:0/4138015863 wait complete. 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.238+0000 7f7137d3fb80 20 client.15090 trim_cache size 0 max 16384 2024-06-06T07:00:27.244 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 1, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24421, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 131.84613025799999, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 229.115510581 2024-06-06T07:00:27.245 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 msgr2=0x7f71100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 0x7f71100a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f71180045b0 tx=0x7f711800b3c0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 msgr2=0x7f7128101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f711c00d950 tx=0x7f711c00dc60 comp rx=0 tx=0).stop 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 shutdown_connections 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71281084f0 0x7f7128101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712806aa00 0x7f7128101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f71100a2c20 0x7f71100a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 --2- 172.21.15.73:0/1683282392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f712806b3c0 0x7f7128101990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:27.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 >> 172.21.15.73:0/1683282392 conn(0x7f7128078300 msgr2=0x7f7128100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:27.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 shutdown_connections 2024-06-06T07:00:27.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:27.239+0000 7f712d4bc700 1 -- 172.21.15.73:0/1683282392 wait complete. 2024-06-06T07:00:27.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T07:00:28.198 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:28.656 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:28 smithi073 ceph-mon[32303]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:28.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:28 smithi136 ceph-mon[35901]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:28 smithi170 ceph-mon[36108]: pgmap v408: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 -- 172.21.15.73:0/855827125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5601013c0 msgr2=0x7fd560101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 --2- 172.21.15.73:0/855827125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5601013c0 0x7fd560101840 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.899 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 -- 172.21.15.73:0/855827125 shutdown_connections 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 --2- 172.21.15.73:0/855827125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd560101d80 0x7fd56010e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 --2- 172.21.15.73:0/855827125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560107430 0x7fd560107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 --2- 172.21.15.73:0/855827125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5601013c0 0x7fd560101840 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 -- 172.21.15.73:0/855827125 >> 172.21.15.73:0/855827125 conn(0x7fd5600783d0 msgr2=0x7fd5600787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 -- 172.21.15.73:0/855827125 shutdown_connections 2024-06-06T07:00:29.900 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.898+0000 7fd56548a700 1 -- 172.21.15.73:0/855827125 wait complete. 2024-06-06T07:00:29.901 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 Processor -- start 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 -- start start 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5601013c0 0x7fd56019e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560101d80 0x7fd56019ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111650 con 0x7fd560107430 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601114d0 con 0x7fd560101d80 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.899+0000 7fd56548a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601117d0 con 0x7fd5601013c0 2024-06-06T07:00:29.902 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58622/0 (socket says 172.21.15.73:58622) 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 -- 172.21.15.73:0/2778294277 learned_addr learned my addr 172.21.15.73:0/2778294277 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55effd700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5601013c0 0x7fd56019e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560101d80 0x7fd56019ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 -- 172.21.15.73:0/2778294277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5601013c0 msgr2=0x7fd56019e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5601013c0 0x7fd56019e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 -- 172.21.15.73:0/2778294277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560101d80 msgr2=0x7fd56019ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560101d80 0x7fd56019ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 -- 172.21.15.73:0/2778294277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5601991f0 con 0x7fd560107430 2024-06-06T07:00:29.903 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b970 tx=0x7fd55800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd557fff700 1 -- 172.21.15.73:0/2778294277 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55800d860 con 0x7fd560107430 2024-06-06T07:00:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd557fff700 1 -- 172.21.15.73:0/2778294277 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55800e430 con 0x7fd560107430 2024-06-06T07:00:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5601994e0 con 0x7fd560107430 2024-06-06T07:00:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd557fff700 1 -- 172.21.15.73:0/2778294277 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558017660 con 0x7fd560107430 2024-06-06T07:00:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.900+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5601a57a0 con 0x7fd560107430 2024-06-06T07:00:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.902+0000 7fd557fff700 1 -- 172.21.15.73:0/2778294277 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd55800e5e0 con 0x7fd560107430 2024-06-06T07:00:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.903+0000 7fd557fff700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 0x7fd54c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.903+0000 7fd55effd700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 0x7fd54c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.903+0000 7fd557fff700 1 -- 172.21.15.73:0/2778294277 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd5580cefe0 con 0x7fd560107430 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd55effd700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 0x7fd54c0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd560199bd0 tx=0x7fd548009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 Processor -- start 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 -- start start 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fe2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5a174f7c0 con 0x55a5a18035f0 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5a16f76d0 con 0x55a5a1803f10 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a5a174b780 con 0x55a5a17fded0 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.904+0000 7fd55effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58634/0 (socket says 172.21.15.73:58634) 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 -- 172.21.15.73:0/3466597128 learned_addr learned my addr 172.21.15.73:0/3466597128 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fe2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 -- 172.21.15.73:0/3466597128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 msgr2=0x55a5a17fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fe2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 -- 172.21.15.73:0/3466597128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 msgr2=0x55a5a17fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 -- 172.21.15.73:0/3466597128 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a16edf00 con 0x55a5a18035f0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd55effd700 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd548009fd0 tx=0x7fd548011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab2270701788c1e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd5557fa700 1 -- 172.21.15.73:0/3466597128 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd54801b040 con 0x55a5a18035f0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd5557fa700 1 -- 172.21.15.73:0/3466597128 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548024070 con 0x55a5a18035f0 2024-06-06T07:00:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.905+0000 7fd5557fa700 1 -- 172.21.15.73:0/3466597128 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5480136d0 con 0x55a5a18035f0 2024-06-06T07:00:29.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3466597128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 msgr2=0x55a5a18039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fd548009fd0 tx=0x7fd548011000 comp rx=0 tx=0).stop 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3466597128 shutdown_connections 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fe2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a18039d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3466597128 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3466597128 >> 172.21.15.73:0/3466597128 conn(0x55a5a1728c70 msgr2=0x55a5a17f8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3466597128 shutdown_connections 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3466597128 wait complete. 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.906+0000 7fd56fd0db80 1 -- start start 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 Processor -- start 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a1809c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a18074b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5a180ed90 con 0x55a5a18035f0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5a16efba0 con 0x55a5a1803f10 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a5a180ef10 con 0x55a5a17fded0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd56fd0db80 10 client.0 _ll_register_callbacks cb 0x55a5a16e98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58638/0 (socket says 172.21.15.73:58638) 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 -- 172.21.15.73:0/4240101294 learned_addr learned my addr 172.21.15.73:0/4240101294 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 -- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 msgr2=0x55a5a1809c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a1809c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 -- 172.21.15.73:0/4240101294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 msgr2=0x55a5a18074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a18074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 -- 172.21.15.73:0/4240101294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a169e880 con 0x55a5a18035f0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd55000bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a6493c278ea853 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.907+0000 7fd5467fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500054e0 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd550005680 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd548024070 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd55002f050 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 10 client.15102 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5a1819630 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd548024070 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd55002a030 con 0x55a5a18035f0 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 10 client.15102 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 10 client.15102 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 20 client.15102 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a181bd30 0x55a5a181d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 4 client.15102 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:00:29.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5a175ccc0 con 0x55a5a181bd30 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.908+0000 7fd55effd700 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a181bd30 0x55a5a181d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.909+0000 7fd5467fc700 10 client.15102 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.909+0000 7fd55effd700 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a181bd30 0x55a5a181d1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77a5c431e9520f6d server_cookie=198c2a151c0d8693 in_seq=0 out_seq=0 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd5467fc700 1 -- 172.21.15.73:0/4240101294 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5a175ccc0 con 0x55a5a181bd30 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd5467fc700 10 client.15102 handle_command_reply: tid=0 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 client.15102 shutdown 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 msgr2=0x55a5a1806f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd55000bf00 comp rx=0 tx=0).stop 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 shutdown_connections 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a181bd30 0x55a5a181d1a0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a1809c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a18074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/4240101294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a1806f70 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 >> 172.21.15.73:0/4240101294 conn(0x55a5a1728c70 msgr2=0x55a5a17fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd5467fc700 0 client.15102 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:00:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 shutdown_connections 2024-06-06T07:00:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 1 -- 172.21.15.73:0/4240101294 wait complete. 2024-06-06T07:00:29.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.910+0000 7fd56fd0db80 20 client.15102 trim_cache size 0 max 16384 2024-06-06T07:00:29.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 Processor -- start 2024-06-06T07:00:29.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 -- start start 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181b4e0 con 0x55a5a18035f0 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181bbb0 con 0x55a5a1803f10 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181c280 con 0x55a5a17fded0 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd55e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.934 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd55e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58646/0 (socket says 172.21.15.73:58646) 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.932+0000 7fd55e7fc700 1 -- 172.21.15.73:0/3370014367 learned_addr learned my addr 172.21.15.73:0/3370014367 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 -- 172.21.15.73:0/3370014367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 msgr2=0x55a5a17fd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 -- 172.21.15.73:0/3370014367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 msgr2=0x55a5a17fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 -- 172.21.15.73:0/3370014367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5a18054e0 con 0x55a5a18035f0 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:29.935 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd55000fb70 tx=0x7fd55000fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985d1fe3b23f8d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd554ff9700 1 -- 172.21.15.73:0/3370014367 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550037020 con 0x55a5a18035f0 2024-06-06T07:00:29.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd554ff9700 1 -- 172.21.15.73:0/3370014367 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55000f660 con 0x55a5a18035f0 2024-06-06T07:00:29.937 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.933+0000 7fd554ff9700 1 -- 172.21.15.73:0/3370014367 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5500224d0 con 0x55a5a18035f0 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3370014367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 msgr2=0x55a5a17f89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fd55000fb70 tx=0x7fd55000fe80 comp rx=0 tx=0).stop 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3370014367 shutdown_connections 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a1803f10 0x55a5a17fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/3370014367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a18035f0 0x55a5a17f89a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3370014367 >> 172.21.15.73:0/3370014367 conn(0x55a5a1728c70 msgr2=0x55a5a1805860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3370014367 shutdown_connections 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- 172.21.15.73:0/3370014367 wait complete. 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 -- start start 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.934+0000 7fd56fd0db80 1 Processor -- start 2024-06-06T07:00:29.941 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fcb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a18035f0 0x55a5a17fd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181d020 con 0x55a5a1803f10 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181c950 con 0x55a5a18035f0 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a5a181d6f0 con 0x55a5a17fded0 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 10 client.0 _ll_register_callbacks cb 0x55a5a16e98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58648/0 (socket says 172.21.15.73:58648) 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 -- 172.21.15.73:0/1324008531 learned_addr learned my addr 172.21.15.73:0/1324008531 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55e7fc700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a18035f0 0x55a5a17fd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.942 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55effd700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fcb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 -- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 msgr2=0x55a5a17fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fcb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 -- 172.21.15.73:0/1324008531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a18035f0 msgr2=0x55a5a17fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a18035f0 0x55a5a17fd0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 -- 172.21.15.73:0/1324008531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5a1825c50 con 0x55a5a1803f10 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55effd700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd55f7fe700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd558097e40 tx=0x7fd55809add0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff372ced01f41b6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd5467fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580b4060 con 0x55a5a1803f10 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fd5580b4400 con 0x55a5a1803f10 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.935+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd55000f660 con 0x55a5a1803f10 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd55809b470 con 0x55a5a1803f10 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 10 client.15108 fetch_fsmap learned FSMap version 29 2024-06-06T07:00:29.943 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5a1826f00 con 0x55a5a1803f10 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd55000f660 con 0x55a5a1803f10 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd55801f030 con 0x55a5a1803f10 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 10 client.15108 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 10 client.15108 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 20 client.15108 populate_metadata read hostname 'smithi073' 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a188bba0 0x55a5a188a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 4 client.15108 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "status"}) -- 0x55a5a1757240 con 0x55a5a188bba0 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.936+0000 7fd55effd700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a188bba0 0x55a5a188a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd55effd700 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a188bba0 0x55a5a188a760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57461a3619ae95ba server_cookie=bd1e3d14a4023640 in_seq=0 out_seq=0 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd5467fc700 10 client.15108 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd5467fc700 1 -- 172.21.15.73:0/1324008531 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x55a5a1757240 con 0x55a5a188bba0 2024-06-06T07:00:29.944 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd5467fc700 10 client.15108 handle_command_reply: tid=0 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 client.15108 shutdown 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 msgr2=0x55a5a1825610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fd558097e40 tx=0x7fd55809add0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 shutdown_connections 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a5a188bba0 0x55a5a188a760 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a5a17fded0 0x55a5a17fcb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a5a1803f10 0x55a5a1825610 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 --2- 172.21.15.73:0/1324008531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a5a18035f0 0x55a5a17fd0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 >> 172.21.15.73:0/1324008531 conn(0x55a5a1728c70 msgr2=0x55a5a17f1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd5467fc700 0 client.15108 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 shutdown_connections 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.937+0000 7fd56fd0db80 1 -- 172.21.15.73:0/1324008531 wait complete. 2024-06-06T07:00:29.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56fd0db80 20 client.15108 trim_cache size 0 max 16384 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24433, 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 200.394647166, 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:00:29.946 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:00:29.947 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:00:29.947 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 226.20675547799999 2024-06-06T07:00:29.947 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:00:29.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 msgr2=0x7fd54c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 0x7fd54c0a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd560199bd0 tx=0x7fd548009190 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 msgr2=0x7fd560198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd55800b970 tx=0x7fd55800bc80 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 shutdown_connections 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fd54c0a2c20 0x7fd54c0a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd5601013c0 0x7fd56019e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd560107430 0x7fd560198a90 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 --2- 172.21.15.73:0/2778294277 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd560101d80 0x7fd56019ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 >> 172.21.15.73:0/2778294277 conn(0x7fd5600783d0 msgr2=0x7fd560100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 shutdown_connections 2024-06-06T07:00:29.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:29.938+0000 7fd56548a700 1 -- 172.21.15.73:0/2778294277 wait complete. 2024-06-06T07:00:30.602 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:30 smithi073 ceph-mon[32303]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:30.633 INFO:tasks.cephadm:Setup complete, yielding 2024-06-06T07:00:30.633 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T07:00:30.643 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:00:30.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-06T07:00:30.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:30 smithi136 ceph-mon[35901]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:30.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:30 smithi170 ceph-mon[36108]: pgmap v409: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:30.885 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 -- 172.21.15.73:0/3142344295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 msgr2=0x7f90dc102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 --2- 172.21.15.73:0/3142344295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc102a40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90d00099c0 tx=0x7f90d0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 -- 172.21.15.73:0/3142344295 shutdown_connections 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 --2- 172.21.15.73:0/3142344295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc1013e0 0x7f90dc101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 --2- 172.21.15.73:0/3142344295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc102f80 0x7f90dc109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 --2- 172.21.15.73:0/3142344295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc102a40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 -- 172.21.15.73:0/3142344295 >> 172.21.15.73:0/3142344295 conn(0x7f90dc0fc980 msgr2=0x7f90dc0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 -- 172.21.15.73:0/3142344295 shutdown_connections 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.642+0000 7f90e13dc700 1 -- 172.21.15.73:0/3142344295 wait complete. 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 Processor -- start 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 -- start start 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 0x7f90dc06b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc102f80 0x7f90dc19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10ce80 con 0x7f90dc1025e0 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10cd00 con 0x7f90dc1013e0 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90e13dc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f90dc10d000 con 0x7f90dc102f80 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:32.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90da7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58670/0 (socket says 172.21.15.73:58670) 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90da7fc700 1 -- 172.21.15.73:0/91789170 learned_addr learned my addr 172.21.15.73:0/91789170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.643+0000 7f90daffd700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 0x7f90dc06b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90db7fe700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc102f80 0x7f90dc19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 -- 172.21.15.73:0/91789170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc102f80 msgr2=0x7f90dc19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc102f80 0x7f90dc19f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 -- 172.21.15.73:0/91789170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 msgr2=0x7f90dc06b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 0x7f90dc06b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 -- 172.21.15.73:0/91789170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90dc1a0290 con 0x7f90dc1025e0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90daffd700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 0x7f90dc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90da7fc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006010 tx=0x7f90d00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0027070 con 0x7f90dc1025e0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90dc1a0520 con 0x7f90dc1025e0 2024-06-06T07:00:32.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d00040d0 con 0x7f90dc1025e0 2024-06-06T07:00:32.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000f460 con 0x7f90dc1025e0 2024-06-06T07:00:32.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.644+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90dc1a09d0 con 0x7f90dc1025e0 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.645+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90dc069180 con 0x7f90dc1025e0 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.646+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90d002a070 con 0x7f90dc1025e0 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.647+0000 7f90c3fff700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 0x7f90c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.647+0000 7f90daffd700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 0x7f90c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.647+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f90d00d0bf0 con 0x7f90dc1025e0 2024-06-06T07:00:32.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.647+0000 7f90daffd700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 0x7f90c40a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005fd0 tx=0x7f90cc005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:32.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.650+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f90d0031070 con 0x7f90dc1025e0 2024-06-06T07:00:32.666 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:32 smithi170 ceph-mon[36108]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:32 smithi073 ceph-mon[32303]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:32.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:32 smithi136 ceph-mon[35901]: pgmap v410: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:32.868 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.867+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f90dc0628d0 con 0x7f90c40a2c20 2024-06-06T07:00:32.869 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.868+0000 7f90c3fff700 1 -- 172.21.15.73:0/91789170 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f90dc0628d0 con 0x7f90c40a2c20 2024-06-06T07:00:32.869 INFO:teuthology.orchestra.run.smithi073.stdout:Backend: cephadm 2024-06-06T07:00:32.870 INFO:teuthology.orchestra.run.smithi073.stdout:Available: Yes 2024-06-06T07:00:32.870 INFO:teuthology.orchestra.run.smithi073.stdout:Paused: No 2024-06-06T07:00:32.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 msgr2=0x7f90c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:32.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 0x7f90c40a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f90cc005fd0 tx=0x7f90cc005f00 comp rx=0 tx=0).stop 2024-06-06T07:00:32.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 msgr2=0x7f90dc19d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:32.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f90d0006010 tx=0x7f90d00047b0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 shutdown_connections 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f90c40a2c20 0x7f90c40a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f90dc102f80 0x7f90dc19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f90dc1013e0 0x7f90dc06b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 --2- 172.21.15.73:0/91789170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90dc1025e0 0x7f90dc19d330 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 >> 172.21.15.73:0/91789170 conn(0x7f90dc0fc980 msgr2=0x7f90dc105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 shutdown_connections 2024-06-06T07:00:32.873 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:32.871+0000 7f90e13dc700 1 -- 172.21.15.73:0/91789170 wait complete. 2024-06-06T07:00:33.531 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-06T07:00:33.772 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:34.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:34 smithi073 ceph-mon[32303]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:34.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:34 smithi073 ceph-mon[32303]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:34 smithi136 ceph-mon[35901]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:34 smithi136 ceph-mon[35901]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:34.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:34 smithi170 ceph-mon[36108]: from='client.15114 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:34.850 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:34 smithi170 ceph-mon[36108]: pgmap v411: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:35.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3194206319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01013f0 msgr2=0x7ff4e0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3194206319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01013f0 0x7ff4e0101810 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff4d40099c0 tx=0x7ff4d4009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3194206319 shutdown_connections 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3194206319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 0x7ff4e0109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3194206319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01013f0 0x7ff4e0101810 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3194206319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01025f0 0x7ff4e0102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3194206319 >> 172.21.15.73:0/3194206319 conn(0x7ff4e00fc990 msgr2=0x7ff4e00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.504+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3194206319 shutdown_connections 2024-06-06T07:00:35.506 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3194206319 wait complete. 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 Processor -- start 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 -- start start 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01013f0 0x7ff4e019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.505+0000 7ff4e70d0700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 0x7ff4e01a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e70d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010cee0 con 0x7ff4e01025f0 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e70d0700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010cd60 con 0x7ff4e0102f90 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e70d0700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e010d060 con 0x7ff4e01013f0 2024-06-06T07:00:35.508 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49542/0 (socket says 172.21.15.73:49542) 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 -- 172.21.15.73:0/3775581666 learned_addr learned my addr 172.21.15.73:0/3775581666 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e566d700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 0x7ff4e01a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e4e6c700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01013f0 0x7ff4e019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 -- 172.21.15.73:0/3775581666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01013f0 msgr2=0x7ff4e019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01013f0 0x7ff4e019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 -- 172.21.15.73:0/3775581666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 msgr2=0x7ff4e01a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 0x7ff4e01a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 -- 172.21.15.73:0/3775581666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01a2560 con 0x7ff4e01025f0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4dffff700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b560 tx=0x7ff4cc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc015020 con 0x7ff4e01025f0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e01a2850 con 0x7ff4e01025f0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4cc011ce0 con 0x7ff4e01025f0 2024-06-06T07:00:35.509 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.506+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc010cb0 con 0x7ff4e01025f0 2024-06-06T07:00:35.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.507+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e01a2e10 con 0x7ff4e01025f0 2024-06-06T07:00:35.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.508+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e0069180 con 0x7ff4e01025f0 2024-06-06T07:00:35.510 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.509+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4cc018020 con 0x7ff4e01025f0 2024-06-06T07:00:35.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.509+0000 7ff4ddffb700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 0x7ff4d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:35.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.510+0000 7ff4e4e6c700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 0x7ff4d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:35.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.510+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7ff4cc0cead0 con 0x7ff4e01025f0 2024-06-06T07:00:35.511 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.510+0000 7ff4e4e6c700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 0x7ff4d00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:35.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.514+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff4cc01f020 con 0x7ff4e01025f0 2024-06-06T07:00:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.723+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7ff4e00febd0 con 0x7ff4d00a2c20 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.736+0000 7ff4ddffb700 1 -- 172.21.15.73:0/3775581666 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7ff4e00febd0 con 0x7ff4d00a2c20 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:mds.a smithi073 running (3m) 40s ago 3m 17.3M - 19.0.0-2946-g3d89accf b045ac545f82 c07da97e1c21 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:mds.b smithi136 running (3m) 86s ago 3m 16.8M - 19.0.0-2946-g3d89accf b045ac545f82 fc98bd09bfb5 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:mds.c smithi170 running (3m) 86s ago 3m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 da2c64af4805 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:mds.d smithi073 running (4m) 40s ago 4m 16.8M - 19.0.0-2946-g3d89accf b045ac545f82 541006db1515 2024-06-06T07:00:35.737 INFO:teuthology.orchestra.run.smithi073.stdout:mds.e smithi136 running (4m) 86s ago 4m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 b8cc2f31a968 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.f smithi170 running (4m) 86s ago 4m 20.3M - 19.0.0-2946-g3d89accf b045ac545f82 b25d2fbd876b 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.g smithi073 running (3m) 40s ago 3m 17.4M - 19.0.0-2946-g3d89accf b045ac545f82 f128b7647a77 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.h smithi136 running (3m) 86s ago 3m 17.9M - 19.0.0-2946-g3d89accf b045ac545f82 a1394f712336 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.i smithi170 running (3m) 86s ago 3m 26.1M - 19.0.0-2946-g3d89accf b045ac545f82 4f18b496457b 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.j smithi073 running (4m) 40s ago 4m 17.0M - 19.0.0-2946-g3d89accf b045ac545f82 84afdb3d0407 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.k smithi136 running (4m) 86s ago 4m 18.7M - 19.0.0-2946-g3d89accf b045ac545f82 42a32b1367dc 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mds.l smithi170 running (4m) 86s ago 4m 16.0M - 19.0.0-2946-g3d89accf b045ac545f82 badabd1d0e54 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mgr.x smithi073 *:9283,8765 running (11m) 40s ago 11m 517M - 19.0.0-2946-g3d89accf b045ac545f82 80dbb3aa06b0 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mgr.y smithi136 *:8443,8765 running (9m) 86s ago 9m 455M - 19.0.0-2946-g3d89accf b045ac545f82 b8c0e20d91ae 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mgr.z smithi170 *:8443,8765 running (9m) 86s ago 9m 454M - 19.0.0-2946-g3d89accf b045ac545f82 4d278b441d69 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a smithi073 running (11m) 40s ago 11m 50.3M 2048M 19.0.0-2946-g3d89accf b045ac545f82 ddd01b50ec79 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b smithi136 running (9m) 86s ago 9m 39.7M 2048M 19.0.0-2946-g3d89accf b045ac545f82 93d43cff2aa5 2024-06-06T07:00:35.738 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c smithi170 running (9m) 86s ago 9m 42.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 6dabbe5f80af 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 smithi073 running (9m) 40s ago 9m 56.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 752566785d0d 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 smithi136 running (8m) 86s ago 8m 56.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 2dfa3cfd6fec 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 smithi170 running (8m) 86s ago 8m 58.1M 4272M 19.0.0-2946-g3d89accf b045ac545f82 5c6ca650074c 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 smithi073 running (8m) 40s ago 8m 53.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a9ba292d5087 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 smithi136 running (7m) 86s ago 7m 44.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 91e6b58100ff 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5 smithi170 running (7m) 86s ago 7m 56.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 5cac18921775 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6 smithi073 running (6m) 40s ago 6m 53.8M 4272M 19.0.0-2946-g3d89accf b045ac545f82 36724d157a1b 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 smithi136 running (6m) 86s ago 6m 47.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 098018c69f4a 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8 smithi170 running (5m) 86s ago 5m 44.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 a6d9a45fdb87 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9 smithi073 running (5m) 40s ago 5m 55.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 50d1c8632d66 2024-06-06T07:00:35.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10 smithi136 running (5m) 86s ago 5m 45.7M 4272M 19.0.0-2946-g3d89accf b045ac545f82 7b79ac4ced86 2024-06-06T07:00:35.740 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11 smithi170 running (4m) 86s ago 4m 52.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 7e5e7c202a19 2024-06-06T07:00:35.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 msgr2=0x7ff4d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:35.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 0x7ff4d00a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000c00 tx=0x7ff4d4005fb0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 msgr2=0x7ff4e019ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc00b560 tx=0x7ff4cc00b870 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 shutdown_connections 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7ff4d00a2c20 0x7ff4d00a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff4e01013f0 0x7ff4e019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff4e0102f90 0x7ff4e01a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 --2- 172.21.15.73:0/3775581666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4e01025f0 0x7ff4e019ae40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 >> 172.21.15.73:0/3775581666 conn(0x7ff4e00fc990 msgr2=0x7ff4e00fe4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.738+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 shutdown_connections 2024-06-06T07:00:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:35.739+0000 7ff4e70d0700 1 -- 172.21.15.73:0/3775581666 wait complete. 2024-06-06T07:00:36.516 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:36 smithi073 ceph-mon[32303]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:36.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-06T07:00:36.786 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:36 smithi136 ceph-mon[35901]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:36 smithi170 ceph-mon[36108]: pgmap v412: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:37 smithi073 ceph-mon[32303]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:37 smithi136 ceph-mon[35901]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:37.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:37 smithi170 ceph-mon[36108]: from='client.15120 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 -- 172.21.15.73:0/2145830767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd8074ef0 msgr2=0x7fccd8075370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/2145830767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd8074ef0 0x7fccd8075370 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fccc8009990 tx=0x7fccc8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 -- 172.21.15.73:0/2145830767 shutdown_connections 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/2145830767 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd810ead0 0x7fccd8110ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/2145830767 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd80764c0 0x7fccd8074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/2145830767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd8074ef0 0x7fccd8075370 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 -- 172.21.15.73:0/2145830767 >> 172.21.15.73:0/2145830767 conn(0x7fccd80fda90 msgr2=0x7fccd80ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:38.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 -- 172.21.15.73:0/2145830767 shutdown_connections 2024-06-06T07:00:38.408 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 -- 172.21.15.73:0/2145830767 wait complete. 2024-06-06T07:00:38.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.406+0000 7fccdfeb4700 1 Processor -- start 2024-06-06T07:00:38.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 -- start start 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8074ef0 0x7fccd819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 0x7fccd81a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114580 con 0x7fccd80764c0 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114400 con 0x7fccd810ead0 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdfeb4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fccd8114700 con 0x7fccd8074ef0 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49560/0 (socket says 172.21.15.73:49560) 2024-06-06T07:00:38.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 -- 172.21.15.73:0/3786854918 learned_addr learned my addr 172.21.15.73:0/3786854918 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccddc50700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8074ef0 0x7fccd819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccde451700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 0x7fccd81a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 -- 172.21.15.73:0/3786854918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8074ef0 msgr2=0x7fccd819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8074ef0 0x7fccd819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 -- 172.21.15.73:0/3786854918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 msgr2=0x7fccd81a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 0x7fccd81a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 -- 172.21.15.73:0/3786854918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccd81a6870 con 0x7fccd80764c0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccde451700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 0x7fccd81a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.407+0000 7fccdd44f700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fccc80038b0 tx=0x7fccc8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.408+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8027070 con 0x7fccd80764c0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.408+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fccc8004090 con 0x7fccd80764c0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.408+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccd81a6b00 con 0x7fccd80764c0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.408+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccc8022d90 con 0x7fccd80764c0 2024-06-06T07:00:38.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.408+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccd81a6fe0 con 0x7fccd80764c0 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.409+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccd8069180 con 0x7fccd80764c0 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.410+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fccc802a070 con 0x7fccd80764c0 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.410+0000 7fccceffd700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.411+0000 7fccddc50700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.411+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fccc80d09b0 con 0x7fccd80764c0 2024-06-06T07:00:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.411+0000 7fccddc50700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 0x7fccc40a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fccd81a0100 tx=0x7fccd400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:38.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.414+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fccc8031070 con 0x7fccd80764c0 2024-06-06T07:00:38.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.622+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fccd80628d0 con 0x7fccc40a2c20 2024-06-06T07:00:38.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.628+0000 7fccceffd700 1 -- 172.21.15.73:0/3786854918 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fccd80628d0 con 0x7fccc40a2c20 2024-06-06T07:00:38.629 INFO:teuthology.orchestra.run.smithi073.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-06T07:00:38.629 INFO:teuthology.orchestra.run.smithi073.stdout:mds.all 12/12 89s ago 4m smithi073=a;smithi073=d;smithi073=g;smithi073=j;smithi136=b;smithi136=e;smithi136=h;smithi136=k;smithi170=c;smithi170=f;smithi170=i;smithi170=l;count:12 2024-06-06T07:00:38.629 INFO:teuthology.orchestra.run.smithi073.stdout:mgr 3/3 89s ago 9m smithi073=x;smithi136=y;smithi170=z;count:3 2024-06-06T07:00:38.629 INFO:teuthology.orchestra.run.smithi073.stdout:mon 3/3 89s ago 10m smithi073:172.21.15.73=a;smithi136:172.21.15.136=b;smithi170:172.21.15.170=c;count:3 2024-06-06T07:00:38.629 INFO:teuthology.orchestra.run.smithi073.stdout:osd 12 89s ago - 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 msgr2=0x7fccc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 0x7fccc40a50e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fccd81a0100 tx=0x7fccd400a380 comp rx=0 tx=0).stop 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 msgr2=0x7fccd819f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fccc80038b0 tx=0x7fccc8009580 comp rx=0 tx=0).stop 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 shutdown_connections 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fccc40a2c20 0x7fccc40a50e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fccd8074ef0 0x7fccd819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fccd810ead0 0x7fccd81a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.631 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 --2- 172.21.15.73:0/3786854918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccd80764c0 0x7fccd819f160 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:38.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 >> 172.21.15.73:0/3786854918 conn(0x7fccd80fda90 msgr2=0x7fccd8101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:38.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 shutdown_connections 2024-06-06T07:00:38.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:38.630+0000 7fccdfeb4700 1 -- 172.21.15.73:0/3786854918 wait complete. 2024-06-06T07:00:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:38 smithi073 ceph-mon[32303]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:38 smithi136 ceph-mon[35901]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:38 smithi170 ceph-mon[36108]: pgmap v413: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:39.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-06T07:00:39.528 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:39.650 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:39 smithi073 ceph-mon[32303]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:39.650 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:39 smithi073 ceph-mon[32303]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:39 smithi136 ceph-mon[35901]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:39 smithi136 ceph-mon[35901]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:39 smithi170 ceph-mon[36108]: from='client.15126 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:39 smithi170 ceph-mon[36108]: pgmap v414: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- 172.21.15.73:0/3920556642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca4104090 msgr2=0x7f8ca410a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 --2- 172.21.15.73:0/3920556642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca4104090 0x7f8ca410a940 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8c9c00b0d0 tx=0x7f8c9c00b3e0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- 172.21.15.73:0/3920556642 shutdown_connections 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 --2- 172.21.15.73:0/3920556642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca4104090 0x7f8ca410a940 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 --2- 172.21.15.73:0/3920556642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 0x7f8ca4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 --2- 172.21.15.73:0/3920556642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca41036f0 0x7f8ca4103b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- 172.21.15.73:0/3920556642 >> 172.21.15.73:0/3920556642 conn(0x7f8ca40fda90 msgr2=0x7f8ca40ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- 172.21.15.73:0/3920556642 shutdown_connections 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- 172.21.15.73:0/3920556642 wait complete. 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 Processor -- start 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.129+0000 7f8cac680700 1 -- start start 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 0x7f8ca41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca4104090 0x7f8ca419fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410df70 con 0x7f8ca41036f0 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410ddf0 con 0x7f8ca41024f0 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410e0f0 con 0x7f8ca4104090 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49582/0 (socket says 172.21.15.73:49582) 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 -- 172.21.15.73:0/1325952962 learned_addr learned my addr 172.21.15.73:0/1325952962 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8caac1d700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca4104090 0x7f8ca419fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8caa41c700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 0x7f8ca41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 -- 172.21.15.73:0/1325952962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca4104090 msgr2=0x7f8ca419fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca4104090 0x7f8ca419fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 -- 172.21.15.73:0/1325952962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 msgr2=0x7f8ca41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 0x7f8ca41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 -- 172.21.15.73:0/1325952962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca41a0210 con 0x7f8ca41036f0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8ca9c1b700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c950 tx=0x7f8c9400cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c94007ac0 con 0x7f8ca41036f0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c94004ce0 con 0x7f8ca41036f0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca41a0500 con 0x7f8ca41036f0 2024-06-06T07:00:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c940055e0 con 0x7f8ca41036f0 2024-06-06T07:00:41.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.130+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41a0a40 con 0x7f8ca41036f0 2024-06-06T07:00:41.134 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.131+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ca4069180 con 0x7f8ca41036f0 2024-06-06T07:00:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.132+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c94004750 con 0x7f8ca41036f0 2024-06-06T07:00:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.133+0000 7f8c9b7fe700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 0x7f8c900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.133+0000 7f8caa41c700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 0x7f8c900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.133+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f8c940cf400 con 0x7f8ca41036f0 2024-06-06T07:00:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.133+0000 7f8caa41c700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 0x7f8c900a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4103550 tx=0x7f8ca000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:41.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.136+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c94014030 con 0x7f8ca41036f0 2024-06-06T07:00:41.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.341+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8ca40628d0 con 0x7f8c900a2c20 2024-06-06T07:00:41.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.343+0000 7f8c9b7fe700 1 -- 172.21.15.73:0/1325952962 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8ca40628d0 con 0x7f8c900a2c20 2024-06-06T07:00:41.343 INFO:teuthology.orchestra.run.smithi073.stdout:HOST ADDR LABELS STATUS 2024-06-06T07:00:41.344 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073 172.21.15.73 2024-06-06T07:00:41.344 INFO:teuthology.orchestra.run.smithi073.stdout:smithi136 172.21.15.136 2024-06-06T07:00:41.344 INFO:teuthology.orchestra.run.smithi073.stdout:smithi170 172.21.15.170 2024-06-06T07:00:41.344 INFO:teuthology.orchestra.run.smithi073.stdout:3 hosts in cluster 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 msgr2=0x7f8c900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 0x7f8c900a50e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8ca4103550 tx=0x7f8ca000b5b0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 msgr2=0x7f8ca4198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8c9400c950 tx=0x7f8c9400cd10 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 shutdown_connections 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f8c900a2c20 0x7f8c900a50e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8ca4104090 0x7f8ca419fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8ca41024f0 0x7f8ca41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 --2- 172.21.15.73:0/1325952962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ca41036f0 0x7f8ca4198af0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 >> 172.21.15.73:0/1325952962 conn(0x7f8ca40fda90 msgr2=0x7f8ca4106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:41.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 shutdown_connections 2024-06-06T07:00:41.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:41.345+0000 7f8cac680700 1 -- 172.21.15.73:0/1325952962 wait complete. 2024-06-06T07:00:42.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-06T07:00:42.298 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:42.700 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:42 smithi073 ceph-mon[32303]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:42.700 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:42 smithi073 ceph-mon[32303]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:42.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:42 smithi136 ceph-mon[35901]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:42.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:42 smithi136 ceph-mon[35901]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:42.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:42 smithi170 ceph-mon[36108]: from='client.15132 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:42.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:42 smithi170 ceph-mon[36108]: pgmap v415: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 -- 172.21.15.73:0/1448569316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2001013f0 msgr2=0x7fb200101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 --2- 172.21.15.73:0/1448569316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2001013f0 0x7fb200101810 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00099c0 tx=0x7fb1f0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 -- 172.21.15.73:0/1448569316 shutdown_connections 2024-06-06T07:00:43.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 --2- 172.21.15.73:0/1448569316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb200102f90 0x7fb200109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 --2- 172.21.15.73:0/1448569316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2001013f0 0x7fb200101810 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 --2- 172.21.15.73:0/1448569316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 0x7fb200102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.910+0000 7fb2056ba700 1 -- 172.21.15.73:0/1448569316 >> 172.21.15.73:0/1448569316 conn(0x7fb2000fc990 msgr2=0x7fb2000fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- 172.21.15.73:0/1448569316 shutdown_connections 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- 172.21.15.73:0/1448569316 wait complete. 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 Processor -- start 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- start start 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2001013f0 0x7fb20006acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 0x7fb20019d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:43.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010ce90 con 0x7fb200102f90 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010cd10 con 0x7fb2001025f0 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.911+0000 7fb2056ba700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb20010d010 con 0x7fb2001013f0 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49612/0 (socket says 172.21.15.73:49612) 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 -- 172.21.15.73:0/2490102621 learned_addr learned my addr 172.21.15.73:0/2490102621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1feffd700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2001013f0 0x7fb20006acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1fe7fc700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 0x7fb20019d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 -- 172.21.15.73:0/2490102621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2001013f0 msgr2=0x7fb20006acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2001013f0 0x7fb20006acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 -- 172.21.15.73:0/2490102621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 msgr2=0x7fb20019d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:43.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 0x7fb20019d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 -- 172.21.15.73:0/2490102621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2001971b0 con 0x7fb200102f90 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1ff7fe700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007ee0 tx=0x7fb1f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f8004bb0 con 0x7fb200102f90 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1f8004d50 con 0x7fb200102f90 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2001974a0 con 0x7fb200102f90 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f80058e0 con 0x7fb200102f90 2024-06-06T07:00:43.916 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.912+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2001979e0 con 0x7fb200102f90 2024-06-06T07:00:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.913+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb200069180 con 0x7fb200102f90 2024-06-06T07:00:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.914+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1f801a020 con 0x7fb200102f90 2024-06-06T07:00:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.915+0000 7fb1e7fff700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 0x7fb1e80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.915+0000 7fb1feffd700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 0x7fb1e80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:43.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.915+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fb1f80ceb40 con 0x7fb200102f90 2024-06-06T07:00:43.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.916+0000 7fb1feffd700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 0x7fb1e80a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009990 tx=0x7fb1f0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:43.920 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:43.920+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1f8010070 con 0x7fb200102f90 2024-06-06T07:00:44.126 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.125+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 --> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fb200108490 con 0x7fb1e80a2c20 2024-06-06T07:00:44.128 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.127+0000 7fb1e7fff700 1 -- 172.21.15.73:0/2490102621 <== mgr.14152 v2:172.21.15.73:6800/793833713 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fb200108490 con 0x7fb1e80a2c20 2024-06-06T07:00:44.128 INFO:teuthology.orchestra.run.smithi073.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-06T07:00:44.128 INFO:teuthology.orchestra.run.smithi073.stdout:smithi073 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040115400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-06-06T07:00:44.128 INFO:teuthology.orchestra.run.smithi073.stdout:smithi136 /dev/nvme0n1 ssd INTEL_SSDPEDMX020T4_CVPD514100122P0SGN 1863G No 5m ago Has a FileSystem, LVM detected 2024-06-06T07:00:44.128 INFO:teuthology.orchestra.run.smithi073.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-06-06T07:00:44.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 msgr2=0x7fb1e80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:44.130 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 0x7fb1e80a50e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0009990 tx=0x7fb1f0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 msgr2=0x7fb20019fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8007ee0 tx=0x7fb1f800d3b0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 shutdown_connections 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fb1e80a2c20 0x7fb1e80a50e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb2001013f0 0x7fb20006acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb200102f90 0x7fb20019fa30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 --2- 172.21.15.73:0/2490102621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb2001025f0 0x7fb20019d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.129+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 >> 172.21.15.73:0/2490102621 conn(0x7fb2000fc990 msgr2=0x7fb200105a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.130+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 shutdown_connections 2024-06-06T07:00:44.131 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:44.130+0000 7fb2056ba700 1 -- 172.21.15.73:0/2490102621 wait complete. 2024-06-06T07:00:44.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:44 smithi073 ceph-mon[32303]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:44 smithi136 ceph-mon[35901]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:44.822 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-06T07:00:44.927 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:00:44.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-06T07:00:44.928 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:44 smithi170 ceph-mon[36108]: pgmap v416: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:45.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:45 smithi073 ceph-mon[32303]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:45 smithi136 ceph-mon[35901]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:45 smithi170 ceph-mon[36108]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:00:46.650 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:46 smithi073 ceph-mon[32303]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:46.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:46 smithi136 ceph-mon[35901]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:46.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:46 smithi170 ceph-mon[36108]: pgmap v417: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.905+0000 7eff534fb700 1 -- 172.21.15.73:0/769242010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1013f0 msgr2=0x7eff4c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.905+0000 7eff534fb700 1 --2- 172.21.15.73:0/769242010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1013f0 0x7eff4c101810 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7eff4400b0d0 tx=0x7eff4400b3e0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 -- 172.21.15.73:0/769242010 shutdown_connections 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 --2- 172.21.15.73:0/769242010 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c102f90 0x7eff4c109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 --2- 172.21.15.73:0/769242010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1013f0 0x7eff4c101810 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 --2- 172.21.15.73:0/769242010 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c1025f0 0x7eff4c102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 -- 172.21.15.73:0/769242010 >> 172.21.15.73:0/769242010 conn(0x7eff4c0fc990 msgr2=0x7eff4c0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 -- 172.21.15.73:0/769242010 shutdown_connections 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 -- 172.21.15.73:0/769242010 wait complete. 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 Processor -- start 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 -- start start 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c1013f0 0x7eff4c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.906+0000 7eff534fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:46.910 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff534fb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 0x7eff4c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff534fb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10ce60 con 0x7eff4c1025f0 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff534fb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10cce0 con 0x7eff4c1013f0 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff534fb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7eff4c10cfe0 con 0x7eff4c102f90 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50936/0 (socket says 172.21.15.73:50936) 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 -- 172.21.15.73:0/2745285207 learned_addr learned my addr 172.21.15.73:0/2745285207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff51a98700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 0x7eff4c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff51297700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c1013f0 0x7eff4c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 -- 172.21.15.73:0/2745285207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 msgr2=0x7eff4c19eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 0x7eff4c19eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 -- 172.21.15.73:0/2745285207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c1013f0 msgr2=0x7eff4c197500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c1013f0 0x7eff4c197500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:46.911 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 -- 172.21.15.73:0/2745285207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff4c19f160 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff51a98700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 0x7eff4c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.907+0000 7eff50a96700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7eff40011b80 tx=0x7eff40011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.908+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400079d0 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.908+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff4c06a1d0 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.908+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff4000c280 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.908+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff400193e0 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.908+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff4c06a760 con 0x7eff4c1025f0 2024-06-06T07:00:46.912 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.909+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff4c069180 con 0x7eff4c1025f0 2024-06-06T07:00:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.911+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff4000a440 con 0x7eff4c1025f0 2024-06-06T07:00:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.912+0000 7eff4a7fc700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 0x7eff340a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.912+0000 7eff51297700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 0x7eff340a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.912+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7eff400d3000 con 0x7eff4c1025f0 2024-06-06T07:00:46.913 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.913+0000 7eff51297700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 0x7eff340a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff44009fd0 tx=0x7eff4400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:46.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:46.915+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7eff4009fb90 con 0x7eff4c1025f0 2024-06-06T07:00:47.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.176+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7eff4c108230 con 0x7eff4c1025f0 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.177+0000 7eff4a7fc700 1 -- 172.21.15.73:0/2745285207 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3092 (secure 0 0 0) 0x7eff4009f490 con 0x7eff4c1025f0 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:e29 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:legacy client fscid: 1 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:47.178 INFO:teuthology.orchestra.run.smithi073.stdout:Filesystem 'cephfs' (1) 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:fs_name cephfs 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 29 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-06-06T06:57:09.432568+0000 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-06-06T06:58:21.359605+0000 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:tableserver 0 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:root 0 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:session_timeout 300 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:session_autoclose 300 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:max_file_size 1099511627776 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:max_xattr_size 65536 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:required_client_features {} 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:last_failure 0 2024-06-06T07:00:47.179 INFO:teuthology.orchestra.run.smithi073.stdout:last_failure_osd_epoch 0 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:max_mds 5 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:in 0,1,2,3,4 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:up {0=24433,1=24421,2=24419,3=24415,4=24404} 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:failed 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:damaged 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:stopped 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:data_pools [3,4] 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:metadata_pool 2 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:inline_data disabled 2024-06-06T07:00:47.180 INFO:teuthology.orchestra.run.smithi073.stdout:balancer 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:bal_rank_mask -1 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:standby_count_wanted 0 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:qdb_cluster leader: 24433 members: 24404,24415,24421,24433,24419 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.i{0:24433} state up:active seq 8 addr [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.e{0:24380} state up:standby-replay seq 1 addr [v2:172.21.15.136:6834/687158910,v1:172.21.15.136:6835/687158910] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c{1:24421} state up:active seq 26 addr [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.k{1:24365} state up:standby-replay seq 1 addr [v2:172.21.15.136:6832/464039391,v1:172.21.15.136:6833/464039391] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b{2:24419} state up:active seq 22 addr [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a{2:14583} state up:standby-replay seq 1 addr [v2:172.21.15.73:6840/552212635,v1:172.21.15.73:6841/552212635] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.f{3:24415} state up:active seq 27 addr [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.g{3:14571} state up:standby-replay seq 1 addr [v2:172.21.15.73:6838/2070842628,v1:172.21.15.73:6839/2070842628] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.h{4:24404} state up:active seq 26 addr [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d{4:14529} state up:standby-replay seq 1 addr [v2:172.21.15.73:6836/38418017,v1:172.21.15.73:6837/38418017] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:47.181 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:47.182 INFO:teuthology.orchestra.run.smithi073.stdout:Standby daemons: 2024-06-06T07:00:47.182 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:47.182 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.l{-1:14505} state up:standby seq 1 addr [v2:172.21.15.170:6832/3776485988,v1:172.21.15.170:6833/3776485988] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.182 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.j{-1:14523} state up:standby seq 1 addr [v2:172.21.15.73:6834/2227329730,v1:172.21.15.73:6835/2227329730] compat {c=[1],r=[1],i=[1fff]}] 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 msgr2=0x7eff340a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 0x7eff340a5160 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7eff44009fd0 tx=0x7eff4400b040 comp rx=0 tx=0).stop 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 msgr2=0x7eff4c197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7eff40011b80 tx=0x7eff40011e90 comp rx=0 tx=0).stop 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 shutdown_connections 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7eff340a2ca0 0x7eff340a5160 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7eff4c102f90 0x7eff4c19eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7eff4c1013f0 0x7eff4c197500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:47.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 --2- 172.21.15.73:0/2745285207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff4c1025f0 0x7eff4c197a40 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:47.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 >> 172.21.15.73:0/2745285207 conn(0x7eff4c0fc990 msgr2=0x7eff4c105990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:47.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 shutdown_connections 2024-06-06T07:00:47.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:47.179+0000 7eff534fb700 1 -- 172.21.15.73:0/2745285207 wait complete. 2024-06-06T07:00:47.184 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:47 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2745285207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:00:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:47 smithi073 ceph-mon[32303]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:47 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2745285207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:00:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:47 smithi136 ceph-mon[35901]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:47.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-06T07:00:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:47 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2745285207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-06T07:00:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:47 smithi170 ceph-mon[36108]: pgmap v418: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:48.072 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:49.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 -- 172.21.15.73:0/3154471693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8101400 msgr2=0x7f00e8101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:49.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/3154471693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8101400 0x7f00e8101820 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f00d80099c0 tx=0x7f00d8009cd0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 -- 172.21.15.73:0/3154471693 shutdown_connections 2024-06-06T07:00:49.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/3154471693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e8109850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/3154471693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8101400 0x7f00e8101820 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/3154471693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8102600 0x7f00e8102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.758+0000 7f00ed6f1700 1 -- 172.21.15.73:0/3154471693 >> 172.21.15.73:0/3154471693 conn(0x7f00e80fc9c0 msgr2=0x7f00e80fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.759+0000 7f00ed6f1700 1 -- 172.21.15.73:0/3154471693 shutdown_connections 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.759+0000 7f00ed6f1700 1 -- 172.21.15.73:0/3154471693 wait complete. 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.759+0000 7f00ed6f1700 1 Processor -- start 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.759+0000 7f00ed6f1700 1 -- start start 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.759+0000 7f00ed6f1700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8101400 0x7f00e8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:49.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00e810ce70 con 0x7f00e8102600 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00e810ccf0 con 0x7f00e8101400 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00e810cff0 con 0x7f00e8102fa0 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50960/0 (socket says 172.21.15.73:50960) 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 -- 172.21.15.73:0/1979884687 learned_addr learned my addr 172.21.15.73:0/1979884687 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e6ffd700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8101400 0x7f00e8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e77fe700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 -- 172.21.15.73:0/1979884687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 msgr2=0x7f00e819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 -- 172.21.15.73:0/1979884687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8101400 msgr2=0x7f00e8198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8101400 0x7f00e8198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 -- 172.21.15.73:0/1979884687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00e81a0320 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e77fe700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00e67fc700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c950 tx=0x7f00dc00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007ac0 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00e81a0610 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00dc004ce0 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00dc0055e0 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.760+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00e81a0ba0 con 0x7f00e8102600 2024-06-06T07:00:49.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.761+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00e8069180 con 0x7f00e8102600 2024-06-06T07:00:49.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.763+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f00dc004750 con 0x7f00e8102600 2024-06-06T07:00:49.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.763+0000 7f00cffff700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 0x7f00d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:49.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.763+0000 7f00e6ffd700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 0x7f00d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:49.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.763+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f00dc0cf390 con 0x7f00e8102600 2024-06-06T07:00:49.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.764+0000 7f00e6ffd700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 0x7f00d00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d8005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:49.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.767+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f00dc014030 con 0x7f00e8102600 2024-06-06T07:00:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.963+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f00e81a0fd0 con 0x7f00e8102600 2024-06-06T07:00:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.964+0000 7f00cffff700 1 -- 172.21.15.73:0/1979884687 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v100) ==== 56+0+5695 (secure 0 0 0) 0x7f00dc09bfc0 con 0x7f00e8102600 2024-06-06T07:00:49.965 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 100 2024-06-06T07:00:49.965 INFO:teuthology.orchestra.run.smithi073.stdout:fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T07:00:49.965 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-06-06T06:48:47.254289+0000 2024-06-06T07:00:49.965 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-06-06T06:58:53.383839+0000 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:crush_version 27 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:full_ratio 0.95 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:backfillfull_ratio 0.9 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:nearfull_ratio 0.85 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:require_min_compat_client luminous 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:min_compat_client luminous 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:require_osd_release squid 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:stretch_mode_enabled false 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-06T07:00:49.966 INFO:teuthology.orchestra.run.smithi073.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 96 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:max_osd 12 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 up in weight 1 up_from 8 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6802/3665933649,v1:172.21.15.73:6803/3665933649] [v2:172.21.15.73:6804/3665933649,v1:172.21.15.73:6805/3665933649] exists,up f1af7d41-9f4c-4b4a-8a05-c38530312412 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 up in weight 1 up_from 13 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/2860362689,v1:172.21.15.136:6801/2860362689] [v2:172.21.15.136:6802/2860362689,v1:172.21.15.136:6803/2860362689] exists,up ec8a9433-7b32-4f79-8a39-4137d445acfc 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 up in weight 1 up_from 18 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6800/2309158853,v1:172.21.15.170:6801/2309158853] [v2:172.21.15.170:6802/2309158853,v1:172.21.15.170:6803/2309158853] exists,up 3443c257-8d23-4b70-ab00-5d3add9a8b3d 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 up in weight 1 up_from 26 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6810/3983384250,v1:172.21.15.73:6811/3983384250] [v2:172.21.15.73:6812/3983384250,v1:172.21.15.73:6813/3983384250] exists,up e503419a-b7b8-4a0e-ab38-9459dbf2fdf9 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 up in weight 1 up_from 34 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6808/846852962,v1:172.21.15.136:6809/846852962] [v2:172.21.15.136:6810/846852962,v1:172.21.15.136:6811/846852962] exists,up f622b141-b741-4f89-9c8c-39615ff7919f 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5 up in weight 1 up_from 39 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6808/686557154,v1:172.21.15.170:6809/686557154] [v2:172.21.15.170:6810/686557154,v1:172.21.15.170:6811/686557154] exists,up 27b58b5d-313a-46d9-a7f5-e6c72db937c7 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6 up in weight 1 up_from 47 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6818/135647786,v1:172.21.15.73:6819/135647786] [v2:172.21.15.73:6820/135647786,v1:172.21.15.73:6821/135647786] exists,up de018a90-38be-4bea-b847-951585c33b94 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 up in weight 1 up_from 52 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6816/4094155747,v1:172.21.15.136:6817/4094155747] [v2:172.21.15.136:6818/4094155747,v1:172.21.15.136:6819/4094155747] exists,up 51b08617-a3c2-4de4-b0f7-3f505b13e7e6 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8 up in weight 1 up_from 60 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6816/65881112,v1:172.21.15.170:6817/65881112] [v2:172.21.15.170:6818/65881112,v1:172.21.15.170:6819/65881112] exists,up edab0c0c-df86-49f8-81b9-90f481b58410 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9 up in weight 1 up_from 68 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6826/2353282105,v1:172.21.15.73:6827/2353282105] [v2:172.21.15.73:6828/2353282105,v1:172.21.15.73:6829/2353282105] exists,up cd7bb4f4-bc75-4e2d-9a15-b414a84302ce 2024-06-06T07:00:49.967 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10 up in weight 1 up_from 73 up_thru 99 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6824/2185418171,v1:172.21.15.136:6825/2185418171] [v2:172.21.15.136:6826/2185418171,v1:172.21.15.136:6827/2185418171] exists,up cac56766-4505-4e9f-9b82-2dbf14ba3162 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11 up in weight 1 up_from 81 up_thru 97 down_at 0 last_clean_interval [0,0) [v2:172.21.15.170:6824/3729886831,v1:172.21.15.170:6825/3729886831] [v2:172.21.15.170:6826/3729886831,v1:172.21.15.170:6827/3729886831] exists,up c8610f25-d950-4f87-b337-29b59b3f71a7 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 2.0 [8,4] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 2.8 [2,11] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 2.1b [2,4] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 2.1d [8,11] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.3 [4,9] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.e [4,6] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.1a [5,7] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.21 [5,9] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.27 [3,9] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 3.3a [4,8] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 4.f [5,8] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 4.13 [9,10] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 4.1b [9,8] 2024-06-06T07:00:49.968 INFO:teuthology.orchestra.run.smithi073.stdout:pg_upmap_items 4.1c [9,8] 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:6800/3054677757 expires 2024-06-07T06:49:51.216738+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/1159052463 expires 2024-06-07T06:49:51.216738+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:6801/3054677757 expires 2024-06-07T06:49:51.216738+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/2836597527 expires 2024-06-07T06:49:51.216738+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/905820354 expires 2024-06-07T06:49:24.637645+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/27064626 expires 2024-06-07T06:49:24.637645+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/1626604755 expires 2024-06-07T06:49:24.637645+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:0/2368928774 expires 2024-06-07T06:49:51.216738+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:6800/961923295 expires 2024-06-07T06:49:24.637645+0000 2024-06-06T07:00:49.969 INFO:teuthology.orchestra.run.smithi073.stdout:blocklist 172.21.15.73:6801/961923295 expires 2024-06-07T06:49:24.637645+0000 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 msgr2=0x7f00d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 0x7f00d00a50e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f00d8000c00 tx=0x7f00d8005fd0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 msgr2=0x7f00e8198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c950 tx=0x7f00dc00cd10 comp rx=0 tx=0).stop 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 shutdown_connections 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f00d00a2c20 0x7f00d00a50e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f00e8102fa0 0x7f00e819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00e8101400 0x7f00e8198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 --2- 172.21.15.73:0/1979884687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00e8102600 0x7f00e8198c50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.966+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 >> 172.21.15.73:0/1979884687 conn(0x7f00e80fc9c0 msgr2=0x7f00e806ac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.967+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 shutdown_connections 2024-06-06T07:00:49.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:49.967+0000 7f00ed6f1700 1 -- 172.21.15.73:0/1979884687 wait complete. 2024-06-06T07:00:50.669 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:50 smithi073 ceph-mon[32303]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:50.670 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:50 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1979884687' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:00:50.699 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-06T07:00:50.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:00:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:50 smithi136 ceph-mon[35901]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:50.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:50 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1979884687' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:00:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:50 smithi170 ceph-mon[36108]: pgmap v419: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:00:50.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:50 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1979884687' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-06T07:00:50.952 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 -- 172.21.15.73:0/2166170425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8101a60 msgr2=0x7fe8a8105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 --2- 172.21.15.73:0/2166170425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8101a60 0x7fe8a8105820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe898009990 tx=0x7fe898009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 -- 172.21.15.73:0/2166170425 shutdown_connections 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 --2- 172.21.15.73:0/2166170425 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a81010b0 0x7fe8a8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 --2- 172.21.15.73:0/2166170425 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a8105f50 0x7fe8a810da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 --2- 172.21.15.73:0/2166170425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8101a60 0x7fe8a8105820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 -- 172.21.15.73:0/2166170425 >> 172.21.15.73:0/2166170425 conn(0x7fe8a80766b0 msgr2=0x7fe8a8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:52.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 -- 172.21.15.73:0/2166170425 shutdown_connections 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.516+0000 7fe8b0428700 1 -- 172.21.15.73:0/2166170425 wait complete. 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 Processor -- start 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 -- start start 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a81010b0 0x7fe8a819d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101a60 0x7fe8a819dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110df0 con 0x7fe8a8105f50 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110c70 con 0x7fe8a81010b0 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8b0428700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110f70 con 0x7fe8a8101a60 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50976/0 (socket says 172.21.15.73:50976) 2024-06-06T07:00:52.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 -- 172.21.15.73:0/1288100462 learned_addr learned my addr 172.21.15.73:0/1288100462 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ad9c3700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101a60 0x7fe8a819dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae1c4700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a81010b0 0x7fe8a819d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 -- 172.21.15.73:0/1288100462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101a60 msgr2=0x7fe8a819dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101a60 0x7fe8a819dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 -- 172.21.15.73:0/1288100462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a81010b0 msgr2=0x7fe8a819d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a81010b0 0x7fe8a819d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 -- 172.21.15.73:0/1288100462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a81980b0 con 0x7fe8a8105f50 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.517+0000 7fe8ae9c5700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000b970 tx=0x7fe8a000bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.518+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0019070 con 0x7fe8a8105f50 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.518+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8a0014440 con 0x7fe8a8105f50 2024-06-06T07:00:52.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.518+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a81983a0 con 0x7fe8a8105f50 2024-06-06T07:00:52.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.518+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8a000ce60 con 0x7fe8a8105f50 2024-06-06T07:00:52.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.518+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a4660 con 0x7fe8a8105f50 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.519+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8069180 con 0x7fe8a8105f50 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.520+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe8a001c070 con 0x7fe8a8105f50 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.520+0000 7fe89f7fe700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 0x7fe8940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.521+0000 7fe8ae1c4700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 0x7fe8940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.521+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fe8a00ce800 con 0x7fe8a8105f50 2024-06-06T07:00:52.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.521+0000 7fe8ae1c4700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 0x7fe8940a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8198a90 tx=0x7fe8a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:52.526 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.525+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe8a009b390 con 0x7fe8a8105f50 2024-06-06T07:00:52.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.765+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8a81a4950 con 0x7fe8a8105f50 2024-06-06T07:00:52.766 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:52 smithi073 ceph-mon[32303]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:52.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.766+0000 7fe89f7fe700 1 -- 172.21.15.73:0/1288100462 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7fe8a00b2eb0 con 0x7fe8a8105f50 2024-06-06T07:00:52.767 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:52.767 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 msgr2=0x7fe8940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 0x7fe8940a5160 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8198a90 tx=0x7fe8a4008040 comp rx=0 tx=0).stop 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 msgr2=0x7fe8a8197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000b970 tx=0x7fe8a000bd30 comp rx=0 tx=0).stop 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 shutdown_connections 2024-06-06T07:00:52.769 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8a8105f50 0x7fe8a8197950 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe8a81010b0 0x7fe8a819d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fe8940a2ca0 0x7fe8940a5160 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 --2- 172.21.15.73:0/1288100462 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe8a8101a60 0x7fe8a819dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 >> 172.21.15.73:0/1288100462 conn(0x7fe8a80766b0 msgr2=0x7fe8a8104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 shutdown_connections 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:52.768+0000 7fe8b0428700 1 -- 172.21.15.73:0/1288100462 wait complete. 2024-06-06T07:00:52.770 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:52.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:52 smithi136 ceph-mon[35901]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:52.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:52 smithi170 ceph-mon[36108]: pgmap v420: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:00:53.482 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:00:53.723 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:53 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1288100462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:53 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1288100462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:53 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1288100462' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:54.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:54 smithi073 ceph-mon[32303]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:54.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:54 smithi136 ceph-mon[35901]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:54 smithi170 ceph-mon[36108]: pgmap v421: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:55.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.435+0000 7f13406de700 1 -- 172.21.15.73:0/2690756642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 msgr2=0x7f13381088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:55.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.435+0000 7f13406de700 1 --2- 172.21.15.73:0/2690756642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f13381088d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1334009990 tx=0x7f1334009ca0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 -- 172.21.15.73:0/2690756642 shutdown_connections 2024-06-06T07:00:55.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 --2- 172.21.15.73:0/2690756642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 0x7f133810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 --2- 172.21.15.73:0/2690756642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f13381088d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 --2- 172.21.15.73:0/2690756642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f13381029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 -- 172.21.15.73:0/2690756642 >> 172.21.15.73:0/2690756642 conn(0x7f1338079650 msgr2=0x7f1338079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 -- 172.21.15.73:0/2690756642 shutdown_connections 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 -- 172.21.15.73:0/2690756642 wait complete. 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 Processor -- start 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 -- start start 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.436+0000 7f13406de700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f1338074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 0x7f1338074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:55.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1338112620 con 0x7f13381084f0 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f13381124a0 con 0x7f1338102f10 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f13381127a0 con 0x7f1338102550 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55082/0 (socket says 172.21.15.73:55082) 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 -- 172.21.15.73:0/2394824664 learned_addr learned my addr 172.21.15.73:0/2394824664 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133e47a700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f1338074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133dc79700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 0x7f1338074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 -- 172.21.15.73:0/2394824664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 msgr2=0x7f1338074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f1338074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 -- 172.21.15.73:0/2394824664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 msgr2=0x7f1338074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 0x7f1338074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 -- 172.21.15.73:0/2394824664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13381a70c0 con 0x7f13381084f0 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133e47a700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f1338074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:00:55.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f133ec7b700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1330007ee0 tx=0x7f133000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000dbd0 con 0x7f13381084f0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f133000dd70 con 0x7f13381084f0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13381a73b0 con 0x7f13381084f0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330005990 con 0x7f13381084f0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.437+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13381a78f0 con 0x7f13381084f0 2024-06-06T07:00:55.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.438+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1338103ba0 con 0x7f13381084f0 2024-06-06T07:00:55.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.439+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f133001a020 con 0x7f13381084f0 2024-06-06T07:00:55.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.440+0000 7f132f7fe700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:55.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.440+0000 7f133e47a700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:55.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.440+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f13300cea30 con 0x7f13381084f0 2024-06-06T07:00:55.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.441+0000 7f133e47a700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 0x7f13240a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1334006010 tx=0x7f133400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:55.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.445+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1330010070 con 0x7f13381084f0 2024-06-06T07:00:55.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.689+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1338075cc0 con 0x7f13381084f0 2024-06-06T07:00:55.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.690+0000 7f132f7fe700 1 -- 172.21.15.73:0/2394824664 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11932 (secure 0 0 0) 0x7f133009b5c0 con 0x7f13381084f0 2024-06-06T07:00:55.691 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:55.691 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T06:58:21.359605+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24404,24415,24421,24433,24419]},"id":1}]} 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 msgr2=0x7f13240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 0x7f13240a5160 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f1334006010 tx=0x7f133400b540 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 msgr2=0x7f1338075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f1330007ee0 tx=0x7f133000d3b0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 shutdown_connections 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f13240a2ca0 0x7f13240a5160 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1338102550 0x7f1338074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13381084f0 0x7f1338075210 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 --2- 172.21.15.73:0/2394824664 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1338102f10 0x7f1338074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:55.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 >> 172.21.15.73:0/2394824664 conn(0x7f1338079650 msgr2=0x7f1338100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:55.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 shutdown_connections 2024-06-06T07:00:55.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:55.693+0000 7f13406de700 1 -- 172.21.15.73:0/2394824664 wait complete. 2024-06-06T07:00:55.695 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 29 2024-06-06T07:00:56.358 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:00:56.611 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:56.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:56 smithi073 ceph-mon[32303]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:56.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:56 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2394824664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:56.736 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:56 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:00:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:56 smithi136 ceph-mon[35901]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:56 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2394824664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:56.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:56 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:00:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:56 smithi170 ceph-mon[36108]: pgmap v422: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:00:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:56 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2394824664' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:00:56.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:56 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:00:57.745 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.745 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.745 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: pgmap v423: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:00:57.746 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:57 smithi073 ceph-mon[32303]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: pgmap v423: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:00:58.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:57 smithi136 ceph-mon[35901]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: pgmap v423: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:00:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:00:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:00:58.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:57 smithi170 ceph-mon[36108]: from='mgr.14152 172.21.15.73:0/3070486241' entity='mgr.x' 2024-06-06T07:00:58.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 -- 172.21.15.73:0/1169087741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 msgr2=0x7f977810f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:58.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 --2- 172.21.15.73:0/1169087741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810f3a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f977000b0d0 tx=0x7f977000b3e0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 -- 172.21.15.73:0/1169087741 shutdown_connections 2024-06-06T07:00:58.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 --2- 172.21.15.73:0/1169087741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 0x7f97781089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 --2- 172.21.15.73:0/1169087741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810f3a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 --2- 172.21.15.73:0/1169087741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 0x7f97781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 -- 172.21.15.73:0/1169087741 >> 172.21.15.73:0/1169087741 conn(0x7f9778079640 msgr2=0x7f9778079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 -- 172.21.15.73:0/1169087741 shutdown_connections 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.325+0000 7f977d879700 1 -- 172.21.15.73:0/1169087741 wait complete. 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 Processor -- start 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 -- start start 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 0x7f977810e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 0x7f97781a3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f97781127b0 con 0x7f9778102ef0 2024-06-06T07:00:58.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9778112630 con 0x7f9778102530 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f977d879700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9778112930 con 0x7f97781085d0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f976ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f976ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55112/0 (socket says 172.21.15.73:55112) 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.326+0000 7f976ffff700 1 -- 172.21.15.73:0/1583406752 learned_addr learned my addr 172.21.15.73:0/1583406752 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f9776ffd700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 0x7f977810e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f97777fe700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 0x7f97781a3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 -- 172.21.15.73:0/1583406752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 msgr2=0x7f97781a3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 0x7f97781a3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 -- 172.21.15.73:0/1583406752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 msgr2=0x7f977810e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 0x7f977810e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 -- 172.21.15.73:0/1583406752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97781a41d0 con 0x7f9778102ef0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f976ffff700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f976000b530 tx=0x7f976000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:58.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760015020 con 0x7f9778102ef0 2024-06-06T07:00:58.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9760011ce0 con 0x7f9778102ef0 2024-06-06T07:00:58.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97781a4460 con 0x7f9778102ef0 2024-06-06T07:00:58.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760010e40 con 0x7f9778102ef0 2024-06-06T07:00:58.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.327+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97781abdb0 con 0x7f9778102ef0 2024-06-06T07:00:58.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.328+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9778103c40 con 0x7f9778102ef0 2024-06-06T07:00:58.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.330+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9760018020 con 0x7f9778102ef0 2024-06-06T07:00:58.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.331+0000 7f977c877700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 0x7f97640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:00:58.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.331+0000 7f9776ffd700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 0x7f97640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:00:58.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.331+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f97600cf1f0 con 0x7f9778102ef0 2024-06-06T07:00:58.332 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.331+0000 7f9776ffd700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 0x7f97640a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9768009a10 tx=0x7f9768009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:00:58.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.334+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f976001f020 con 0x7f9778102ef0 2024-06-06T07:00:58.531 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.530+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f97781abf80 con 0x7f9778102ef0 2024-06-06T07:00:58.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.532+0000 7f977c877700 1 -- 172.21.15.73:0/1583406752 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24816 (secure 0 0 0) 0x7f9760010380 con 0x7f9778102ef0 2024-06-06T07:00:58.533 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:00:58.533 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:58:53.383839+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 msgr2=0x7f97640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 0x7f97640a50e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9768009a10 tx=0x7f9768009190 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 msgr2=0x7f977810ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f976000b530 tx=0x7f976000b840 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 shutdown_connections 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f97640a2c20 0x7f97640a50e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f97781085d0 0x7f97781a3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9778102530 0x7f977810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 --2- 172.21.15.73:0/1583406752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9778102ef0 0x7f977810ead0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 >> 172.21.15.73:0/1583406752 conn(0x7f9778079640 msgr2=0x7f9778100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:00:58.535 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 shutdown_connections 2024-06-06T07:00:58.536 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:00:58.534+0000 7f977d879700 1 -- 172.21.15.73:0/1583406752 wait complete. 2024-06-06T07:00:59.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:58 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1583406752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:59.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:58 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1583406752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:59.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:58 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1583406752' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:00:59.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-06T06:57:09.432568+0000', 'modified': '2024-06-06T06:58:21.359605+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24433, 'mds_1': 24421, 'mds_2': 24419, 'mds_3': 24415, 'mds_4': 24404}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14529': {'gid': 14529, 'name': 'd', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6837/38418017', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 38418017}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 38418017}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14571': {'gid': 14571, 'name': 'g', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6839/2070842628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2070842628}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2070842628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14583': {'gid': 14583, 'name': 'a', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6841/552212635', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6840', 'nonce': 552212635}, {'type': 'v1', 'addr': '172.21.15.73:6841', 'nonce': 552212635}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24365': {'gid': 24365, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6833/464039391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6832', 'nonce': 464039391}, {'type': 'v1', 'addr': '172.21.15.136:6833', 'nonce': 464039391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24380': {'gid': 24380, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.136:6835/687158910', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6834', 'nonce': 687158910}, {'type': 'v1', 'addr': '172.21.15.136:6835', 'nonce': 687158910}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24404': {'gid': 24404, 'name': 'h', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.136:6837/450719145', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6836', 'nonce': 450719145}, {'type': 'v1', 'addr': '172.21.15.136:6837', 'nonce': 450719145}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 27, 'addr': '172.21.15.170:6835/3477948531', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6834', 'nonce': 3477948531}, {'type': 'v1', 'addr': '172.21.15.170:6835', 'nonce': 3477948531}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'b', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.136:6839/2417852679', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.136:6838', 'nonce': 2417852679}, {'type': 'v1', 'addr': '172.21.15.136:6839', 'nonce': 2417852679}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.170:6837/1210322990', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6836', 'nonce': 1210322990}, {'type': 'v1', 'addr': '172.21.15.170:6837', 'nonce': 1210322990}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.170:6839/3230829204', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.170:6838', 'nonce': 3230829204}, {'type': 'v1', 'addr': '172.21.15.170:6839', 'nonce': 3230829204}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24433, 'qdb_cluster': [24404, 24415, 24421, 24433, 24419]} 2024-06-06T07:00:59.172 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-06T07:00:59.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-06T07:00:59.413 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:00:59.816 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:00:59 smithi073 ceph-mon[32303]: pgmap v424: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:00.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:00:59 smithi136 ceph-mon[35901]: pgmap v424: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:00.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:00:59 smithi170 ceph-mon[36108]: pgmap v424: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 -- 172.21.15.73:0/3700382538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101fd0 msgr2=0x7f327c105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 --2- 172.21.15.73:0/3700382538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101fd0 0x7f327c105fd0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f326c0099c0 tx=0x7f326c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 -- 172.21.15.73:0/3700382538 shutdown_connections 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 --2- 172.21.15.73:0/3700382538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c101620 0x7f327c101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 --2- 172.21.15.73:0/3700382538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c1066b0 0x7f327c10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.143 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 --2- 172.21.15.73:0/3700382538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101fd0 0x7f327c105fd0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 -- 172.21.15.73:0/3700382538 >> 172.21.15.73:0/3700382538 conn(0x7f327c078360 msgr2=0x7f327c078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 -- 172.21.15.73:0/3700382538 shutdown_connections 2024-06-06T07:01:01.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.141+0000 7f328080a700 1 -- 172.21.15.73:0/3700382538 wait complete. 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 Processor -- start 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 -- start start 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c101fd0 0x7f327c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c1066b0 0x7f327c198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f327c1126a0 con 0x7f327c101620 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f327c112520 con 0x7f327c101fd0 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f328080a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f327c112820 con 0x7f327c1066b0 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55134/0 (socket says 172.21.15.73:55134) 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 -- 172.21.15.73:0/3823980399 learned_addr learned my addr 172.21.15.73:0/3823980399 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327ad9d700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c1066b0 0x7f327c198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 -- 172.21.15.73:0/3823980399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c1066b0 msgr2=0x7f327c198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c1066b0 0x7f327c198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f327a59c700 1 -- 172.21.15.73:0/3823980399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c101fd0 msgr2=0x7f327c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.142+0000 7f3279d9b700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c101fd0 0x7f327c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f327a59c700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c101fd0 0x7f327c19ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f327a59c700 1 -- 172.21.15.73:0/3823980399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f327c199160 con 0x7f327c101620 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f327a59c700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f326b7fe700 1 -- 172.21.15.73:0/3823980399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270017070 con 0x7f327c101620 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f326b7fe700 1 -- 172.21.15.73:0/3823980399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3270005020 con 0x7f327c101620 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f327c199450 con 0x7f327c101620 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f326b7fe700 1 -- 172.21.15.73:0/3823980399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3270009b50 con 0x7f327c101620 2024-06-06T07:01:01.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.143+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f327c1a5710 con 0x7f327c101620 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.145+0000 7f326b7fe700 1 -- 172.21.15.73:0/3823980399 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3270009cf0 con 0x7f327c101620 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.146+0000 7f326b7fe700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.146+0000 7f3279d9b700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.146+0000 7f326b7fe700 1 -- 172.21.15.73:0/3823980399 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f32700cf380 con 0x7f327c101620 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.146+0000 7f3279d9b700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 0x7f32640a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f326c006010 tx=0x7f326c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.146+0000 7f328b08db80 1 Processor -- start 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 -- start start 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a056aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a0631c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557659f44f10 con 0x55765a05c420 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557659fa3800 con 0x55765a056620 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f328b08db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557659f0b410 con 0x55765a056fe0 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55142/0 (socket says 172.21.15.73:55142) 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 -- 172.21.15.73:0/1735948720 learned_addr learned my addr 172.21.15.73:0/1735948720 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327ad9d700 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a0631c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f3279d9b700 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a056aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 -- 172.21.15.73:0/1735948720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 msgr2=0x55765a0631c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a0631c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 -- 172.21.15.73:0/1735948720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 msgr2=0x55765a056aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.153 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a056aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.147+0000 7f327a59c700 1 -- 172.21.15.73:0/1735948720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557659ef2880 con 0x55765a05c420 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f327a59c700 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3270097c70 tx=0x7f327009b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75ccd848dfe5a12e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f3268ff9700 1 -- 172.21.15.73:0/1735948720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327009fbc0 con 0x55765a05c420 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f3268ff9700 1 -- 172.21.15.73:0/1735948720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32700a4480 con 0x55765a05c420 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f3268ff9700 1 -- 172.21.15.73:0/1735948720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327009ba70 con 0x55765a05c420 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f328b08db80 1 -- 172.21.15.73:0/1735948720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 msgr2=0x55765a05c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.148+0000 7f328b08db80 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f3270097c70 tx=0x7f327009b480 comp rx=0 tx=0).stop 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- 172.21.15.73:0/1735948720 shutdown_connections 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a0631c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a05c800 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- 172.21.15.73:0/1735948720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a056aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.154 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- 172.21.15.73:0/1735948720 >> 172.21.15.73:0/1735948720 conn(0x557659f7f030 msgr2=0x55765a04d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- 172.21.15.73:0/1735948720 shutdown_connections 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- 172.21.15.73:0/1735948720 wait complete. 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- start start 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 Processor -- start 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a062ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 0x55765a0601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557659f43f70 con 0x55765a05c420 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557659f9f7c0 con 0x55765a056fe0 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55765a063700 con 0x55765a056620 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.149+0000 7f328b08db80 10 client.0 _ll_register_callbacks cb 0x557659f39560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3279d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 0x55765a0601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.155 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55152/0 (socket says 172.21.15.73:55152) 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 -- 172.21.15.73:0/2960187835 learned_addr learned my addr 172.21.15.73:0/2960187835 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3279d9b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 0x55765a0601c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:48242/0 (socket says 172.21.15.73:48242) 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327a59c700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a062ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 -- 172.21.15.73:0/2960187835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 msgr2=0x55765a062ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a062ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 -- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 msgr2=0x55765a0601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 0x55765a0601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 -- 172.21.15.73:0/2960187835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557659f69700 con 0x55765a05c420 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f327ad9d700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3274009f80 tx=0x7f3274004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb9e7fe2575afc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327401a070 con 0x55765a05c420 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32700a4480 con 0x55765a05c420 2024-06-06T07:01:01.156 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f3274007700 con 0x55765a05c420 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3274026050 con 0x55765a05c420 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f328b08db80 10 client.15180 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.150+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55765a06dea0 con 0x55765a05c420 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32740048d0 con 0x55765a05c420 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3274016ad0 con 0x55765a05c420 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 10 client.15180 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 20 client.15180 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0704b0 0x55765a071920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 4 client.15180 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f32700a4480 con 0x55765a0704b0 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.151+0000 7f327a59c700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0704b0 0x55765a071920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.152+0000 7f327a59c700 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0704b0 0x55765a071920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3d5b2d67e4981a4c server_cookie=6b90a5fa1cc09ed8 in_seq=0 out_seq=0 2024-06-06T07:01:01.158 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.152+0000 7f3261ffb700 10 client.15180 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.152+0000 7f3261ffb700 1 -- 172.21.15.73:0/2960187835 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f32700a4480 con 0x55765a0704b0 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.152+0000 7f3261ffb700 10 client.15180 handle_command_reply: tid=0 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.152+0000 7f328b08db80 1 client.15180 shutdown 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 msgr2=0x55765a060700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3274009f80 tx=0x7f3274004040 comp rx=0 tx=0).stop 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 shutdown_connections 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0704b0 0x55765a071920 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a062ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a060700 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 --2- 172.21.15.73:0/2960187835 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056fe0 0x55765a0601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 >> 172.21.15.73:0/2960187835 conn(0x557659f7f030 msgr2=0x55765a04c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f3261ffb700 0 client.15180 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:01:01.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 shutdown_connections 2024-06-06T07:01:01.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 1 -- 172.21.15.73:0/2960187835 wait complete. 2024-06-06T07:01:01.160 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.153+0000 7f328b08db80 20 client.15180 trim_cache size 0 max 16384 2024-06-06T07:01:01.176 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 Processor -- start 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 -- start start 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a06c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a04ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55765a06fc60 con 0x55765a05c420 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55765a070330 con 0x55765a056620 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f328b08db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55765a070a00 con 0x55765a056fe0 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55168/0 (socket says 172.21.15.73:55168) 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 -- 172.21.15.73:0/3075598418 learned_addr learned my addr 172.21.15.73:0/3075598418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327a59c700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a06c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f3279d9b700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a04ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 -- 172.21.15.73:0/3075598418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 msgr2=0x55765a04ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a04ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 -- 172.21.15.73:0/3075598418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 msgr2=0x55765a06c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a06c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327ad9d700 1 -- 172.21.15.73:0/3075598418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55765a04ddf0 con 0x55765a05c420 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f327a59c700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.174+0000 7f3279d9b700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a04ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f327ad9d700 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3274016ea0 tx=0x7f3274003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d53be9e14c1fe1e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f3263fff700 1 -- 172.21.15.73:0/3075598418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327401a070 con 0x55765a05c420 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f3263fff700 1 -- 172.21.15.73:0/3075598418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327401ee20 con 0x55765a05c420 2024-06-06T07:01:01.178 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f3263fff700 1 -- 172.21.15.73:0/3075598418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3274016430 con 0x55765a05c420 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 -- 172.21.15.73:0/3075598418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 msgr2=0x55765a04d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f3274016ea0 tx=0x7f3274003c50 comp rx=0 tx=0).stop 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 -- 172.21.15.73:0/3075598418 shutdown_connections 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a05c420 0x55765a04d370 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a056620 0x55765a06c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 --2- 172.21.15.73:0/3075598418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056fe0 0x55765a04ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 -- 172.21.15.73:0/3075598418 >> 172.21.15.73:0/3075598418 conn(0x557659f7f030 msgr2=0x55765a05eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 -- 172.21.15.73:0/3075598418 shutdown_connections 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.175+0000 7f328b08db80 1 -- 172.21.15.73:0/3075598418 wait complete. 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 -- start start 2024-06-06T07:01:01.182 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 Processor -- start 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a07ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a05c420 0x55765a075d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55765a0717a0 con 0x55765a056fe0 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55765a0710d0 con 0x55765a05c420 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55765a071e70 con 0x55765a056620 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f328b08db80 10 client.0 _ll_register_callbacks cb 0x557659f39560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55180/0 (socket says 172.21.15.73:55180) 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 -- 172.21.15.73:0/2762066037 learned_addr learned my addr 172.21.15.73:0/2762066037 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f327a59c700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a07ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 -- 172.21.15.73:0/2762066037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 msgr2=0x55765a07ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a07ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 -- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a05c420 msgr2=0x55765a075d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a05c420 0x55765a075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.176+0000 7f3279d9b700 1 -- 172.21.15.73:0/2762066037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55765a06ca50 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3279d9b700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f326c0099c0 tx=0x7f326c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14be391256314dc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f326c035060 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327401ee20 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f326c00bba0 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f326c03c050 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 10 client.15186 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55765a044fa0 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327401ee20 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f326c02f070 con 0x55765a056fe0 2024-06-06T07:01:01.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 10 client.15186 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 10 client.15186 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 20 client.15186 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0e0f00 0x55765a0dfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 4 client.15186 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.177+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "status"}) -- 0x557659fb0d00 con 0x55765a0e0f00 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f327a59c700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0e0f00 0x55765a0dfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f3261ffb700 10 client.15186 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f327a59c700 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0e0f00 0x55765a0dfc10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=40bfafacce4f5660 server_cookie=85e8af90471bc486 in_seq=0 out_seq=0 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f3261ffb700 1 -- 172.21.15.73:0/2762066037 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x557659fb0d00 con 0x55765a0e0f00 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f3261ffb700 10 client.15186 handle_command_reply: tid=0 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f328b08db80 1 client.15186 shutdown 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 msgr2=0x55765a07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.178+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f326c0099c0 tx=0x7f326c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.185 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 shutdown_connections 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55765a0e0f00 0x55765a0dfc10 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55765a056620 0x55765a07ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55765a05c420 0x55765a075d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 --2- 172.21.15.73:0/2762066037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55765a056fe0 0x55765a07b2e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 >> 172.21.15.73:0/2762066037 conn(0x557659f7f030 msgr2=0x55765a05b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f3261ffb700 0 client.15186 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 shutdown_connections 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 1 -- 172.21.15.73:0/2762066037 wait complete. 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328b08db80 20 client.15186 trim_cache size 0 max 16384 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 4, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24404, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 162.769583036, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 260.25307084799999 2024-06-06T07:01:01.187 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 msgr2=0x7f32640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 0x7f32640a50e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f326c006010 tx=0x7f326c000bc0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 msgr2=0x7f327c19e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f327000d980 tx=0x7f327000dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 shutdown_connections 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f32640a2c20 0x7f32640a50e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f327c1066b0 0x7f327c198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f327c101620 0x7f327c19e840 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 --2- 172.21.15.73:0/3823980399 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f327c101fd0 0x7f327c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:01.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.179+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 >> 172.21.15.73:0/3823980399 conn(0x7f327c078360 msgr2=0x7f327c0ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:01.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.180+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 shutdown_connections 2024-06-06T07:01:01.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:01.180+0000 7f328080a700 1 -- 172.21.15.73:0/3823980399 wait complete. 2024-06-06T07:01:01.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.f status 2024-06-06T07:01:02.139 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:02 smithi073 ceph-mon[32303]: pgmap v425: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:02 smithi136 ceph-mon[35901]: pgmap v425: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:02 smithi170 ceph-mon[36108]: pgmap v425: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:04.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 -- 172.21.15.73:0/3060985084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c1073f0 msgr2=0x7f513c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 --2- 172.21.15.73:0/3060985084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c1073f0 0x7f513c1077d0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f512c009990 tx=0x7f512c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 -- 172.21.15.73:0/3060985084 shutdown_connections 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 --2- 172.21.15.73:0/3060985084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 0x7f513c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 --2- 172.21.15.73:0/3060985084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c1073f0 0x7f513c1077d0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 --2- 172.21.15.73:0/3060985084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c101380 0x7f513c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 -- 172.21.15.73:0/3060985084 >> 172.21.15.73:0/3060985084 conn(0x7f513c078300 msgr2=0x7f513c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 -- 172.21.15.73:0/3060985084 shutdown_connections 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.012+0000 7f51415b7700 1 -- 172.21.15.73:0/3060985084 wait complete. 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 Processor -- start 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 -- start start 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 0x7f513c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 0x7f513c198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f513c111540 con 0x7f513c101380 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1113c0 con 0x7f513c1073f0 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f51415b7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f513c1116c0 con 0x7f513c101d40 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f513affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f513affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55186/0 (socket says 172.21.15.73:55186) 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.013+0000 7f513affd700 1 -- 172.21.15.73:0/779955076 learned_addr learned my addr 172.21.15.73:0/779955076 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f5133fff700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 0x7f513c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513b7fe700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 0x7f513c198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 -- 172.21.15.73:0/779955076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 msgr2=0x7f513c19ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 0x7f513c19ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 -- 172.21.15.73:0/779955076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 msgr2=0x7f513c198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 0x7f513c198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 -- 172.21.15.73:0/779955076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513c199090 con 0x7f513c101380 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513b7fe700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 0x7f513c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f513affd700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f512c000c00 tx=0x7f512c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f5138ff9700 1 -- 172.21.15.73:0/779955076 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c027070 con 0x7f513c101380 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f5138ff9700 1 -- 172.21.15.73:0/779955076 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f512c00bdc0 con 0x7f513c101380 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f5138ff9700 1 -- 172.21.15.73:0/779955076 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c022d30 con 0x7f513c101380 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513c199320 con 0x7f513c101380 2024-06-06T07:01:04.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.014+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f513c1a5640 con 0x7f513c101380 2024-06-06T07:01:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.017+0000 7f5138ff9700 1 -- 172.21.15.73:0/779955076 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f512c02a070 con 0x7f513c101380 2024-06-06T07:01:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f5138ff9700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 0x7f51280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f5133fff700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 0x7f51280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f5138ff9700 1 -- 172.21.15.73:0/779955076 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f512c0d00f0 con 0x7f513c101380 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f5133fff700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 0x7f51280a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f513c078870 tx=0x7f512400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f514be3ab80 1 Processor -- start 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.018+0000 7f514be3ab80 1 -- start start 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2143700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d213d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561d202bf10 con 0x5561d213db60 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561d208a800 con 0x5561d2143320 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f514be3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561d1ff2410 con 0x5561d2143c40 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55190/0 (socket says 172.21.15.73:55190) 2024-06-06T07:01:04.021 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 -- 172.21.15.73:0/509423001 learned_addr learned my addr 172.21.15.73:0/509423001 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f5133fff700 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d213d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513affd700 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2143700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 -- 172.21.15.73:0/509423001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 msgr2=0x5561d213d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d213d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 -- 172.21.15.73:0/509423001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 msgr2=0x5561d2143700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2143700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 -- 172.21.15.73:0/509423001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d1fd9880 con 0x5561d213db60 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.019+0000 7f513b7fe700 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5134009dd0 tx=0x7f51340077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99269bba06db9a67 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.020+0000 7f51317fa700 1 -- 172.21.15.73:0/509423001 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134004bb0 con 0x5561d213db60 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.020+0000 7f51317fa700 1 -- 172.21.15.73:0/509423001 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5134004d50 con 0x5561d213db60 2024-06-06T07:01:04.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.020+0000 7f51317fa700 1 -- 172.21.15.73:0/509423001 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134014670 con 0x5561d213db60 2024-06-06T07:01:04.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.020+0000 7f514be3ab80 1 -- 172.21.15.73:0/509423001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 msgr2=0x5561d213dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.020+0000 7f514be3ab80 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5134009dd0 tx=0x7f51340077e0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- 172.21.15.73:0/509423001 shutdown_connections 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d213dfa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2143700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- 172.21.15.73:0/509423001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d213d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- 172.21.15.73:0/509423001 >> 172.21.15.73:0/509423001 conn(0x5561d2066030 msgr2=0x5561d21341d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- 172.21.15.73:0/509423001 shutdown_connections 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- 172.21.15.73:0/509423001 wait complete. 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- start start 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 Processor -- start 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 0x5561d21498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d2147130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561d202af70 con 0x5561d2143320 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561d20867c0 con 0x5561d213db60 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561d214a630 con 0x5561d2143c40 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f514be3ab80 10 client.0 _ll_register_callbacks cb 0x5561d2020560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f5133fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f5133fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55202/0 (socket says 172.21.15.73:55202) 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.021+0000 7f5133fff700 1 -- 172.21.15.73:0/3975963432 learned_addr learned my addr 172.21.15.73:0/3975963432 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f513b7fe700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d2147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f513affd700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 0x5561d21498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 -- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 msgr2=0x5561d2147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d2147130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 -- 172.21.15.73:0/3975963432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 msgr2=0x5561d21498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 0x5561d21498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 -- 172.21.15.73:0/3975963432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d2050700 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f513affd700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 0x5561d21498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f5133fff700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f51240116f0 tx=0x7f51240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddad5883c86a9608 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f51227fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124028670 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f5124028810 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5134004d50 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5124031050 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f514be3ab80 10 client.15198 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d2154d70 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.022+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5134004d50 con 0x5561d2143320 2024-06-06T07:01:04.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f512401fa10 con 0x5561d2143320 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 10 client.15198 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 10 client.15198 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 20 client.15198 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21573e0 0x5561d2158850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 4 client.15198 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561d2097d00 con 0x5561d21573e0 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f513affd700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21573e0 0x5561d2158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f51227fc700 10 client.15198 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.023+0000 7f513affd700 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21573e0 0x5561d2158850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a9190881b0997493 server_cookie=6b3d52bde112db2b in_seq=0 out_seq=0 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f51227fc700 1 -- 172.21.15.73:0/3975963432 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561d2097d00 con 0x5561d21573e0 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f51227fc700 10 client.15198 handle_command_reply: tid=0 2024-06-06T07:01:04.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 client.15198 shutdown 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 msgr2=0x5561d2146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f51240116f0 tx=0x7f51240047c0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 shutdown_connections 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21573e0 0x5561d2158850 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143c40 0x5561d2147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f51227fc700 0 client.15198 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d213db60 0x5561d21498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 --2- 172.21.15.73:0/3975963432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143320 0x5561d2146bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 >> 172.21.15.73:0/3975963432 conn(0x5561d2066030 msgr2=0x5561d21336b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 shutdown_connections 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.024+0000 7f514be3ab80 1 -- 172.21.15.73:0/3975963432 wait complete. 2024-06-06T07:01:04.031 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.025+0000 7f514be3ab80 20 client.15198 trim_cache size 0 max 16384 2024-06-06T07:01:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.045+0000 7f514be3ab80 1 Processor -- start 2024-06-06T07:01:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.045+0000 7f514be3ab80 1 -- start start 2024-06-06T07:01:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143320 0x5561d2133e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143c40 0x5561d2134370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561d2156b90 con 0x5561d213db60 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561d2157260 con 0x5561d2143c40 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f514be3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561d2157930 con 0x5561d2143320 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55212/0 (socket says 172.21.15.73:55212) 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 -- 172.21.15.73:0/2564463422 learned_addr learned my addr 172.21.15.73:0/2564463422 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 -- 172.21.15.73:0/2564463422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143320 msgr2=0x5561d2133e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143320 0x5561d2133e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 -- 172.21.15.73:0/2564463422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143c40 msgr2=0x5561d2134370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143c40 0x5561d2134370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 -- 172.21.15.73:0/2564463422 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561d2135a00 con 0x5561d213db60 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f513affd700 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f512c009350 tx=0x7f512c022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=935a0ba7ac00779b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.046+0000 7f5130ff9700 1 -- 172.21.15.73:0/2564463422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c09c610 con 0x5561d213db60 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f5130ff9700 1 -- 172.21.15.73:0/2564463422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f512c098040 con 0x5561d213db60 2024-06-06T07:01:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f5130ff9700 1 -- 172.21.15.73:0/2564463422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512c09c870 con 0x5561d213db60 2024-06-06T07:01:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 -- 172.21.15.73:0/2564463422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 msgr2=0x5561d2147810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f512c009350 tx=0x7f512c022470 comp rx=0 tx=0).stop 2024-06-06T07:01:04.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 -- 172.21.15.73:0/2564463422 shutdown_connections 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143c40 0x5561d2134370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d213db60 0x5561d2147810 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 --2- 172.21.15.73:0/2564463422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d2143320 0x5561d2133e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 -- 172.21.15.73:0/2564463422 >> 172.21.15.73:0/2564463422 conn(0x5561d2066030 msgr2=0x5561d2149710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 -- 172.21.15.73:0/2564463422 shutdown_connections 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.047+0000 7f514be3ab80 1 -- 172.21.15.73:0/2564463422 wait complete. 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 -- start start 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 Processor -- start 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d213db60 0x5561d21620b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2133040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561d21586d0 con 0x5561d2143c40 2024-06-06T07:01:04.053 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561d2158000 con 0x5561d2143320 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561d2158da0 con 0x5561d213db60 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f514be3ab80 10 client.0 _ll_register_callbacks cb 0x5561d2020560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55216/0 (socket says 172.21.15.73:55216) 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 -- 172.21.15.73:0/1463798393 learned_addr learned my addr 172.21.15.73:0/1463798393 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f5133fff700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2133040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513affd700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d213db60 0x5561d21620b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 -- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d213db60 msgr2=0x5561d21620b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d213db60 0x5561d21620b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 -- 172.21.15.73:0/1463798393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 msgr2=0x5561d2133040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2133040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f513b7fe700 1 -- 172.21.15.73:0/1463798393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561d2133ac0 con 0x5561d2143c40 2024-06-06T07:01:04.054 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.048+0000 7f5133fff700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2133040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f513b7fe700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5134014b20 tx=0x7f5134014c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48c2b0afa8b23276 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134022070 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f5134027460 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f512c098040 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5134030050 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 10 client.15204 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561d21609a0 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f512c098040 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f513401e420 con 0x5561d2143c40 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 10 client.15204 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 10 client.15204 resolve_mds: resolved name 'f' to daemon mds.f 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 20 client.15204 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:04.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21c7190 0x5561d21c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 4 client.15204 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.049+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "status"}) -- 0x5561d2092280 con 0x5561d21c7190 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f513affd700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21c7190 0x5561d21c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f513affd700 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21c7190 0x5561d21c5ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8dbec26716a143b6 server_cookie=8337dbb58395b51b in_seq=0 out_seq=0 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f51227fc700 10 client.15204 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f51227fc700 1 -- 172.21.15.73:0/1463798393 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5561d2092280 con 0x5561d21c7190 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f51227fc700 10 client.15204 handle_command_reply: tid=0 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 client.15204 shutdown 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 msgr2=0x5561d2133580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f5134014b20 tx=0x7f5134014c00 comp rx=0 tx=0).stop 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 shutdown_connections 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5561d21c7190 0x5561d21c5ea0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561d213db60 0x5561d21620b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561d2143c40 0x5561d2133580 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 --2- 172.21.15.73:0/1463798393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561d2143320 0x5561d2133040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 >> 172.21.15.73:0/1463798393 conn(0x5561d2066030 msgr2=0x5561d2144020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.050+0000 7f51227fc700 0 client.15204 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 shutdown_connections 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f514be3ab80 1 -- 172.21.15.73:0/1463798393 wait complete. 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f514be3ab80 20 client.15204 trim_cache size 0 max 16384 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 3, 2024-06-06T07:01:04.057 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24415, 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 166.64629891600001, 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 268.675140253 2024-06-06T07:01:04.058 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 msgr2=0x7f51280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 0x7f51280a5160 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f513c078870 tx=0x7f512400b410 comp rx=0 tx=0).stop 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 msgr2=0x7f513c19e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f512c000c00 tx=0x7f512c003a80 comp rx=0 tx=0).stop 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 shutdown_connections 2024-06-06T07:01:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f513c1073f0 0x7f513c198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f513c101380 0x7f513c19e770 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f51280a2ca0 0x7f51280a5160 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 --2- 172.21.15.73:0/779955076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f513c101d40 0x7f513c19ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 >> 172.21.15.73:0/779955076 conn(0x7f513c078300 msgr2=0x7f513c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 shutdown_connections 2024-06-06T07:01:04.060 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:04.051+0000 7f51415b7700 1 -- 172.21.15.73:0/779955076 wait complete. 2024-06-06T07:01:04.739 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:04 smithi073 ceph-mon[32303]: pgmap v426: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:04.766 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-06T07:01:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:04 smithi136 ceph-mon[35901]: pgmap v426: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:04 smithi170 ceph-mon[36108]: pgmap v426: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:05.007 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:06.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:06 smithi073 ceph-mon[32303]: pgmap v427: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:01:06.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:06 smithi136 ceph-mon[35901]: pgmap v427: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:01:06.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:06 smithi170 ceph-mon[36108]: pgmap v427: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 -- 172.21.15.73:0/3031926822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070101df0 msgr2=0x7fc0701021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 --2- 172.21.15.73:0/3031926822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070101df0 0x7fc0701021d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0099c0 tx=0x7fc06c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 -- 172.21.15.73:0/3031926822 shutdown_connections 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 --2- 172.21.15.73:0/3031926822 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc070106750 0x7fc07010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 --2- 172.21.15.73:0/3031926822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070101df0 0x7fc0701021d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 --2- 172.21.15.73:0/3031926822 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc0701027a0 0x7fc070106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 -- 172.21.15.73:0/3031926822 >> 172.21.15.73:0/3031926822 conn(0x7fc070077a00 msgr2=0x7fc0700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 -- 172.21.15.73:0/3031926822 shutdown_connections 2024-06-06T07:01:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.945+0000 7fc0784e9700 1 -- 172.21.15.73:0/3031926822 wait complete. 2024-06-06T07:01:06.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 Processor -- start 2024-06-06T07:01:06.948 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 -- start start 2024-06-06T07:01:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc070101df0 0x7fc07019d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0701027a0 0x7fc07019dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc0701115a0 con 0x7fc070106750 2024-06-06T07:01:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc070111420 con 0x7fc070101df0 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc0784e9700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc070111720 con 0x7fc0701027a0 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc076a86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc076a86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38958/0 (socket says 172.21.15.73:38958) 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.946+0000 7fc076a86700 1 -- 172.21.15.73:0/2150252116 learned_addr learned my addr 172.21.15.73:0/2150252116 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076285700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc070101df0 0x7fc07019d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc075a84700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0701027a0 0x7fc07019dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 -- 172.21.15.73:0/2150252116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0701027a0 msgr2=0x7fc07019dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0701027a0 0x7fc07019dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 -- 172.21.15.73:0/2150252116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc070101df0 msgr2=0x7fc07019d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc070101df0 0x7fc07019d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 -- 172.21.15.73:0/2150252116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc070198020 con 0x7fc070106750 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc076a86700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b970 tx=0x7fc06800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc0677fe700 1 -- 172.21.15.73:0/2150252116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800d860 con 0x7fc070106750 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc070198310 con 0x7fc070106750 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc0677fe700 1 -- 172.21.15.73:0/2150252116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06800e430 con 0x7fc070106750 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc0677fe700 1 -- 172.21.15.73:0/2150252116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068017660 con 0x7fc070106750 2024-06-06T07:01:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.947+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc070100a80 con 0x7fc070106750 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.949+0000 7fc0677fe700 1 -- 172.21.15.73:0/2150252116 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc06800e5e0 con 0x7fc070106750 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.950+0000 7fc0677fe700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 0x7fc05c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.950+0000 7fc076285700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 0x7fc05c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.950+0000 7fc0677fe700 1 -- 172.21.15.73:0/2150252116 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc0680cef50 con 0x7fc070106750 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.950+0000 7fc076285700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 0x7fc05c0a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 Processor -- start 2024-06-06T07:01:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 -- start start 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x561959752990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x5619597532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619596a47c0 con 0x5619597585f0 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56195964c6d0 con 0x561959758f10 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619596a0780 con 0x561959752ed0 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38970/0 (socket says 172.21.15.73:38970) 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc076285700 1 -- 172.21.15.73:0/1046667686 learned_addr learned my addr 172.21.15.73:0/1046667686 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.951+0000 7fc075a84700 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x561959752990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 -- 172.21.15.73:0/1046667686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 msgr2=0x5619597532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x5619597532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 -- 172.21.15.73:0/1046667686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 msgr2=0x561959752990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x561959752990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 -- 172.21.15.73:0/1046667686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561959642f00 con 0x5619597585f0 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc076285700 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0099c0 tx=0x7fc06c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bf78fc5a44824 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc064d70700 1 -- 172.21.15.73:0/1046667686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00f840 con 0x5619597585f0 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc064d70700 1 -- 172.21.15.73:0/1046667686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06c024410 con 0x5619597585f0 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.952+0000 7fc064d70700 1 -- 172.21.15.73:0/1046667686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c02f600 con 0x5619597585f0 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- 172.21.15.73:0/1046667686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 msgr2=0x5619597589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0099c0 tx=0x7fc06c020000 comp rx=0 tx=0).stop 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- 172.21.15.73:0/1046667686 shutdown_connections 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x5619597532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x5619597589d0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/1046667686 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x561959752990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- 172.21.15.73:0/1046667686 >> 172.21.15.73:0/1046667686 conn(0x56195967dc70 msgr2=0x56195974d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- 172.21.15.73:0/1046667686 shutdown_connections 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- 172.21.15.73:0/1046667686 wait complete. 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 -- start start 2024-06-06T07:01:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.953+0000 7fc082d6cb80 1 Processor -- start 2024-06-06T07:01:06.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x56195975c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x56195975c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56195975f8c0 con 0x561959752ed0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561959644ba0 con 0x5619597585f0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56195975fa40 con 0x561959758f10 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 10 client.0 _ll_register_callbacks cb 0x56195963e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38972/0 (socket says 172.21.15.73:38972) 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 -- 172.21.15.73:0/3291416753 learned_addr learned my addr 172.21.15.73:0/3291416753 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc075a84700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x56195975c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076a86700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x56195975c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 -- 172.21.15.73:0/3291416753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 msgr2=0x56195975c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x56195975c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 -- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 msgr2=0x56195975c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x56195975c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 -- 172.21.15.73:0/3291416753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619595f3880 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076a86700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x56195975c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc076285700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc06c024db0 tx=0x7fc06c00fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808593bf0139c196 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c03c060 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc06c037400 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.954+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc06c024410 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc06c043050 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 10 client.15216 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56195976a2b0 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06c024410 con 0x561959752ed0 2024-06-06T07:01:06.962 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc06c02f400 con 0x561959752ed0 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 10 client.15216 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 20 client.15216 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56195976b600 0x56195976d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 4 client.15216 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.955+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619596b1cc0 con 0x56195976b600 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.956+0000 7fc075a84700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56195976b600 0x56195976d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.956+0000 7fc059ffb700 10 client.15216 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.956+0000 7fc075a84700 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56195976b600 0x56195976d7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=110f20f1ac3b6d75 server_cookie=5792c89b6d16efe0 in_seq=0 out_seq=0 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc059ffb700 1 -- 172.21.15.73:0/3291416753 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619596b1cc0 con 0x56195976b600 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc059ffb700 10 client.15216 handle_command_reply: tid=0 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 client.15216 shutdown 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 msgr2=0x56195975ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc06c024db0 tx=0x7fc06c00fac0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 shutdown_connections 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x56195976b600 0x56195976d7a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x56195975c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x56195975ed60 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc059ffb700 0 client.15216 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3291416753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x56195975c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 >> 172.21.15.73:0/3291416753 conn(0x56195967dc70 msgr2=0x561959751980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 shutdown_connections 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3291416753 wait complete. 2024-06-06T07:01:06.964 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.957+0000 7fc082d6cb80 20 client.15216 trim_cache size 0 max 16384 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.978+0000 7fc082d6cb80 1 Processor -- start 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.978+0000 7fc082d6cb80 1 -- start start 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x561959759c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x5619597474c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56195976bae0 con 0x561959752ed0 2024-06-06T07:01:06.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56195976c1b0 con 0x5619597585f0 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56195976c880 con 0x561959758f10 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38974/0 (socket says 172.21.15.73:38974) 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 -- 172.21.15.73:0/3902056292 learned_addr learned my addr 172.21.15.73:0/3902056292 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc075a84700 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x561959759c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 -- 172.21.15.73:0/3902056292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 msgr2=0x5619597474c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x5619597474c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 -- 172.21.15.73:0/3902056292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 msgr2=0x561959759c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.986 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x561959759c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 -- 172.21.15.73:0/3902056292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195974d330 con 0x561959752ed0 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc076285700 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0098e0 tx=0x7fc06c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f6a9c46a2cf461 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc05bfff700 1 -- 172.21.15.73:0/3902056292 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c03c040 con 0x561959752ed0 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc05bfff700 1 -- 172.21.15.73:0/3902056292 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06c024aa0 con 0x561959752ed0 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.979+0000 7fc05bfff700 1 -- 172.21.15.73:0/3902056292 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c02fc20 con 0x561959752ed0 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3902056292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 msgr2=0x561959752990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc06c0098e0 tx=0x7fc06c00bf10 comp rx=0 tx=0).stop 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3902056292 shutdown_connections 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959758f10 0x5619597474c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561959752ed0 0x561959752990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3902056292 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619597585f0 0x561959759c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3902056292 >> 172.21.15.73:0/3902056292 conn(0x56195967dc70 msgr2=0x561959746db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3902056292 shutdown_connections 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3902056292 wait complete. 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 -- start start 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.980+0000 7fc082d6cb80 1 Processor -- start 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x561959770930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x5619597713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56195976d620 con 0x5619597585f0 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56195976cf50 con 0x561959758f10 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56195976dcf0 con 0x561959752ed0 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 10 client.0 _ll_register_callbacks cb 0x56195963e8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:06.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38980/0 (socket says 172.21.15.73:38980) 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 -- 172.21.15.73:0/3639929117 learned_addr learned my addr 172.21.15.73:0/3639929117 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc076285700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x561959770930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc076a86700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x5619597713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 -- 172.21.15.73:0/3639929117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 msgr2=0x561959770930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x561959770930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 -- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 msgr2=0x5619597713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x5619597713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 -- 172.21.15.73:0/3639929117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561959775120 con 0x5619597585f0 2024-06-06T07:01:06.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc076a86700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x5619597713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc075a84700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56195975d630 tx=0x7fc060011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7480a1d5802ce38 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc059ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060004220 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fc0600043c0 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.981+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc06c024aa0 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc06002e050 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 10 client.15222 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5619597d7f70 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc06c024aa0 con 0x5619597585f0 2024-06-06T07:01:06.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc06001d6b0 con 0x5619597585f0 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 10 client.15222 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 10 client.15222 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 20 client.15222 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x5619597dcb20 0x5619597db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 4 client.15222 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "status"}) -- 0x5619596ac240 con 0x5619597dcb20 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.982+0000 7fc076285700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x5619597dcb20 0x5619597db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc059ffb700 10 client.15222 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc076285700 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x5619597dcb20 0x5619597db830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fd34f4c54aa7a81 server_cookie=14f820140823d8f1 in_seq=0 out_seq=0 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc059ffb700 1 -- 172.21.15.73:0/3639929117 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+328 (crc 0 0 0) 0x5619596ac240 con 0x5619597dcb20 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc059ffb700 10 client.15222 handle_command_reply: tid=0 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 client.15222 shutdown 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 msgr2=0x561959770e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.991 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x56195975d630 tx=0x7fc060011720 comp rx=0 tx=0).stop 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 shutdown_connections 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x5619597dcb20 0x5619597db830 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561959752ed0 0x561959770930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561959758f10 0x5619597713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc059ffb700 0 client.15222 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 --2- 172.21.15.73:0/3639929117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619597585f0 0x561959770e70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 >> 172.21.15.73:0/3639929117 conn(0x56195967dc70 msgr2=0x561959759a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.992 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 shutdown_connections 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 1 -- 172.21.15.73:0/3639929117 wait complete. 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.983+0000 7fc082d6cb80 20 client.15222 trim_cache size 0 max 16384 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 2, 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24419, 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:01:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 170.585437633, 2024-06-06T07:01:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:01:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:01:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:01:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 252.27763201400001 2024-06-06T07:01:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 msgr2=0x7fc05c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 0x7fc05c0a5160 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc06c009990 tx=0x7fc06c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 msgr2=0x7fc0701978c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc06800b970 tx=0x7fc06800bc80 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 shutdown_connections 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc070106750 0x7fc0701978c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc070101df0 0x7fc07019d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fc05c0a2ca0 0x7fc05c0a5160 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 --2- 172.21.15.73:0/2150252116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc0701027a0 0x7fc07019dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 >> 172.21.15.73:0/2150252116 conn(0x7fc070077a00 msgr2=0x7fc0700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:06.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 shutdown_connections 2024-06-06T07:01:06.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:06.984+0000 7fc0784e9700 1 -- 172.21.15.73:0/2150252116 wait complete. 2024-06-06T07:01:07.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-06T07:01:08.015 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:08.747 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:08 smithi073 ceph-mon[32303]: pgmap v428: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:08.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:08 smithi136 ceph-mon[35901]: pgmap v428: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:08.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:08 smithi170 ceph-mon[36108]: pgmap v428: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:09.703 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:09 smithi073 ceph-mon[32303]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- 172.21.15.73:0/3379642354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01009e0 msgr2=0x7f03c0100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 --2- 172.21.15.73:0/3379642354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01009e0 0x7f03c0100e60 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f03b00099c0 tx=0x7f03b0009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- 172.21.15.73:0/3379642354 shutdown_connections 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 --2- 172.21.15.73:0/3379642354 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c01013a0 0x7f03c010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 --2- 172.21.15.73:0/3379642354 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c0108560 0x7f03c0108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 --2- 172.21.15.73:0/3379642354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01009e0 0x7f03c0100e60 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- 172.21.15.73:0/3379642354 >> 172.21.15.73:0/3379642354 conn(0x7f03c0077990 msgr2=0x7f03c00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- 172.21.15.73:0/3379642354 shutdown_connections 2024-06-06T07:01:09.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- 172.21.15.73:0/3379642354 wait complete. 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 Processor -- start 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.713+0000 7f03c743b700 1 -- start start 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 0x7f03c019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c0108560 0x7f03c01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03c0112650 con 0x7f03c01013a0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01124d0 con 0x7f03c01009e0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03c01127d0 con 0x7f03c0108560 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38992/0 (socket says 172.21.15.73:38992) 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 -- 172.21.15.73:0/3765200191 learned_addr learned my addr 172.21.15.73:0/3765200191 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c59d8700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c0108560 0x7f03c01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 0x7f03c019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 -- 172.21.15.73:0/3765200191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c0108560 msgr2=0x7f03c01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c0108560 0x7f03c01989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 -- 172.21.15.73:0/3765200191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 msgr2=0x7f03c019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 0x7f03c019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 -- 172.21.15.73:0/3765200191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c0199110 con 0x7f03c01013a0 2024-06-06T07:01:09.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 0x7f03c019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009fd0 tx=0x7f03b00047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03b67fc700 1 -- 172.21.15.73:0/3765200191 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0027070 con 0x7f03c01013a0 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c01993a0 con 0x7f03c01013a0 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03b67fc700 1 -- 172.21.15.73:0/3765200191 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b0004050 con 0x7f03c01013a0 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.714+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c0101af0 con 0x7f03c01013a0 2024-06-06T07:01:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.715+0000 7f03b67fc700 1 -- 172.21.15.73:0/3765200191 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0022d80 con 0x7f03c01013a0 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.717+0000 7f03b67fc700 1 -- 172.21.15.73:0/3765200191 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f03b002a070 con 0x7f03c01013a0 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.717+0000 7f03b67fc700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 0x7f03ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.717+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 0x7f03ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.717+0000 7f03b67fc700 1 -- 172.21.15.73:0/3765200191 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f03b00d0350 con 0x7f03c01013a0 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 0x7f03ac0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005e20 tx=0x7f03bc005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 Processor -- start 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 -- start start 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f495f0 0x5652d1f499d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.720 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f43ed0 0x5652d1f442f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652d1e957c0 con 0x5652d1f49f10 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652d1e3d6d0 con 0x5652d1f495f0 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.718+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5652d1e91780 con 0x5652d1f43ed0 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38996/0 (socket says 172.21.15.73:38996) 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 -- 172.21.15.73:0/1638039596 learned_addr learned my addr 172.21.15.73:0/1638039596 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c51d7700 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f495f0 0x5652d1f499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c59d8700 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f43ed0 0x5652d1f442f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 -- 172.21.15.73:0/1638039596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f43ed0 msgr2=0x5652d1f442f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f43ed0 0x5652d1f442f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 -- 172.21.15.73:0/1638039596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f495f0 msgr2=0x5652d1f499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f495f0 0x5652d1f499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 -- 172.21.15.73:0/1638039596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d1e33f00 con 0x5652d1f49f10 2024-06-06T07:01:09.721 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03c49d6700 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03b009c920 tx=0x7f03b0005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e00abe7242a9eda server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03ab7fe700 1 -- 172.21.15.73:0/1638039596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b009c990 con 0x5652d1f49f10 2024-06-06T07:01:09.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03ab7fe700 1 -- 172.21.15.73:0/1638039596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b00a0d40 con 0x5652d1f49f10 2024-06-06T07:01:09.722 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.719+0000 7f03ab7fe700 1 -- 172.21.15.73:0/1638039596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b009cbf0 con 0x5652d1f49f10 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/1638039596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 msgr2=0x5652d1f43990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f03b009c920 tx=0x7f03b0005fb0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/1638039596 shutdown_connections 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f43ed0 0x5652d1f442f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f495f0 0x5652d1f499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/1638039596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f43990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/1638039596 >> 172.21.15.73:0/1638039596 conn(0x5652d1e6ec70 msgr2=0x5652d1f3e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/1638039596 shutdown_connections 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/1638039596 wait complete. 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 -- start start 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.720+0000 7f03d1cbeb80 1 Processor -- start 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f4fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.726 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f49f10 0x5652d1f4d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f506a0 con 0x5652d1f495f0 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652d1e35ba0 con 0x5652d1f43ed0 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f50820 con 0x5652d1f49f10 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03d1cbeb80 10 client.0 _ll_register_callbacks cb 0x5652d1e2f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39000/0 (socket says 172.21.15.73:39000) 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 -- 172.21.15.73:0/2980475786 learned_addr learned my addr 172.21.15.73:0/2980475786 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 -- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f49f10 msgr2=0x5652d1f4d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f49f10 0x5652d1f4d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 -- 172.21.15.73:0/2980475786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 msgr2=0x5652d1f4fc90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f4fc90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.727 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 -- 172.21.15.73:0/2980475786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d1de4880 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03c49d6700 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f03b0000c00 tx=0x7f03b00a5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e429076ef2b5a26 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.721+0000 7f03a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b00b9020 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f03b009cbf0 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b00a0d40 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03b009c990 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 10 client.15234 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652d1f506a0 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b00a0d40 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f03b009bf80 con 0x5652d1f495f0 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 10 client.15234 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 10 client.15234 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 20 client.15234 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:09.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1f5d280 0x5652d1f5e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 4 client.15234 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.722+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652d1ea2cc0 con 0x5652d1f5d280 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.723+0000 7f03c51d7700 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1f5d280 0x5652d1f5e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.723+0000 7f03c51d7700 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1f5d280 0x5652d1f5e6f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3023a13525cb6cff server_cookie=18037b1138fbe1d8 in_seq=0 out_seq=0 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.723+0000 7f03a8ff9700 10 client.15234 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03a8ff9700 1 -- 172.21.15.73:0/2980475786 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5652d1ea2cc0 con 0x5652d1f5d280 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03a8ff9700 10 client.15234 handle_command_reply: tid=0 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 client.15234 shutdown 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 msgr2=0x5652d1f4cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f03b0000c00 tx=0x7f03b00a5570 comp rx=0 tx=0).stop 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 shutdown_connections 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1f5d280 0x5652d1f5e6f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f49f10 0x5652d1f4d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f4fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2980475786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f495f0 0x5652d1f4cfe0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 >> 172.21.15.73:0/2980475786 conn(0x5652d1e6ec70 msgr2=0x5652d1f3e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03a8ff9700 0 client.15234 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 shutdown_connections 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2980475786 wait complete. 2024-06-06T07:01:09.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.724+0000 7f03d1cbeb80 20 client.15234 trim_cache size 0 max 16384 2024-06-06T07:01:09.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 Processor -- start 2024-06-06T07:01:09.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 -- start start 2024-06-06T07:01:09.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f43990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5ca30 con 0x5652d1f49f10 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5d100 con 0x5652d1f43ed0 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5d7d0 con 0x5652d1f495f0 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39004/0 (socket says 172.21.15.73:39004) 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 -- 172.21.15.73:0/2284523739 learned_addr learned my addr 172.21.15.73:0/2284523739 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c49d6700 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 -- 172.21.15.73:0/2284523739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 msgr2=0x5652d1f5cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f5cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 -- 172.21.15.73:0/2284523739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 msgr2=0x5652d1f43990 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f43990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 -- 172.21.15.73:0/2284523739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652d1f49cf0 con 0x5652d1f49f10 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.746+0000 7f03c59d8700 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03b8008740 tx=0x7f03b8008a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29f7bf97e8deb28 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03aaffd700 1 -- 172.21.15.73:0/2284523739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b8014750 con 0x5652d1f49f10 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03aaffd700 1 -- 172.21.15.73:0/2284523739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b8014da0 con 0x5652d1f49f10 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03aaffd700 1 -- 172.21.15.73:0/2284523739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b80153e0 con 0x5652d1f49f10 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2284523739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 msgr2=0x5652d1f4b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f03b8008740 tx=0x7f03b8008a50 comp rx=0 tx=0).stop 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2284523739 shutdown_connections 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f49f10 0x5652d1f4b8d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f43ed0 0x5652d1f43990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/2284523739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f5cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2284523739 >> 172.21.15.73:0/2284523739 conn(0x5652d1e6ec70 msgr2=0x5652d1f47070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2284523739 shutdown_connections 2024-06-06T07:01:09.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.747+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/2284523739 wait complete. 2024-06-06T07:01:09.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 -- start start 2024-06-06T07:01:09.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 Processor -- start 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f42980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f49f10 0x5652d1f42ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5e570 con 0x5652d1f43ed0 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5dea0 con 0x5652d1f49f10 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5652d1f5ec40 con 0x5652d1f495f0 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03d1cbeb80 10 client.0 _ll_register_callbacks cb 0x5652d1e2f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:09.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03c51d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03c51d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39016/0 (socket says 172.21.15.73:39016) 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03c51d7700 1 -- 172.21.15.73:0/3296853754 learned_addr learned my addr 172.21.15.73:0/3296853754 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.748+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f42980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 -- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 msgr2=0x5652d1f42980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f42980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 -- 172.21.15.73:0/3296853754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f49f10 msgr2=0x5652d1f42ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f49f10 0x5652d1f42ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 -- 172.21.15.73:0/3296853754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652d1f43400 con 0x5652d1f43ed0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03c51d7700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00a520 tx=0x7f03bc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e00ed6165bf4c2d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc01ed30 con 0x5652d1f43ed0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f03bc0252d0 con 0x5652d1f43ed0 2024-06-06T07:01:09.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b8014da0 con 0x5652d1f43ed0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f03bc035050 con 0x5652d1f43ed0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 10 client.15240 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652d1f668c0 con 0x5652d1f43ed0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03b8014da0 con 0x5652d1f43ed0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f03bc0105f0 con 0x5652d1f43ed0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 10 client.15240 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 10 client.15240 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 20 client.15240 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1fcd4d0 0x5652d1fcc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 4 client.15240 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.749+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "status"}) -- 0x5652d1e9d240 con 0x5652d1fcd4d0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1fcd4d0 0x5652d1fcc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:09.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03a8ff9700 10 client.15240 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03c49d6700 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1fcd4d0 0x5652d1fcc1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d83963adde9c4894 server_cookie=fda67bef838f380c in_seq=0 out_seq=0 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03a8ff9700 1 -- 172.21.15.73:0/3296853754 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x5652d1e9d240 con 0x5652d1fcd4d0 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03a8ff9700 10 client.15240 handle_command_reply: tid=0 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 client.15240 shutdown 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 msgr2=0x5652d1f67f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00a520 tx=0x7f03bc0047c0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 shutdown_connections 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x5652d1fcd4d0 0x5652d1fcc1e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5652d1f49f10 0x5652d1f42ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5652d1f43ed0 0x5652d1f67f40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 --2- 172.21.15.73:0/3296853754 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5652d1f495f0 0x5652d1f42980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.750+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 >> 172.21.15.73:0/3296853754 conn(0x5652d1e6ec70 msgr2=0x5652d1f4b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03a8ff9700 0 client.15240 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 shutdown_connections 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03d1cbeb80 1 -- 172.21.15.73:0/3296853754 wait complete. 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03d1cbeb80 20 client.15240 trim_cache size 0 max 16384 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 1, 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24421, 2024-06-06T07:01:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 174.358778521, 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 271.628160552 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:09.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 msgr2=0x7f03ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 0x7f03ac0a5160 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03bc005e20 tx=0x7f03bc005db0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 msgr2=0x7f03c019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009fd0 tx=0x7f03b00047b0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 shutdown_connections 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f03ac0a2ca0 0x7f03ac0a5160 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03c0108560 0x7f03c01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03c01009e0 0x7f03c019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 --2- 172.21.15.73:0/3765200191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03c01013a0 0x7f03c019ed30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.751+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 >> 172.21.15.73:0/3765200191 conn(0x7f03c0077990 msgr2=0x7f03c00ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.752+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 shutdown_connections 2024-06-06T07:01:09.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:09.752+0000 7f03c743b700 1 -- 172.21.15.73:0/3765200191 wait complete. 2024-06-06T07:01:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:09 smithi136 ceph-mon[35901]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:09 smithi170 ceph-mon[36108]: pgmap v429: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:01:10.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-06T07:01:10.817 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:12.564 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:12 smithi073 ceph-mon[32303]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:12.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 -- 172.21.15.73:0/3536130386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c107400 msgr2=0x7f299c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 --2- 172.21.15.73:0/3536130386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c107400 0x7f299c1077e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2984009990 tx=0x7f2984009ca0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 -- 172.21.15.73:0/3536130386 shutdown_connections 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 --2- 172.21.15.73:0/3536130386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c101d50 0x7f299c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 --2- 172.21.15.73:0/3536130386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c107400 0x7f299c1077e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 --2- 172.21.15.73:0/3536130386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 0x7f299c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 -- 172.21.15.73:0/3536130386 >> 172.21.15.73:0/3536130386 conn(0x7f299c078340 msgr2=0x7f299c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 -- 172.21.15.73:0/3536130386 shutdown_connections 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.556+0000 7f29a2313700 1 -- 172.21.15.73:0/3536130386 wait complete. 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 Processor -- start 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 -- start start 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 0x7f299c19d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 0x7f299c1978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f299c111550 con 0x7f299c101d50 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1113d0 con 0x7f299c107400 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.557+0000 7f29a2313700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f299c1116d0 con 0x7f299c101390 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39020/0 (socket says 172.21.15.73:39020) 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 -- 172.21.15.73:0/174212453 learned_addr learned my addr 172.21.15.73:0/174212453 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299bfff700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 0x7f299c19d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29a08b0700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 0x7f299c1978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 -- 172.21.15.73:0/174212453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 msgr2=0x7f299c19d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 0x7f299c19d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 -- 172.21.15.73:0/174212453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 msgr2=0x7f299c1978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 0x7f299c1978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 -- 172.21.15.73:0/174212453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c198010 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29a08b0700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 0x7f299c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f299b7fe700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d980 tx=0x7f298c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29997fa700 1 -- 172.21.15.73:0/174212453 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c017070 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c198300 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29997fa700 1 -- 172.21.15.73:0/174212453 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c005020 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29997fa700 1 -- 172.21.15.73:0/174212453 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c009b50 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.558+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1a45c0 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.560+0000 7f29997fa700 1 -- 172.21.15.73:0/174212453 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f298c009cf0 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.561+0000 7f29997fa700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 0x7f29880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.561+0000 7f29997fa700 1 -- 172.21.15.73:0/174212453 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f298c0cf380 con 0x7f299c101d50 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.561+0000 7f299bfff700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 0x7f29880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299bfff700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 0x7f29880a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f298400b5c0 tx=0x7f2984005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 Processor -- start 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 -- start start 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 0x556fad72b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad725b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fad617f10 con 0x556fad72b830 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556fad676800 con 0x556fad72af10 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f29acb96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556fad5de410 con 0x556fad725750 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39034/0 (socket says 172.21.15.73:39034) 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 -- 172.21.15.73:0/4012485971 learned_addr learned my addr 172.21.15.73:0/4012485971 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 -- 172.21.15.73:0/4012485971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 msgr2=0x556fad725b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299bfff700 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 0x556fad72b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad725b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 -- 172.21.15.73:0/4012485971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 msgr2=0x556fad72b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.562+0000 7f299b7fe700 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 0x556fad72b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f299b7fe700 1 -- 172.21.15.73:0/4012485971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fad5c5880 con 0x556fad72b830 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f299bfff700 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 0x556fad72b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f299b7fe700 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f298c097c70 tx=0x7f298c09b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b6e131f019b2438 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f29927fc700 1 -- 172.21.15.73:0/4012485971 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0a4480 con 0x556fad72b830 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f29927fc700 1 -- 172.21.15.73:0/4012485971 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c099b10 con 0x556fad72b830 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.563+0000 7f29927fc700 1 -- 172.21.15.73:0/4012485971 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c09bcd0 con 0x556fad72b830 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- 172.21.15.73:0/4012485971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 msgr2=0x556fad725210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f298c097c70 tx=0x7f298c09b480 comp rx=0 tx=0).stop 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- 172.21.15.73:0/4012485971 shutdown_connections 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad725b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72af10 0x556fad72b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 --2- 172.21.15.73:0/4012485971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad725210 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- 172.21.15.73:0/4012485971 >> 172.21.15.73:0/4012485971 conn(0x556fad652030 msgr2=0x556fad7201d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- 172.21.15.73:0/4012485971 shutdown_connections 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- 172.21.15.73:0/4012485971 wait complete. 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 -- start start 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.564+0000 7f29acb96b80 1 Processor -- start 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad72e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad72ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fad616f70 con 0x556fad725750 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556fad6727c0 con 0x556fad72b830 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556fad732210 con 0x556fad72af10 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 10 client.0 _ll_register_callbacks cb 0x556fad60c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:12.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39044/0 (socket says 172.21.15.73:39044) 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 -- 172.21.15.73:0/3288975352 learned_addr learned my addr 172.21.15.73:0/3288975352 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad72e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29a08b0700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad72ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 -- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 msgr2=0x556fad72e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad72e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 -- 172.21.15.73:0/3288975352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 msgr2=0x556fad72ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad72ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 -- 172.21.15.73:0/3288975352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fad63c700 con 0x556fad725750 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f299bfff700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29840036f0 tx=0x7f2984009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6823ee377d4af59c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f297b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2984034060 con 0x556fad725750 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f2984024610 con 0x556fad725750 2024-06-06T07:01:12.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.565+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c099b10 con 0x556fad725750 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f298403b050 con 0x556fad725750 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 10 client.15252 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fad73c9a0 con 0x556fad725750 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c099b10 con 0x556fad725750 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f298402f8b0 con 0x556fad725750 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 10 client.15252 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 10 client.15252 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 20 client.15252 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad73efa0 0x556fad740410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 4 client.15252 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fad683d00 con 0x556fad73efa0 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.566+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad73efa0 0x556fad740410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.567+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad73efa0 0x556fad740410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edef8c10dbeef61d server_cookie=bb73a49bdcac3690 in_seq=0 out_seq=0 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.567+0000 7f297b7fe700 10 client.15252 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f297b7fe700 1 -- 172.21.15.73:0/3288975352 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556fad683d00 con 0x556fad73efa0 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f297b7fe700 10 client.15252 handle_command_reply: tid=0 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 client.15252 shutdown 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 msgr2=0x556fad731580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f29840036f0 tx=0x7f2984009ca0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 shutdown_connections 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad73efa0 0x556fad740410 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad72ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad725750 0x556fad731580 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f297b7fe700 0 client.15252 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3288975352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad72e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 >> 172.21.15.73:0/3288975352 conn(0x556fad652030 msgr2=0x556fad71f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 shutdown_connections 2024-06-06T07:01:12.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 1 -- 172.21.15.73:0/3288975352 wait complete. 2024-06-06T07:01:12.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.568+0000 7f29acb96b80 20 client.15252 trim_cache size 0 max 16384 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 Processor -- start 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 -- start start 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad731130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad720370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fad73e750 con 0x556fad72af10 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556fad73ee20 con 0x556fad72b830 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f29acb96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556fad73f4f0 con 0x556fad725750 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.595+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39054/0 (socket says 172.21.15.73:39054) 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 -- 172.21.15.73:0/3209715496 learned_addr learned my addr 172.21.15.73:0/3209715496 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299bfff700 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad731130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f29a08b0700 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad720370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 -- 172.21.15.73:0/3209715496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 msgr2=0x556fad731130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad731130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 -- 172.21.15.73:0/3209715496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 msgr2=0x556fad720370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad720370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 -- 172.21.15.73:0/3209715496 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fad725550 con 0x556fad72af10 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f299b7fe700 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f298c09b7a0 tx=0x7f298c005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9310bf1f14a67e5b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f2991ffb700 1 -- 172.21.15.73:0/3209715496 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c0998b0 con 0x556fad72af10 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f2991ffb700 1 -- 172.21.15.73:0/3209715496 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c099b10 con 0x556fad72af10 2024-06-06T07:01:12.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.596+0000 7f2991ffb700 1 -- 172.21.15.73:0/3209715496 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298c086040 con 0x556fad72af10 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- 172.21.15.73:0/3209715496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 msgr2=0x556fad71fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f298c09b7a0 tx=0x7f298c005750 comp rx=0 tx=0).stop 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- 172.21.15.73:0/3209715496 shutdown_connections 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad725750 0x556fad731130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad72b830 0x556fad720370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 --2- 172.21.15.73:0/3209715496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72af10 0x556fad71fe30 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- 172.21.15.73:0/3209715496 >> 172.21.15.73:0/3209715496 conn(0x556fad652030 msgr2=0x556fad72d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- 172.21.15.73:0/3209715496 shutdown_connections 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- 172.21.15.73:0/3209715496 wait complete. 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.597+0000 7f29acb96b80 1 -- start start 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 Processor -- start 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad725750 0x556fad71f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad71faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556fad740290 con 0x556fad72b830 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556fad73fbc0 con 0x556fad725750 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556fad740960 con 0x556fad72af10 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29acb96b80 10 client.0 _ll_register_callbacks cb 0x556fad60c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39070/0 (socket says 172.21.15.73:39070) 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 -- 172.21.15.73:0/376649423 learned_addr learned my addr 172.21.15.73:0/376649423 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:12.603 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f299bfff700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad725750 0x556fad71f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f299b7fe700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad71faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 -- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 msgr2=0x556fad71faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad71faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 -- 172.21.15.73:0/376649423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad725750 msgr2=0x556fad71f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad725750 0x556fad71f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 -- 172.21.15.73:0/376649423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fad7ab910 con 0x556fad72b830 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f299b7fe700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad71faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f29a08b0700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29940077b0 tx=0x7f2994007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1aad938a88d66da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.598+0000 7f297b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994019020 con 0x556fad72b830 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f2994015600 con 0x556fad72b830 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298c099b10 con 0x556fad72b830 2024-06-06T07:01:12.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2994024050 con 0x556fad72b830 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 10 client.15258 fetch_fsmap learned FSMap version 29 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fad7acc50 con 0x556fad72b830 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f298c099b10 con 0x556fad72b830 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f299401d970 con 0x556fad72b830 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 10 client.15258 fetch_fsmap finished waiting for FSMap version 29 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 10 client.15258 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 20 client.15258 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad7b18b0 0x556fad7b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 4 client.15258 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "status"}) -- 0x556fad67e280 con 0x556fad7b18b0 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.599+0000 7f299bfff700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad7b18b0 0x556fad7b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f299bfff700 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad7b18b0 0x556fad7b05c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739ae401ad746bc2 server_cookie=93a5c38b00fd0a2c in_seq=0 out_seq=0 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f297b7fe700 10 client.15258 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:12.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f297b7fe700 1 -- 172.21.15.73:0/376649423 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x556fad67e280 con 0x556fad7b18b0 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f297b7fe700 10 client.15258 handle_command_reply: tid=0 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 client.15258 shutdown 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 msgr2=0x556fad71ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f29940077b0 tx=0x7f2994007ac0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 shutdown_connections 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556fad7b18b0 0x556fad7b05c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556fad72b830 0x556fad71ffe0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556fad725750 0x556fad71f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 --2- 172.21.15.73:0/376649423 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556fad72af10 0x556fad71faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 >> 172.21.15.73:0/376649423 conn(0x556fad652030 msgr2=0x556fad72ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f297b7fe700 0 client.15258 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 shutdown_connections 2024-06-06T07:01:12.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.600+0000 7f29acb96b80 1 -- 172.21.15.73:0/376649423 wait complete. 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29acb96b80 20 client.15258 trim_cache size 0 max 16384 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "b6cc6aaa-23d0-11ef-bc9b-c7b262605968", 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 24433, 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 243.05768560499999, 2024-06-06T07:01:12.607 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 29, 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 100, 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 98, 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 268.86979433300002 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 msgr2=0x7f29880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 0x7f29880a50e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f298400b5c0 tx=0x7f2984005960 comp rx=0 tx=0).stop 2024-06-06T07:01:12.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 msgr2=0x7f299c19dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d980 tx=0x7f298c00dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 shutdown_connections 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f29880a2c20 0x7f29880a50e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f299c101390 0x7f299c19d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f299c107400 0x7f299c1978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 --2- 172.21.15.73:0/174212453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f299c101d50 0x7f299c19dc30 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 >> 172.21.15.73:0/174212453 conn(0x7f299c078340 msgr2=0x7f299c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 shutdown_connections 2024-06-06T07:01:12.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:12.601+0000 7f29a2313700 1 -- 172.21.15.73:0/174212453 wait complete. 2024-06-06T07:01:12.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:12 smithi136 ceph-mon[35901]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:12.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:12 smithi170 ceph-mon[36108]: pgmap v430: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-06T07:01:13.256 INFO:teuthology.run_tasks:Running task sequential... 2024-06-06T07:01:13.271 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-06T07:01:13.271 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-06T07:01:13.271 INFO:teuthology.task.print:Enabling mgr modules 2024-06-06T07:01:13.271 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-06-06T07:01:13.272 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:01:13.272 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:01:13.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-06-06T07:01:13.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 -- 172.21.15.73:0/1340907409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 msgr2=0x7fabb4107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:13.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 --2- 172.21.15.73:0/1340907409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4107240 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7faba8009900 tx=0x7faba802a410 comp rx=0 tx=0).stop 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 -- 172.21.15.73:0/1340907409 shutdown_connections 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 --2- 172.21.15.73:0/1340907409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabb4107780 0x7fabb4109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 --2- 172.21.15.73:0/1340907409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4107240 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 --2- 172.21.15.73:0/1340907409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 -- 172.21.15.73:0/1340907409 >> 172.21.15.73:0/1340907409 conn(0x7fabb40fbf10 msgr2=0x7fabb40fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 -- 172.21.15.73:0/1340907409 shutdown_connections 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.377+0000 7fabba91b640 1 -- 172.21.15.73:0/1340907409 wait complete. 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 Processor -- start 2024-06-06T07:01:13.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 -- start start 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabb4107780 0x7fabb419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410d5b0 con 0x7fabb4104e50 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410d430 con 0x7fabb4102520 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabba91b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fabb410d730 con 0x7fabb4107780 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39084/0 (socket says 172.21.15.73:39084) 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 -- 172.21.15.73:0/2647155177 learned_addr learned my addr 172.21.15.73:0/2647155177 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:13.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb3fff640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 -- 172.21.15.73:0/2647155177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabb4107780 msgr2=0x7fabb419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabb4107780 0x7fabb419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 -- 172.21.15.73:0/2647155177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 msgr2=0x7fabb4197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb37fe640 1 -- 172.21.15.73:0/2647155177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabb419f7f0 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.378+0000 7fabb3fff640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabb37fe640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faba8009230 tx=0x7faba8004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba803f030 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabb419fa80 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faba8003cb0 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba8035070 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.379+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabb41a0040 con 0x7fabb4104e50 2024-06-06T07:01:13.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.380+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabb40fda00 con 0x7fabb4104e50 2024-06-06T07:01:13.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.381+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faba8042030 con 0x7fabb4104e50 2024-06-06T07:01:13.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.381+0000 7fabb17fa640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 0x7fab9c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:13.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.382+0000 7fabb3fff640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 0x7fab9c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:13.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.382+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7faba80efb60 con 0x7fabb4104e50 2024-06-06T07:01:13.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.382+0000 7fabb3fff640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 0x7fab9c0a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faba40058c0 tx=0x7faba4005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:13.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.386+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faba80bc6f0 con 0x7fabb4104e50 2024-06-06T07:01:13.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:13.613+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fabb40625a0 con 0x7fabb4104e50 2024-06-06T07:01:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.422+0000 7fabb17fa640 1 -- 172.21.15.73:0/2647155177 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7faba80bbff0 con 0x7fabb4104e50 2024-06-06T07:01:14.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 msgr2=0x7fab9c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:14.426 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 0x7fab9c0a50f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7faba40058c0 tx=0x7faba4005d30 comp rx=0 tx=0).stop 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 msgr2=0x7fabb4198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7faba8009230 tx=0x7faba8004340 comp rx=0 tx=0).stop 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 shutdown_connections 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fab9c0a2c30 0x7fab9c0a50f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fabb4107780 0x7fabb419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:14.427 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fabb4104e50 0x7fabb4198110 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:14.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 --2- 172.21.15.73:0/2647155177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fabb4102520 0x7fabb4197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:14.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 >> 172.21.15.73:0/2647155177 conn(0x7fabb40fbf10 msgr2=0x7fabb4102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:14.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.426+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 shutdown_connections 2024-06-06T07:01:14.428 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:14.427+0000 7fabba91b640 1 -- 172.21.15.73:0/2647155177 wait complete. 2024-06-06T07:01:14.445 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-06-06T07:01:14.456 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-06-06T07:01:14.456 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-06-06T07:01:14.457 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-06-06T07:01:14.457 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-06T07:01:14.457 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:14.457 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:14.457 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-06-06T07:01:14.475 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-06-06T07:01:14.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:14.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-06-06T07:01:14.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:14.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-06-06T07:01:14.651 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-06-06T07:01:14.652 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-06T07:01:14.676 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-mon[32303]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:14.676 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:01:14.676 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setuser ceph since I am not root 2024-06-06T07:01:14.676 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: ignoring --setgroup ceph since I am not root 2024-06-06T07:01:14.676 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:14.617+0000 7f990f790200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-mon[35901]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: ignoring --setuser ceph since I am not root 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: ignoring --setgroup ceph since I am not root 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-mgr[37266]: -- 172.21.15.136:0/791750289 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3a7c3c5a0 con 0x55c3a7c11000 2024-06-06T07:01:14.794 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:14.606+0000 7f98a4910200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:01:14.795 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:14.792+0000 7f98a4910200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:14.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-mon[36108]: pgmap v431: 129 pgs: 129 active+clean; 646 KiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-06T07:01:14.796 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-06-06T07:01:14.796 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: ignoring --setuser ceph since I am not root 2024-06-06T07:01:14.797 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: ignoring --setgroup ceph since I am not root 2024-06-06T07:01:14.797 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-mgr[37944]: -- 172.21.15.170:0/2795146083 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e93b9fe5a0 con 0x55e93b9d2c00 2024-06-06T07:01:14.797 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:14.607+0000 7f60ea4c2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-06T07:01:14.797 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:14.794+0000 7f60ea4c2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:14.926 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:14.948 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:14.819+0000 7f990f790200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:14.948 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:14.947+0000 7f990f790200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:01:15.051 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:14.894+0000 7f98a4910200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:01:15.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:14.897+0000 7f60ea4c2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-06T07:01:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:15 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:01:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:15 smithi136 ceph-mon[35901]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:01:15.552 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:15.203+0000 7f98a4910200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:01:15.552 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:15.298+0000 7f98a4910200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:01:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:15 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:01:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:15 smithi170 ceph-mon[36108]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:01:15.597 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:15 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:15.207+0000 7f60ea4c2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:01:15.597 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:15 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:15.293+0000 7f60ea4c2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:01:15.728 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:15 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2647155177' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-06-06T07:01:15.729 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:15 smithi073 ceph-mon[32303]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-06-06T07:01:15.729 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:15.275+0000 7f990f790200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-06T07:01:15.729 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:15.369+0000 7f990f790200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-06T07:01:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 -- 172.21.15.73:0/1417681840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101d50 msgr2=0x7f5a5810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1417681840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101d50 0x7f5a5810e240 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000b0a0 tx=0x7f5a5000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 -- 172.21.15.73:0/1417681840 shutdown_connections 2024-06-06T07:01:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1417681840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58107400 0x7f5a581077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1417681840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101d50 0x7f5a5810e240 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1417681840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58101390 0x7f5a58101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 -- 172.21.15.73:0/1417681840 >> 172.21.15.73:0/1417681840 conn(0x7f5a58078340 msgr2=0x7f5a58078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 -- 172.21.15.73:0/1417681840 shutdown_connections 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 -- 172.21.15.73:0/1417681840 wait complete. 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.726+0000 7f5a5c983700 1 Processor -- start 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 -- start start 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58101d50 0x7f5a58199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58107400 0x7f5a581a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5a58111540 con 0x7f5a58101390 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5a581113c0 con 0x7f5a58107400 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5c983700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5a581116c0 con 0x7f5a58101d50 2024-06-06T07:01:16.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59262/0 (socket says 172.21.15.73:59262) 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 -- 172.21.15.73:0/1214963115 learned_addr learned my addr 172.21.15.73:0/1214963115 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a55d9b700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58101d50 0x7f5a58199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a56d9d700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58107400 0x7f5a581a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58101d50 msgr2=0x7f5a58199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58101d50 0x7f5a58199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58107400 msgr2=0x7f5a581a4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58107400 0x7f5a581a4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 -- 172.21.15.73:0/1214963115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a581a4f60 con 0x7f5a58101390 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.727+0000 7f5a5659c700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800d980 tx=0x7f5a4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.728+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a48017070 con 0x7f5a58101390 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.728+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a48005020 con 0x7f5a58101390 2024-06-06T07:01:16.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.728+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a4800fa90 con 0x7f5a58101390 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.728+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a581a5190 con 0x7f5a58101390 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.728+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a581a56d0 con 0x7f5a58101390 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.729+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a58069180 con 0x7f5a58101390 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.730+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f5a480051d0 con 0x7f5a58101390 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.731+0000 7f5a477fe700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 0x7f5a400a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:16.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.731+0000 7f5a55d9b700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 msgr2=0x7f5a400a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:16.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.731+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f5a480cf1f0 con 0x7f5a58101390 2024-06-06T07:01:16.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.731+0000 7f5a55d9b700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 0x7f5a400a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:16.736 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.736+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5a48020020 con 0x7f5a58101390 2024-06-06T07:01:16.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:16.635+0000 7f990f790200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:01:16.801 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:16.506+0000 7f98a4910200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:01:16.801 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:16.675+0000 7f98a4910200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:16.837 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:16.492+0000 7f60ea4c2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-06T07:01:16.838 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:16.664+0000 7f60ea4c2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:16.932 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.932+0000 7f5a55d9b700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 msgr2=0x7f5a400a51b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:16.933 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.932+0000 7f5a55d9b700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 0x7f5a400a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:01:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.978+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f5a5802dca0 con 0x7f5a58101390 2024-06-06T07:01:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.979+0000 7f5a477fe700 1 -- 172.21.15.73:0/1214963115 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f5a4809bd80 con 0x7f5a58101390 2024-06-06T07:01:16.980 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-06-06T07:01:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 msgr2=0x7f5a400a51b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 0x7f5a400a51b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 msgr2=0x7f5a581999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:16.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5a4800d980 tx=0x7f5a4800dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 shutdown_connections 2024-06-06T07:01:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5a58107400 0x7f5a581a4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a58101390 0x7f5a581999a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f5a400a2cf0 0x7f5a400a51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.983 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 --2- 172.21.15.73:0/1214963115 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5a58101d50 0x7f5a58199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 >> 172.21.15.73:0/1214963115 conn(0x7f5a58078340 msgr2=0x7f5a580fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 shutdown_connections 2024-06-06T07:01:16.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:16.981+0000 7f5a5c983700 1 -- 172.21.15.73:0/1214963115 wait complete. 2024-06-06T07:01:17.031 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:16.816+0000 7f990f790200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:17.031 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:16.989+0000 7f990f790200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:01:17.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:16.837+0000 7f60ea4c2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:01:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:17 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1214963115' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:01:17.140 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:17 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1214963115' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:01:17.140 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:16.847+0000 7f98a4910200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-06T07:01:17.289 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:17 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1214963115' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-06T07:01:17.428 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:17.130+0000 7f60ea4c2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:01:17.428 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:17.223+0000 7f60ea4c2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:01:17.438 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:17.138+0000 7f98a4910200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:01:17.439 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:17.232+0000 7f98a4910200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:01:17.439 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:17.437+0000 7f98a4910200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:01:17.589 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:17.288+0000 7f990f790200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-06T07:01:17.590 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:17.383+0000 7f990f790200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-06T07:01:17.590 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:17.588+0000 7f990f790200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-06T07:01:17.748 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-06T07:01:17.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:17.749 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2024-06-06T07:01:17.801 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:17.631+0000 7f98a4910200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2024-06-06T07:01:17.806 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42005sec preferred_lft 42005sec 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2024-06-06T07:01:17.807 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2024-06-06T07:01:17.808 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-06T07:01:17.808 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T07:01:17.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:17.427+0000 7f60ea4c2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-06T07:01:17.847 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:17.624+0000 7f60ea4c2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:17.896 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:01:17.906 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:17.790+0000 7f990f790200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-06T07:01:17.945 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:17.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:17.952 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-06T07:01:17.978 INFO:teuthology.orchestra.run.smithi073.stdout:1 2024-06-06T07:01:17.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:17.979 DEBUG:teuthology.orchestra.run.smithi073:> route 2024-06-06T07:01:18.037 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2024-06-06T07:01:18.037 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-06T07:01:18.037 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-06-06T07:01:18.037 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-06-06T07:01:18.037 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-06T07:01:18.037 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:01:18.038 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T07:01:18.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-06-06T07:01:18.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-06-06T07:01:18.038 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-06-06T07:01:18.038 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T07:01:18.127 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:01:18.161 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:18.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:18.168 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-06-06T07:01:18.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:18.185 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2024-06-06T07:01:18.241 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:01:18.241 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T07:01:18.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:01:18.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-06T07:01:18.242 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T07:01:18.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:01:18.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:18.364 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-06T07:01:18.364 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T07:01:18.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:01:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:18.483 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-06T07:01:18.483 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2024-06-06T07:01:18.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2024-06-06T07:01:18.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2024-06-06T07:01:18.483 DEBUG:teuthology.orchestra.run.smithi073:> ') 2024-06-06T07:01:18.532 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-06T07:01:18.551 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:18.235+0000 7f98a4910200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:01:18.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:18.561 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-06-06T07:01:18.562 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:01:18.562 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:18.577 INFO:teuthology.orchestra.run.smithi073.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-06T07:01:18.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-06T07:01:18.577 DEBUG:teuthology.orchestra.run.smithi073:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:18.597 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:18.231+0000 7f60ea4c2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:01:18.633 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-06-06T07:01:18.695 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:18.446+0000 7f990f790200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-06T07:01:18.699 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-06-06T07:01:18.756 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T07:01:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/10bdc6c9845f3c7e810a1f43fac4fbdc3c7fc347cceee132a6dd3766b6f20e35/merged 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/872647e4eceff223a218675b1b1a57cacdbe7476433ffbbed2a09dde2dc380c0/merged 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/0 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:01:18.758 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/5f122e402711f91bdb948a37ab4427a2cb160c4613281add4823ce6c3f234454/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/f8301984a014f1ee316b3ef130c4695a127a82eaa5b4327a54a4855b36c128a3/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9dfdb2a592c6db3b58ddb437ebca3591698c75d0f78a4b70557fb56576cd263b/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c0d0882dec388bae6e209ba50dc3afda1427df6cc7348813220120d496780c3e/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ca0b6a041d93f68e6a6e98e8435cab60a48e6a12c5c2be13727794df338edf8/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9d8a3b5d1f0f53e754c3a2710ff933b5c10bda06579b354b33699c61590da80a/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ed2ab3935eea85dcbf9bbf5450eae68da9f481f751b2747095bd1cff9de0e97/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c9c41226ab9529da15d5d56ce6671e5de43eecc0d2b8db8db6fec85947d1bb95/merged 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:01:18.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:18.760 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-06-06T07:01:18.812 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-06T07:01:18.812 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d --client_fs=cephfs) 2024-06-06T07:01:18.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo modprobe fuse 2024-06-06T07:01:18.879 DEBUG:teuthology.orchestra.run.smithi073:> cat /proc/self/mounts | awk '{print $2}' 2024-06-06T07:01:18.935 INFO:teuthology.orchestra.run.smithi073.stdout:/proc 2024-06-06T07:01:18.935 INFO:teuthology.orchestra.run.smithi073.stdout:/sys 2024-06-06T07:01:18.935 INFO:teuthology.orchestra.run.smithi073.stdout:/dev 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/security 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/shm 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/pts 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/run 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/cgroup 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/pstore 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/bpf 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/ 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/selinux 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/tracing 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/mqueue 2024-06-06T07:01:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/debug 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/dev/hugepages 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/fs/fuse/connections 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-sysctl.service 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/sys/kernel/config 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/nfs/rpc_pipefs 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/1000 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/ceph 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/10bdc6c9845f3c7e810a1f43fac4fbdc3c7fc347cceee132a6dd3766b6f20e35/merged 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/872647e4eceff223a218675b1b1a57cacdbe7476433ffbbed2a09dde2dc380c0/merged 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/run/user/0 2024-06-06T07:01:18.937 INFO:teuthology.orchestra.run.smithi073.stdout:/proc/sys/fs/binfmt_misc 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/5f122e402711f91bdb948a37ab4427a2cb160c4613281add4823ce6c3f234454/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/f8301984a014f1ee316b3ef130c4695a127a82eaa5b4327a54a4855b36c128a3/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9dfdb2a592c6db3b58ddb437ebca3591698c75d0f78a4b70557fb56576cd263b/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c0d0882dec388bae6e209ba50dc3afda1427df6cc7348813220120d496780c3e/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ca0b6a041d93f68e6a6e98e8435cab60a48e6a12c5c2be13727794df338edf8/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/9d8a3b5d1f0f53e754c3a2710ff933b5c10bda06579b354b33699c61590da80a/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/0ed2ab3935eea85dcbf9bbf5450eae68da9f481f751b2747095bd1cff9de0e97/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/var/lib/containers/storage/overlay/c9c41226ab9529da15d5d56ce6671e5de43eecc0d2b8db8db6fec85947d1bb95/merged 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:01:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:01:18.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:18.939 DEBUG:teuthology.orchestra.run.smithi073:> ls /sys/fs/fuse/connections 2024-06-06T07:01:18.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi073.stderr:2024-06-06T07:01:18.937+0000 7feac3bd1480 -1 init, newargv = 0x560b9ebb1830 newargc=15 2024-06-06T07:01:18.940 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi073.stderr:ceph-fuse[91947]: starting ceph client 2024-06-06T07:01:18.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi073.stderr:ceph-fuse[91947]: starting fuse 2024-06-06T07:01:18.963 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:18.657+0000 7f60ea4c2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:01:18.963 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:18.756+0000 7f60ea4c2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:01:18.964 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:18.661+0000 7f98a4910200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:01:18.964 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:18.759+0000 7f98a4910200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:01:18.964 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:18.962+0000 7f98a4910200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:01:18.975 INFO:teuthology.orchestra.run.smithi073.stdout:119 2024-06-06T07:01:18.975 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-06T07:01:18.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo stdin-killer -- python3 -c ' 2024-06-06T07:01:18.976 DEBUG:teuthology.orchestra.run.smithi073:> import glob 2024-06-06T07:01:18.976 DEBUG:teuthology.orchestra.run.smithi073:> import re 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> import os 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> import subprocess 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> def _find_admin_socket(client_name): 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> files = glob.glob(asok_path) 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> # Given a non-glob path, it better be there 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> if "*" not in asok_path: 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> assert(len(files) == 1) 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> return files[0] 2024-06-06T07:01:18.977 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> for f in files: 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> contents = proc_f.read() 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> if mountpoint in contents: 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> return f 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> print(_find_admin_socket("client.0")) 2024-06-06T07:01:18.978 DEBUG:teuthology.orchestra.run.smithi073:> ' 2024-06-06T07:01:19.009 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:18.898+0000 7f990f790200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-06T07:01:19.009 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:19.008+0000 7f990f790200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-06T07:01:19.088 INFO:teuthology.orchestra.run.smithi073.stdout:/var/run/ceph/ceph-client.0.91947.asok 2024-06-06T07:01:19.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-06T07:01:19.099 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.91947.asok 2024-06-06T07:01:19.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:19.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.91947.asok status 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "metadata": { 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "entity_id": "0", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": "smithi073", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "pid": "91947", 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "root": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d" 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:01:19.237 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_count": 0, 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "dentry_pinned_count": 0, 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 15294, 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "inst": { 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "name": { 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "client", 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "num": 15294 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1452578725 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:01:19.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": { 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "192.168.0.1:0", 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1452578725 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "inst_str": "client.15294 192.168.0.1:0/1452578725", 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "addr_str": "192.168.0.1:0/1452578725", 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "inode_count": 5, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_epoch": 29, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch": 100, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_epoch_barrier": 0, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "blocklisted": false, 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs" 2024-06-06T07:01:19.239 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:19.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:19.247 DEBUG:teuthology.orchestra.run.smithi073:> stat --file-system '--printf=%T 2024-06-06T07:01:19.247 DEBUG:teuthology.orchestra.run.smithi073:> ' -- /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:19.262 INFO:teuthology.orchestra.run.smithi073.stdout:fuseblk 2024-06-06T07:01:19.263 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:19.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:01:19.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:19.274 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:19.222+0000 7f990f790200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:01:19.302 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T07:01:19.311 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:01:19.311 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:01:19.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-06T07:01:19.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:18.962+0000 7f60ea4c2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-06T07:01:19.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 -- 172.21.15.73:0/1444284317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4107740 msgr2=0x7f2ec4109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/1444284317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4107740 0x7f2ec4109b30 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc009ca0 tx=0x7f2ebc02a8c0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 -- 172.21.15.73:0/1444284317 shutdown_connections 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/1444284317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4107740 0x7f2ec4109b30 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/1444284317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4104e10 0x7f2ec4107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/1444284317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 0x7f2ec41048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 -- 172.21.15.73:0/1444284317 >> 172.21.15.73:0/1444284317 conn(0x7f2ec40fbe70 msgr2=0x7f2ec40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 -- 172.21.15.73:0/1444284317 shutdown_connections 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.447+0000 7f2ec8a38640 1 -- 172.21.15.73:0/1444284317 wait complete. 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 Processor -- start 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 -- start start 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 0x7f2ec4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4107740 0x7f2ec419f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410d530 con 0x7f2ec4104e10 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410d3b0 con 0x7f2ec4107740 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec8a38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2ec410d6b0 con 0x7f2ec41024e0 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59278/0 (socket says 172.21.15.73:59278) 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.448+0000 7f2ec1d9b640 1 -- 172.21.15.73:0/3026251669 learned_addr learned my addr 172.21.15.73:0/3026251669 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec259c640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 0x7f2ec4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec2d9d640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4107740 0x7f2ec419f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 msgr2=0x7f2ec4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 0x7f2ec4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4107740 msgr2=0x7f2ec419f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4107740 0x7f2ec419f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 -- 172.21.15.73:0/3026251669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ec419f790 con 0x7f2ec4104e10 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec1d9b640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9c0 tx=0x7f2eb800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb800cd80 con 0x7f2ec4104e10 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ec419fa80 con 0x7f2ec4104e10 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2eb80054d0 con 0x7f2ec4104e10 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8010430 con 0x7f2ec4104e10 2024-06-06T07:01:19.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.449+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ec41a0020 con 0x7f2ec4104e10 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.450+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec40fd960 con 0x7f2ec4104e10 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.452+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f2eb8004fb0 con 0x7f2ec4104e10 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.452+0000 7f2eb37fe640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 0x7f2eac0a5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.452+0000 7f2ec259c640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 msgr2=0x7f2eac0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.452+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f2eb80cfa80 con 0x7f2ec4104e10 2024-06-06T07:01:19.454 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.452+0000 7f2ec259c640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 0x7f2eac0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:19.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.456+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2eb8053850 con 0x7f2ec4104e10 2024-06-06T07:01:19.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.653+0000 7f2ec259c640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 msgr2=0x7f2eac0a5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:19.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.653+0000 7f2ec259c640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 0x7f2eac0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-06T07:01:19.680 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.679+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f2ec402d2e0 con 0x7f2ec4104e10 2024-06-06T07:01:19.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.731+0000 7f2eb37fe640 1 -- 172.21.15.73:0/3026251669 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v30) ==== 117+0+0 (secure 0 0 0) 0x7f2eb809c610 con 0x7f2ec4104e10 2024-06-06T07:01:19.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 msgr2=0x7f2eac0a5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:19.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 0x7f2eac0a5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 msgr2=0x7f2ec41980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e9c0 tx=0x7f2eb800ee80 comp rx=0 tx=0).stop 2024-06-06T07:01:19.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 shutdown_connections 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2eac0a2c80 0x7f2eac0a5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2ec4107740 0x7f2ec419f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ec4104e10 0x7f2ec41980b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 --2- 172.21.15.73:0/3026251669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2ec41024e0 0x7f2ec4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 >> 172.21.15.73:0/3026251669 conn(0x7f2ec40fbe70 msgr2=0x7f2ec4102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 shutdown_connections 2024-06-06T07:01:19.735 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.733+0000 7f2ec8a38640 1 -- 172.21.15.73:0/3026251669 wait complete. 2024-06-06T07:01:19.746 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-06T07:01:19.757 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-06T07:01:19.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T07:01:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:19 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T07:01:19.801 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:19.497+0000 7f98a4910200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:01:19.801 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:19.590+0000 7f98a4910200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:01:19.832 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:19 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T07:01:19.832 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:19.495+0000 7f60ea4c2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:01:19.832 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:19.586+0000 7f60ea4c2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:01:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.882+0000 7f489523f640 1 -- 172.21.15.73:0/2749870538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901008e0 msgr2=0x7f4890100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.882+0000 7f489523f640 1 --2- 172.21.15.73:0/2749870538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901008e0 0x7f4890100d00 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4880009900 tx=0x7f488002a410 comp rx=0 tx=0).stop 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 -- 172.21.15.73:0/2749870538 shutdown_connections 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 --2- 172.21.15.73:0/2749870538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890102490 0x7f4890108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 --2- 172.21.15.73:0/2749870538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4890101ad0 0x7f4890101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 --2- 172.21.15.73:0/2749870538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48901008e0 0x7f4890100d00 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 -- 172.21.15.73:0/2749870538 >> 172.21.15.73:0/2749870538 conn(0x7f48900fbe50 msgr2=0x7f48900fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 -- 172.21.15.73:0/2749870538 shutdown_connections 2024-06-06T07:01:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 -- 172.21.15.73:0/2749870538 wait complete. 2024-06-06T07:01:19.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.883+0000 7f489523f640 1 Processor -- start 2024-06-06T07:01:19.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 -- start start 2024-06-06T07:01:19.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 0x7f4890104870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 0x7f4890102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f489010c3e0 con 0x7f4890102490 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f489010c260 con 0x7f4890101ad0 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f489523f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f489010c560 con 0x7f48901008e0 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59294/0 (socket says 172.21.15.73:59294) 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 -- 172.21.15.73:0/1274109339 learned_addr learned my addr 172.21.15.73:0/1274109339 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488e59c640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 0x7f4890102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488ed9d640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 0x7f4890104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 -- 172.21.15.73:0/1274109339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 msgr2=0x7f4890104870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 0x7f4890104870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 -- 172.21.15.73:0/1274109339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 msgr2=0x7f4890102eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 0x7f4890102eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488f59e640 1 -- 172.21.15.73:0/1274109339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4890103c80 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488ed9d640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 0x7f4890104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.884+0000 7f488e59c640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 0x7f4890102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f488f59e640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f488800eb10 tx=0x7f488800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888004b20 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4888004cc0 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48901a1dc0 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4888010690 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.885+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48901a2230 con 0x7f4890102490 2024-06-06T07:01:19.888 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.886+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48901a2520 con 0x7f4890102490 2024-06-06T07:01:19.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.887+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f4888005000 con 0x7f4890102490 2024-06-06T07:01:19.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.888+0000 7f4877fff640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 0x7f48780a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:19.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.888+0000 7f488ed9d640 1 -- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 msgr2=0x7f48780a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:19.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.888+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f48880cfc70 con 0x7f4890102490 2024-06-06T07:01:19.889 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.888+0000 7f488ed9d640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 0x7f48780a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:19.892 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:19.892+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4888053850 con 0x7f4890102490 2024-06-06T07:01:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:19 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-06T07:01:20.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:19.771+0000 7f990f790200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-06T07:01:20.025 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:19.872+0000 7f990f790200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-06T07:01:20.076 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.075+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f48901a2790 con 0x7f4890102490 2024-06-06T07:01:20.078 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.077+0000 7f4877fff640 1 -- 172.21.15.73:0/1274109339 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+24816 (secure 0 0 0) 0x7f488809c7a0 con 0x7f4890102490 2024-06-06T07:01:20.078 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:20.078 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":100,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T06:58:53.383839+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:01:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 msgr2=0x7f48780a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 0x7f48780a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 msgr2=0x7f48901033f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f488800eb10 tx=0x7f488800c2d0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 shutdown_connections 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f48780a2be0 0x7f48780a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4890102490 0x7f48901033f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4890101ad0 0x7f4890102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 --2- 172.21.15.73:0/1274109339 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f48901008e0 0x7f4890104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.079+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 >> 172.21.15.73:0/1274109339 conn(0x7f48900fbe50 msgr2=0x7f4890068bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.080+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 shutdown_connections 2024-06-06T07:01:20.081 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.080+0000 7f489523f640 1 -- 172.21.15.73:0/1274109339 wait complete. 2024-06-06T07:01:20.092 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T06:52:17.534158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T06:57:03.217358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T06:57:06.475737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T06:57:16.584799+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T07:01:20.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T07:01:20.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:19.831+0000 7f60ea4c2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:01:20.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:19.966+0000 7f60ea4c2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:01:20.097 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:20.062+0000 7f60ea4c2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:01:20.186 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:19.837+0000 7f98a4910200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:01:20.186 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:19.971+0000 7f98a4910200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:01:20.186 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:20.067+0000 7f98a4910200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:01:20.186 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:20.184+0000 7f98a4910200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 -- 172.21.15.73:0/632735193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d801024e0 msgr2=0x7f7d801048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 --2- 172.21.15.73:0/632735193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d801024e0 0x7f7d801048d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f7d70009900 tx=0x7f7d7002a410 comp rx=0 tx=0).stop 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 -- 172.21.15.73:0/632735193 shutdown_connections 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 --2- 172.21.15.73:0/632735193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 0x7f7d80109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 --2- 172.21.15.73:0/632735193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d80104e10 0x7f7d80107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 --2- 172.21.15.73:0/632735193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d801024e0 0x7f7d801048d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 -- 172.21.15.73:0/632735193 >> 172.21.15.73:0/632735193 conn(0x7f7d800fbe50 msgr2=0x7f7d800fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 -- 172.21.15.73:0/632735193 shutdown_connections 2024-06-06T07:01:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.198+0000 7f7d88600640 1 -- 172.21.15.73:0/632735193 wait complete. 2024-06-06T07:01:20.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 Processor -- start 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 -- start start 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 0x7f7d80197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 0x7f7d8019f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010d530 con 0x7f7d80104e10 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010d3b0 con 0x7f7d80107740 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d88600640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8010d6b0 con 0x7f7d801024e0 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59322/0 (socket says 172.21.15.73:59322) 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 -- 172.21.15.73:0/1225039863 learned_addr learned my addr 172.21.15.73:0/1225039863 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:20.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d8639c640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 0x7f7d80197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d86b9d640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 0x7f7d8019f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 -- 172.21.15.73:0/1225039863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 msgr2=0x7f7d80197cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 0x7f7d80197cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 -- 172.21.15.73:0/1225039863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 msgr2=0x7f7d8019f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 0x7f7d8019f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d85b9b640 1 -- 172.21.15.73:0/1225039863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8019f8f0 con 0x7f7d80104e10 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.199+0000 7f7d8639c640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 0x7f7d80197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d85b9b640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e9f0 tx=0x7f7d7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00ccc0 con 0x7f7d80104e10 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8019fbe0 con 0x7f7d80104e10 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d7c0054d0 con 0x7f7d80104e10 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c010430 con 0x7f7d80104e10 2024-06-06T07:01:20.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.200+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d801a0130 con 0x7f7d80104e10 2024-06-06T07:01:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.201+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d800fd940 con 0x7f7d80104e10 2024-06-06T07:01:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.202+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f7d7c005000 con 0x7f7d80104e10 2024-06-06T07:01:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.203+0000 7f7d777fe640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 0x7f7d6c0a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.203+0000 7f7d8639c640 1 -- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 msgr2=0x7f7d6c0a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:20.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.203+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f7d7c0cfa10 con 0x7f7d80104e10 2024-06-06T07:01:20.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.203+0000 7f7d8639c640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 0x7f7d6c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:20.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.209+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d7c053850 con 0x7f7d80104e10 2024-06-06T07:01:20.378 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:20.133+0000 7f990f790200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-06T07:01:20.378 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:20.276+0000 7f990f790200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-06T07:01:20.378 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:20.377+0000 7f990f790200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-06T07:01:20.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.389+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7d8002d4e0 con 0x7f7d80104e10 2024-06-06T07:01:20.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.390+0000 7f7d777fe640 1 -- 172.21.15.73:0/1225039863 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7f7d7c0052f0 con 0x7f7d80104e10 2024-06-06T07:01:20.391 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 msgr2=0x7f7d6c0a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 0x7f7d6c0a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 msgr2=0x7f7d801981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e9f0 tx=0x7f7d7c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 shutdown_connections 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f7d6c0a2be0 0x7f7d6c0a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d80107740 0x7f7d8019f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d80104e10 0x7f7d801981f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 --2- 172.21.15.73:0/1225039863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d801024e0 0x7f7d80197cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 >> 172.21.15.73:0/1225039863 conn(0x7f7d800fbe50 msgr2=0x7f7d80104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 shutdown_connections 2024-06-06T07:01:20.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.392+0000 7f7d88600640 1 -- 172.21.15.73:0/1225039863 wait complete. 2024-06-06T07:01:20.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T07:01:20.486 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:20.179+0000 7f60ea4c2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/2270114221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4102490 msgr2=0x7f9ce4108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/2270114221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4102490 0x7f9ce4108d20 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009ca0 tx=0x7f9cdc02a8c0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/2270114221 shutdown_connections 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/2270114221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4102490 0x7f9ce4108d20 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/2270114221 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/2270114221 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce41008e0 0x7f9ce4100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/2270114221 >> 172.21.15.73:0/2270114221 conn(0x7f9ce40fbe50 msgr2=0x7f9ce40fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/2270114221 shutdown_connections 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.511+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/2270114221 wait complete. 2024-06-06T07:01:20.515 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 Processor -- start 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 -- start start 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 0x7f9ce4104870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce4102490 0x7f9ce41033f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410c410 con 0x7f9ce4101ad0 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410c290 con 0x7f9ce4102490 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ceb3b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce410c590 con 0x7f9ce41008e0 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59338/0 (socket says 172.21.15.73:59338) 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 -- 172.21.15.73:0/1146642190 learned_addr learned my addr 172.21.15.73:0/1146642190 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce9153640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 0x7f9ce4104870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce9954640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce4102490 0x7f9ce41033f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 -- 172.21.15.73:0/1146642190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 msgr2=0x7f9ce4104870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 0x7f9ce4104870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 -- 172.21.15.73:0/1146642190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce4102490 msgr2=0x7f9ce41033f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce4102490 0x7f9ce41033f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce8952640 1 -- 172.21.15.73:0/1146642190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce4103c80 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.512+0000 7f9ce9153640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 0x7f9ce4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9ce8952640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd400cd80 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9cd40054d0 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce41a1d50 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4010490 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.513+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce41a2190 con 0x7f9ce4101ad0 2024-06-06T07:01:20.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.514+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce41a2480 con 0x7f9ce4101ad0 2024-06-06T07:01:20.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.515+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f9cd4005020 con 0x7f9ce4101ad0 2024-06-06T07:01:20.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.516+0000 7f9cda7fc640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 0x7f9cd00a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.516+0000 7f9ce9153640 1 -- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 msgr2=0x7f9cd00a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:20.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.516+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f9cd40cf840 con 0x7f9ce4101ad0 2024-06-06T07:01:20.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.516+0000 7f9ce9153640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 0x7f9cd00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:20.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.520+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9cd409c3d0 con 0x7f9ce4101ad0 2024-06-06T07:01:20.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.703+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f9ce41a2650 con 0x7f9ce4101ad0 2024-06-06T07:01:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.704+0000 7f9cda7fc640 1 -- 172.21.15.73:0/1146642190 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7f9cd409bcd0 con 0x7f9ce4101ad0 2024-06-06T07:01:20.705 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 msgr2=0x7f9cd00a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 0x7f9cd00a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 msgr2=0x7f9ce4102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9cd400e9f0 tx=0x7f9cd400edf0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 shutdown_connections 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f9cd00a2be0 0x7f9cd00a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9ce4102490 0x7f9ce41033f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ce4101ad0 0x7f9ce4102eb0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 --2- 172.21.15.73:0/1146642190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9ce41008e0 0x7f9ce4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 >> 172.21.15.73:0/1146642190 conn(0x7f9ce40fbe50 msgr2=0x7f9ce40fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 shutdown_connections 2024-06-06T07:01:20.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.706+0000 7f9ceb3b7640 1 -- 172.21.15.73:0/1146642190 wait complete. 2024-06-06T07:01:20.717 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:20.504+0000 7f990f790200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-06T07:01:20.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T07:01:20.739 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:20.485+0000 7f60ea4c2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:01:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T07:01:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:01:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1274109339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:20.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1225039863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:01:20.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1146642190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:01:20.802 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:20.492+0000 7f98a4910200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2056012913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 msgr2=0x7fbfa81048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2056012913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa81048f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009900 tx=0x7fbfa402a410 comp rx=0 tx=0).stop 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2056012913 shutdown_connections 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2056012913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8107760 0x7fbfa8109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2056012913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8104e30 0x7fbfa8107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2056012913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa81048f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2056012913 >> 172.21.15.73:0/2056012913 conn(0x7fbfa80fbe50 msgr2=0x7fbfa80fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2056012913 shutdown_connections 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.824+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2056012913 wait complete. 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 Processor -- start 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 -- start start 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8104e30 0x7fbfa81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8107760 0x7fbfa819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810d500 con 0x7fbfa8102500 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810d380 con 0x7fbfa8107760 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfafe6b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfa810d680 con 0x7fbfa8104e30 2024-06-06T07:01:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59360/0 (socket says 172.21.15.73:59360) 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 -- 172.21.15.73:0/2786784929 learned_addr learned my addr 172.21.15.73:0/2786784929 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfae408640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8107760 0x7fbfa819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfad406640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8104e30 0x7fbfa81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 -- 172.21.15.73:0/2786784929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8104e30 msgr2=0x7fbfa81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8104e30 0x7fbfa81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 -- 172.21.15.73:0/2786784929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8107760 msgr2=0x7fbfa819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.825+0000 7fbfadc07640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8107760 0x7fbfa819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbfadc07640 1 -- 172.21.15.73:0/2786784929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfa819f8e0 con 0x7fbfa8102500 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbfadc07640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4005940 tx=0x7fbfa40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa403f030 con 0x7fbfa8102500 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4003cb0 con 0x7fbfa8102500 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfa819fb70 con 0x7fbfa8102500 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4035070 con 0x7fbfa8102500 2024-06-06T07:01:20.829 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.826+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfa81a0060 con 0x7fbfa8102500 2024-06-06T07:01:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.827+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfa80fd940 con 0x7fbfa8102500 2024-06-06T07:01:20.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.828+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7fbfa4042030 con 0x7fbfa8102500 2024-06-06T07:01:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.828+0000 7fbf9effd640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 0x7fbf940a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.829+0000 7fbfad406640 1 -- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 msgr2=0x7fbf940a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.829+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7fbfa40ef9d0 con 0x7fbfa8102500 2024-06-06T07:01:20.831 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.829+0000 7fbfad406640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 0x7fbf940a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:20.833 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:20.833+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbfa40bbf10 con 0x7fbfa8102500 2024-06-06T07:01:21.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.015+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fbfa80625a0 con 0x7fbfa8102500 2024-06-06T07:01:21.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.016+0000 7fbf9effd640 1 -- 172.21.15.73:0/2786784929 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7fbfa40c0210 con 0x7fbfa8102500 2024-06-06T07:01:21.017 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 msgr2=0x7fbf940a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 0x7fbf940a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 msgr2=0x7fbfa8197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4005940 tx=0x7fbfa40042f0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 shutdown_connections 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7fbf940a2be0 0x7fbf940a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfa8107760 0x7fbfa819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfa8104e30 0x7fbfa81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 --2- 172.21.15.73:0/2786784929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfa8102500 0x7fbfa8197ca0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.019 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 >> 172.21.15.73:0/2786784929 conn(0x7fbfa80fbe50 msgr2=0x7fbfa81043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:21.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 shutdown_connections 2024-06-06T07:01:21.020 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.018+0000 7fbfafe6b640 1 -- 172.21.15.73:0/2786784929 wait complete. 2024-06-06T07:01:21.024 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:20.818+0000 7f990f790200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-06T07:01:21.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T07:01:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:01:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1274109339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1225039863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:20 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1146642190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T07:01:21.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3026251669' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-06T07:01:21.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:01:21.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1274109339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:21.093 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1225039863' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.094 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:20 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1146642190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.137 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.134+0000 7f2e3f930640 1 -- 172.21.15.73:0/2415906302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38102460 msgr2=0x7f2e38104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.134+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2415906302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38102460 0x7f2e38104850 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009900 tx=0x7f2e2802a410 comp rx=0 tx=0).stop 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- 172.21.15.73:0/2415906302 shutdown_connections 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2415906302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 0x7f2e38109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2415906302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38104d90 0x7f2e38107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2415906302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38102460 0x7f2e38104850 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- 172.21.15.73:0/2415906302 >> 172.21.15.73:0/2415906302 conn(0x7f2e380fbe50 msgr2=0x7f2e380fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- 172.21.15.73:0/2415906302 shutdown_connections 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- 172.21.15.73:0/2415906302 wait complete. 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 Processor -- start 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- start start 2024-06-06T07:01:21.138 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38102460 0x7f2e38197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 0x7f2e3819f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3810d540 con 0x7f2e38104d90 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3810d3c0 con 0x7f2e381076c0 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3f930640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2e3810d6c0 con 0x7f2e38102460 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3cecb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3cecb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59382/0 (socket says 172.21.15.73:59382) 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.135+0000 7f2e3cecb640 1 -- 172.21.15.73:0/2442218774 learned_addr learned my addr 172.21.15.73:0/2442218774 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3d6cc640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38102460 0x7f2e38197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3decd640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 0x7f2e3819f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 -- 172.21.15.73:0/2442218774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38102460 msgr2=0x7f2e38197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38102460 0x7f2e38197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.139 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 -- 172.21.15.73:0/2442218774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 msgr2=0x7f2e3819f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 0x7f2e3819f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 -- 172.21.15.73:0/2442218774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e3819f900 con 0x7f2e38104d90 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3cecb640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e3400ccc0 con 0x7f2e38104d90 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e3819fbf0 con 0x7f2e38104d90 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e340054d0 con 0x7f2e38104d90 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34010430 con 0x7f2e38104d90 2024-06-06T07:01:21.140 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.136+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e381a00f0 con 0x7f2e38104d90 2024-06-06T07:01:21.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.137+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e380fd940 con 0x7f2e38104d90 2024-06-06T07:01:21.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.138+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 137850+0+0 (secure 0 0 0) 0x7f2e34005000 con 0x7f2e38104d90 2024-06-06T07:01:21.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.140+0000 7f2e2e7fc640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 0x7f2e240a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:21.141 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.140+0000 7f2e3d6cc640 1 -- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 msgr2=0x7f2e240a50a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6800/793833713 2024-06-06T07:01:21.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.140+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9208+0+0 (secure 0 0 0) 0x7f2e340cfa70 con 0x7f2e38104d90 2024-06-06T07:01:21.142 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.140+0000 7f2e3d6cc640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 0x7f2e240a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-06T07:01:21.144 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.143+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e34053850 con 0x7f2e38104d90 2024-06-06T07:01:21.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.321+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2e3802d530 con 0x7f2e38104d90 2024-06-06T07:01:21.322 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.321+0000 7f2e2e7fc640 1 -- 172.21.15.73:0/2442218774 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7f2e340052f0 con 0x7f2e38104d90 2024-06-06T07:01:21.322 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-06-06T07:01:21.324 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 msgr2=0x7f2e240a50a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 0x7f2e240a50a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 msgr2=0x7f2e38198200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400e9f0 tx=0x7f2e3400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 shutdown_connections 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:6800/793833713,v1:172.21.15.73:6801/793833713] conn(0x7f2e240a2be0 0x7f2e240a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2e381076c0 0x7f2e3819f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e38104d90 0x7f2e38198200 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 --2- 172.21.15.73:0/2442218774 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2e38102460 0x7f2e38197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 >> 172.21.15.73:0/2442218774 conn(0x7f2e380fbe50 msgr2=0x7f2e38104310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 shutdown_connections 2024-06-06T07:01:21.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:21.324+0000 7f2e3f930640 1 -- 172.21.15.73:0/2442218774 wait complete. 2024-06-06T07:01:21.335 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-06T07:01:21.336 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:01:21.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:21.091+0000 7f60ea4c2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:01:21.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z[37940]: 2024-06-06T07:01:21.311+0000 7f60ea4c2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:01:21.551 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:21.096+0000 7f98a4910200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:01:21.552 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y[37261]: 2024-06-06T07:01:21.312+0000 7f98a4910200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:01:21.581 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:21.690 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.453+0000 7f990f790200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-06T07:01:21.690 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.689+0000 7f990f790200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-06T07:01:21.942 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2786784929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.942 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Standby manager daemon z restarted 2024-06-06T07:01:21.942 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Standby manager daemon z started 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Standby manager daemon y restarted 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Standby manager daemon y started 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2442218774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:01:21.943 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Active manager daemon x restarted 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Activating manager daemon x 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: mgrmap e20: x(active, starting, since 0.0116761s), standbys: y, z 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:01:21.944 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:01:21.945 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:01:21.946 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-mon[32303]: Manager daemon x is now available 2024-06-06T07:01:22.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2786784929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:01:22.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Standby manager daemon z restarted 2024-06-06T07:01:22.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Standby manager daemon z started 2024-06-06T07:01:22.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Standby manager daemon y restarted 2024-06-06T07:01:22.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Standby manager daemon y started 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2442218774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Active manager daemon x restarted 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Activating manager daemon x 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:01:22.053 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: mgrmap e20: x(active, starting, since 0.0116761s), standbys: y, z 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:01:22.054 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:01:22.055 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:01:22.056 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:21 smithi136 ceph-mon[35901]: Manager daemon x is now available 2024-06-06T07:01:22.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2786784929' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:01:22.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Standby manager daemon z restarted 2024-06-06T07:01:22.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Standby manager daemon z started 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Standby manager daemon y restarted 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Standby manager daemon y started 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.170:0/740054081' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.? 172.21.15.136:0/3043860026' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2442218774' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Active manager daemon x restarted 2024-06-06T07:01:22.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Activating manager daemon x 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: osdmap e101: 12 total, 12 up, 12 in 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: mgrmap e20: x(active, starting, since 0.0116761s), standbys: y, z 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:01:22.099 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-06T07:01:22.100 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-06T07:01:22.101 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:21 smithi170 ceph-mon[36108]: Manager daemon x is now available 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f98718c8700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f98718c8700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f98718c8700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f98718c8700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f98718c8700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f986e8c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f986e8c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f986e8c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.276 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f986e8c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:22.276 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:01:21 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:01:21.943+0000 7f986e8c2700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:01:23.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:23.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:01:23.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:01:23.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:22 smithi073 ceph-mon[32303]: mgrmap e21: x(active, since 1.01795s), standbys: y, z 2024-06-06T07:01:23.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:23.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:01:23.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:01:23.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:22 smithi136 ceph-mon[35901]: mgrmap e21: x(active, since 1.01795s), standbys: y, z 2024-06-06T07:01:23.057 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:23.057 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-06T07:01:23.057 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-06T07:01:23.057 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:22 smithi170 ceph-mon[36108]: mgrmap e21: x(active, since 1.01795s), standbys: y, z 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.435+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/3628052309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c103150 msgr2=0x7fe39c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.435+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/3628052309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c103150 0x7fe39c107140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fe38c0099c0 tx=0x7fe38c009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/3628052309 shutdown_connections 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/3628052309 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c1027a0 0x7fe39c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/3628052309 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 0x7fe39c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/3628052309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c103150 0x7fe39c107140 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/3628052309 >> 172.21.15.73:0/3628052309 conn(0x7fe39c079570 msgr2=0x7fe39c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:23.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/3628052309 shutdown_connections 2024-06-06T07:01:23.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/3628052309 wait complete. 2024-06-06T07:01:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 Processor -- start 2024-06-06T07:01:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.436+0000 7fe3a0f7f700 1 -- start start 2024-06-06T07:01:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c103150 0x7fe39c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 0x7fe39c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c112630 con 0x7fe39c1027a0 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c1124b0 con 0x7fe39c107870 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe39c1127b0 con 0x7fe39c103150 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59522/0 (socket says 172.21.15.73:59522) 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 -- 172.21.15.73:0/1690579136 learned_addr learned my addr 172.21.15.73:0/1690579136 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe399d9b700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c103150 0x7fe39c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39ad9d700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 0x7fe39c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 -- 172.21.15.73:0/1690579136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c103150 msgr2=0x7fe39c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c103150 0x7fe39c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 -- 172.21.15.73:0/1690579136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 msgr2=0x7fe39c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 0x7fe39c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 -- 172.21.15.73:0/1690579136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39c199050 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe39a59c700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d980 tx=0x7fe39000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390016070 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe390004830 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39c199340 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.437+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe390005530 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.438+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39c1a5600 con 0x7fe39c1027a0 2024-06-06T07:01:23.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.438+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe39c069180 con 0x7fe39c1027a0 2024-06-06T07:01:23.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.440+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) ==== 137645+0+0 (secure 0 0 0) 0x7fe3900049e0 con 0x7fe39c1027a0 2024-06-06T07:01:23.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.440+0000 7fe38b7fe700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 0x7fe3840a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:23.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.441+0000 7fe399d9b700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 0x7fe3840a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:23.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.441+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fe3900cf470 con 0x7fe39c1027a0 2024-06-06T07:01:23.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.441+0000 7fe399d9b700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 0x7fe3840a4ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe39c102090 tx=0x7fe38c005fb0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:23.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.444+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe39001f020 con 0x7fe39c1027a0 2024-06-06T07:01:23.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.698+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe39c02dca0 con 0x7fe39c1027a0 2024-06-06T07:01:23.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.699+0000 7fe38b7fe700 1 -- 172.21.15.73:0/1690579136 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11932 (secure 0 0 0) 0x7fe39009c1b0 con 0x7fe39c1027a0 2024-06-06T07:01:23.700 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:23.700 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T07:01:19.680600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24419,24433,24421,24404,24415]},"id":1}]} 2024-06-06T07:01:23.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 msgr2=0x7fe3840a4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:23.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 0x7fe3840a4ee0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe39c102090 tx=0x7fe38c005fb0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 msgr2=0x7fe39c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:23.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d980 tx=0x7fe39000dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:23.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 shutdown_connections 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3840a2a20 0x7fe3840a4ee0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe39c107870 0x7fe39c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe39c1027a0 0x7fe39c19e730 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 --2- 172.21.15.73:0/1690579136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe39c103150 0x7fe39c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 >> 172.21.15.73:0/1690579136 conn(0x7fe39c079570 msgr2=0x7fe39c100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 shutdown_connections 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:23.702+0000 7fe3a0f7f700 1 -- 172.21.15.73:0/1690579136 wait complete. 2024-06-06T07:01:23.704 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 30 2024-06-06T07:01:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: [06/Jun/2024:07:01:22] ENGINE Bus STARTING 2024-06-06T07:01:24.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: [06/Jun/2024:07:01:22] ENGINE Serving on http://172.21.15.73:8765 2024-06-06T07:01:24.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:24.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: [06/Jun/2024:07:01:22] ENGINE Serving on https://172.21.15.73:7150 2024-06-06T07:01:24.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: [06/Jun/2024:07:01:22] ENGINE Bus STARTED 2024-06-06T07:01:24.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:23 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1690579136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: [06/Jun/2024:07:01:22] ENGINE Bus STARTING 2024-06-06T07:01:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: [06/Jun/2024:07:01:22] ENGINE Serving on http://172.21.15.73:8765 2024-06-06T07:01:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: [06/Jun/2024:07:01:22] ENGINE Serving on https://172.21.15.73:7150 2024-06-06T07:01:24.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: [06/Jun/2024:07:01:22] ENGINE Bus STARTED 2024-06-06T07:01:24.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:23 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1690579136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: [06/Jun/2024:07:01:22] ENGINE Bus STARTING 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: [06/Jun/2024:07:01:22] ENGINE Serving on http://172.21.15.73:8765 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: [06/Jun/2024:07:01:22] ENGINE Serving on https://172.21.15.73:7150 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: [06/Jun/2024:07:01:22] ENGINE Bus STARTED 2024-06-06T07:01:24.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:23 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1690579136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:24.508 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-06T07:01:24.508 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-06T07:01:24.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:01:24.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/4284745803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68101ad0 msgr2=0x7f5b68101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/4284745803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68101ad0 0x7f5b68101f50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b50009900 tx=0x7f5b5002a410 comp rx=0 tx=0).stop 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/4284745803 shutdown_connections 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/4284745803 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b68102490 0x7f5b68108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/4284745803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68101ad0 0x7f5b68101f50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/4284745803 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b681008e0 0x7f5b68100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/4284745803 >> 172.21.15.73:0/4284745803 conn(0x7f5b680fbe50 msgr2=0x7f5b680fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/4284745803 shutdown_connections 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/4284745803 wait complete. 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 Processor -- start 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- start start 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b681008e0 0x7f5b68197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b68101ad0 0x7f5b681981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810c3f0 con 0x7f5b68102490 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810c270 con 0x7f5b681008e0 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6ef5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6810c570 con 0x7f5b68101ad0 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.613+0000 7f5b6d4f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b67fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b68101ad0 0x7f5b681981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59556/0 (socket says 172.21.15.73:59556) 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 -- 172.21.15.73:0/1954094616 learned_addr learned my addr 172.21.15.73:0/1954094616 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6ccf6640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b681008e0 0x7f5b68197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 -- 172.21.15.73:0/1954094616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b68101ad0 msgr2=0x7f5b681981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b68101ad0 0x7f5b681981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 -- 172.21.15.73:0/1954094616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b681008e0 msgr2=0x7f5b68197ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b681008e0 0x7f5b68197ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 -- 172.21.15.73:0/1954094616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6819f8e0 con 0x7f5b68102490 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6d4f7640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009fd0 tx=0x7f5b5800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58017070 con 0x7f5b68102490 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b58003e20 con 0x7f5b68102490 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6819fbd0 con 0x7f5b68102490 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58010460 con 0x7f5b68102490 2024-06-06T07:01:24.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.614+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b681a00a0 con 0x7f5b68102490 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.615+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b681a0270 con 0x7f5b68102490 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.616+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f5b58003970 con 0x7f5b68102490 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.617+0000 7f5b65ffb640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 0x7f5b540a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.617+0000 7f5b6ccf6640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 0x7f5b540a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.617+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f5b580cf580 con 0x7f5b68102490 2024-06-06T07:01:24.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.618+0000 7f5b6ccf6640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 0x7f5b540a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5b68198b70 tx=0x7f5b6000b390 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:24.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.621+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b5809b7a0 con 0x7f5b68102490 2024-06-06T07:01:24.803 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.802+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b680692f0 con 0x7f5b540a2d00 2024-06-06T07:01:24.812 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.812+0000 7f5b65ffb640 1 -- 172.21.15.73:0/1954094616 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339927 (secure 0 0 0) 0x7f5b680692f0 con 0x7f5b540a2d00 2024-06-06T07:01:24.818 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:24.820 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T07:01:24.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 msgr2=0x7f5b540a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 0x7f5b540a51c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5b68198b70 tx=0x7f5b6000b390 comp rx=0 tx=0).stop 2024-06-06T07:01:24.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 msgr2=0x7f5b6819f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b58009fd0 tx=0x7f5b5800b260 comp rx=0 tx=0).stop 2024-06-06T07:01:24.822 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 shutdown_connections 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b68102490 0x7f5b6819f240 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f5b540a2d00 0x7f5b540a51c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5b68101ad0 0x7f5b681981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 --2- 172.21.15.73:0/1954094616 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5b681008e0 0x7f5b68197ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.815+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 >> 172.21.15.73:0/1954094616 conn(0x7f5b680fbe50 msgr2=0x7f5b68068be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.816+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 shutdown_connections 2024-06-06T07:01:24.823 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.816+0000 7f5b6ef5a640 1 -- 172.21.15.73:0/1954094616 wait complete. 2024-06-06T07:01:24.827 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-06-06T07:01:23.708112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195190,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2122,"num_read_kb":1233,"num_write":578,"num_write_kb":4421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":956,"ondisk_log_size":956,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387960,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553864,"statfs":{"total":1151940427776,"available":1151543156736,"internally_reserved":0,"allocated":14139392,"data_stored":10339226,"data_compressed":592079,"data_compressed_allocated":802816,"data_compressed_original":4284416,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989506"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960251+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T07:01:21.960251+0000","last_peered":"2024-06-06T07:01:21.960251+0000","last_clean":"2024-06-06T07:01:21.960251+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T07:01:21.960251+0000","last_undegraded":"2024-06-06T07:01:21.960251+0000","last_fullsized":"2024-06-06T07:01:21.960251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960220+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T07:01:21.960220+0000","last_peered":"2024-06-06T07:01:21.960220+0000","last_clean":"2024-06-06T07:01:21.960220+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T07:01:21.960220+0000","last_undegraded":"2024-06-06T07:01:21.960220+0000","last_fullsized":"2024-06-06T07:01:21.960220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960212+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T07:01:21.960212+0000","last_peered":"2024-06-06T07:01:21.960212+0000","last_clean":"2024-06-06T07:01:21.960212+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T07:01:21.960212+0000","last_undegraded":"2024-06-06T07:01:21.960212+0000","last_fullsized":"2024-06-06T07:01:21.960212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950531+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T07:01:21.950531+0000","last_peered":"2024-06-06T07:01:21.950531+0000","last_clean":"2024-06-06T07:01:21.950531+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T07:01:21.950531+0000","last_undegraded":"2024-06-06T07:01:21.950531+0000","last_fullsized":"2024-06-06T07:01:21.950531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797379+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T07:01:21.797379+0000","last_peered":"2024-06-06T07:01:21.797379+0000","last_clean":"2024-06-06T07:01:21.797379+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T07:01:21.797379+0000","last_undegraded":"2024-06-06T07:01:21.797379+0000","last_fullsized":"2024-06-06T07:01:21.797379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698596+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T07:01:21.698596+0000","last_peered":"2024-06-06T07:01:21.698596+0000","last_clean":"2024-06-06T07:01:21.698596+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T07:01:21.698596+0000","last_undegraded":"2024-06-06T07:01:21.698596+0000","last_fullsized":"2024-06-06T07:01:21.698596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949896+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T07:01:21.949896+0000","last_peered":"2024-06-06T07:01:21.949896+0000","last_clean":"2024-06-06T07:01:21.949896+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T07:01:21.949896+0000","last_undegraded":"2024-06-06T07:01:21.949896+0000","last_fullsized":"2024-06-06T07:01:21.949896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796424+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T07:01:21.796424+0000","last_peered":"2024-06-06T07:01:21.796424+0000","last_clean":"2024-06-06T07:01:21.796424+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T07:01:21.796424+0000","last_undegraded":"2024-06-06T07:01:21.796424+0000","last_fullsized":"2024-06-06T07:01:21.796424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955413+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T07:01:21.955413+0000","last_peered":"2024-06-06T07:01:21.955413+0000","last_clean":"2024-06-06T07:01:21.955413+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T07:01:21.955413+0000","last_undegraded":"2024-06-06T07:01:21.955413+0000","last_fullsized":"2024-06-06T07:01:21.955413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699282+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T07:01:21.699282+0000","last_peered":"2024-06-06T07:01:21.699282+0000","last_clean":"2024-06-06T07:01:21.699282+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T07:01:21.699282+0000","last_undegraded":"2024-06-06T07:01:21.699282+0000","last_fullsized":"2024-06-06T07:01:21.699282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950661+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T07:01:21.950661+0000","last_peered":"2024-06-06T07:01:21.950661+0000","last_clean":"2024-06-06T07:01:21.950661+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T07:01:21.950661+0000","last_undegraded":"2024-06-06T07:01:21.950661+0000","last_fullsized":"2024-06-06T07:01:21.950661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950393+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T07:01:21.950393+0000","last_peered":"2024-06-06T07:01:21.950393+0000","last_clean":"2024-06-06T07:01:21.950393+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T07:01:21.950393+0000","last_undegraded":"2024-06-06T07:01:21.950393+0000","last_fullsized":"2024-06-06T07:01:21.950393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955249+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T07:01:21.955249+0000","last_peered":"2024-06-06T07:01:21.955249+0000","last_clean":"2024-06-06T07:01:21.955249+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T07:01:21.955249+0000","last_undegraded":"2024-06-06T07:01:21.955249+0000","last_fullsized":"2024-06-06T07:01:21.955249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959991+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T07:01:21.959991+0000","last_peered":"2024-06-06T07:01:21.959991+0000","last_clean":"2024-06-06T07:01:21.959991+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T07:01:21.959991+0000","last_undegraded":"2024-06-06T07:01:21.959991+0000","last_fullsized":"2024-06-06T07:01:21.959991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959909+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T07:01:21.959909+0000","last_peered":"2024-06-06T07:01:21.959909+0000","last_clean":"2024-06-06T07:01:21.959909+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T07:01:21.959909+0000","last_undegraded":"2024-06-06T07:01:21.959909+0000","last_fullsized":"2024-06-06T07:01:21.959909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954785+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T07:01:21.954785+0000","last_peered":"2024-06-06T07:01:21.954785+0000","last_clean":"2024-06-06T07:01:21.954785+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T07:01:21.954785+0000","last_undegraded":"2024-06-06T07:01:21.954785+0000","last_fullsized":"2024-06-06T07:01:21.954785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960031+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T07:01:21.960031+0000","last_peered":"2024-06-06T07:01:21.960031+0000","last_clean":"2024-06-06T07:01:21.960031+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T07:01:21.960031+0000","last_undegraded":"2024-06-06T07:01:21.960031+0000","last_fullsized":"2024-06-06T07:01:21.960031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696962+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T07:01:21.696962+0000","last_peered":"2024-06-06T07:01:21.696962+0000","last_clean":"2024-06-06T07:01:21.696962+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T07:01:21.696962+0000","last_undegraded":"2024-06-06T07:01:21.696962+0000","last_fullsized":"2024-06-06T07:01:21.696962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955638+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T07:01:21.955638+0000","last_peered":"2024-06-06T07:01:21.955638+0000","last_clean":"2024-06-06T07:01:21.955638+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T07:01:21.955638+0000","last_undegraded":"2024-06-06T07:01:21.955638+0000","last_fullsized":"2024-06-06T07:01:21.955638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955116+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T07:01:21.955116+0000","last_peered":"2024-06-06T07:01:21.955116+0000","last_clean":"2024-06-06T07:01:21.955116+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T07:01:21.955116+0000","last_undegraded":"2024-06-06T07:01:21.955116+0000","last_fullsized":"2024-06-06T07:01:21.955116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696861+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T07:01:21.696861+0000","last_peered":"2024-06-06T07:01:21.696861+0000","last_clean":"2024-06-06T07:01:21.696861+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T07:01:21.696861+0000","last_undegraded":"2024-06-06T07:01:21.696861+0000","last_fullsized":"2024-06-06T07:01:21.696861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798424+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T07:01:21.798424+0000","last_peered":"2024-06-06T07:01:21.798424+0000","last_clean":"2024-06-06T07:01:21.798424+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T07:01:21.798424+0000","last_undegraded":"2024-06-06T07:01:21.798424+0000","last_fullsized":"2024-06-06T07:01:21.798424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949488+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T07:01:21.949488+0000","last_peered":"2024-06-06T07:01:21.949488+0000","last_clean":"2024-06-06T07:01:21.949488+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T07:01:21.949488+0000","last_undegraded":"2024-06-06T07:01:21.949488+0000","last_fullsized":"2024-06-06T07:01:21.949488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798829+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T07:01:21.798829+0000","last_peered":"2024-06-06T07:01:21.798829+0000","last_clean":"2024-06-06T07:01:21.798829+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T07:01:21.798829+0000","last_undegraded":"2024-06-06T07:01:21.798829+0000","last_fullsized":"2024-06-06T07:01:21.798829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960215+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T07:01:21.960215+0000","last_peered":"2024-06-06T07:01:21.960215+0000","last_clean":"2024-06-06T07:01:21.960215+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T07:01:21.960215+0000","last_undegraded":"2024-06-06T07:01:21.960215+0000","last_fullsized":"2024-06-06T07:01:21.960215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950690+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T07:01:21.950690+0000","last_peered":"2024-06-06T07:01:21.950690+0000","last_clean":"2024-06-06T07:01:21.950690+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T07:01:21.950690+0000","last_undegraded":"2024-06-06T07:01:21.950690+0000","last_fullsized":"2024-06-06T07:01:21.950690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":33,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960125+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T07:01:21.960125+0000","last_peered":"2024-06-06T07:01:21.960125+0000","last_clean":"2024-06-06T07:01:21.960125+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T07:01:21.960125+0000","last_undegraded":"2024-06-06T07:01:21.960125+0000","last_fullsized":"2024-06-06T07:01:21.960125+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955097+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T07:01:21.955097+0000","last_peered":"2024-06-06T07:01:21.955097+0000","last_clean":"2024-06-06T07:01:21.955097+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T07:01:21.955097+0000","last_undegraded":"2024-06-06T07:01:21.955097+0000","last_fullsized":"2024-06-06T07:01:21.955097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949550+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T07:01:21.949550+0000","last_peered":"2024-06-06T07:01:21.949550+0000","last_clean":"2024-06-06T07:01:21.949550+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T07:01:21.949550+0000","last_undegraded":"2024-06-06T07:01:21.949550+0000","last_fullsized":"2024-06-06T07:01:21.949550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797295+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T07:01:21.797295+0000","last_peered":"2024-06-06T07:01:21.797295+0000","last_clean":"2024-06-06T07:01:21.797295+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T07:01:21.797295+0000","last_undegraded":"2024-06-06T07:01:21.797295+0000","last_fullsized":"2024-06-06T07:01:21.797295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698810+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T07:01:21.698810+0000","last_peered":"2024-06-06T07:01:21.698810+0000","last_clean":"2024-06-06T07:01:21.698810+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T07:01:21.698810+0000","last_undegraded":"2024-06-06T07:01:21.698810+0000","last_fullsized":"2024-06-06T07:01:21.698810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955250+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T07:01:21.955250+0000","last_peered":"2024-06-06T07:01:21.955250+0000","last_clean":"2024-06-06T07:01:21.955250+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T07:01:21.955250+0000","last_undegraded":"2024-06-06T07:01:21.955250+0000","last_fullsized":"2024-06-06T07:01:21.955250+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'7","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680167+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T07:01:22.680167+0000","last_peered":"2024-06-06T07:01:22.680167+0000","last_clean":"2024-06-06T07:01:22.680167+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T07:01:22.680167+0000","last_undegraded":"2024-06-06T07:01:22.680167+0000","last_fullsized":"2024-06-06T07:01:22.680167+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796975+0000","last_change":"2024-06-06T06:58:53.476855+0000","last_active":"2024-06-06T07:01:21.796975+0000","last_peered":"2024-06-06T07:01:21.796975+0000","last_clean":"2024-06-06T07:01:21.796975+0000","last_became_active":"2024-06-06T06:58:53.476098+0000","last_became_peered":"2024-06-06T06:58:53.476098+0000","last_unstale":"2024-06-06T07:01:21.796975+0000","last_undegraded":"2024-06-06T07:01:21.796975+0000","last_fullsized":"2024-06-06T07:01:21.796975+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:07:20.623344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955349+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T07:01:21.955349+0000","last_peered":"2024-06-06T07:01:21.955349+0000","last_clean":"2024-06-06T07:01:21.955349+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T07:01:21.955349+0000","last_undegraded":"2024-06-06T07:01:21.955349+0000","last_fullsized":"2024-06-06T07:01:21.955349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698559+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T07:01:21.698559+0000","last_peered":"2024-06-06T07:01:21.698559+0000","last_clean":"2024-06-06T07:01:21.698559+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T07:01:21.698559+0000","last_undegraded":"2024-06-06T07:01:21.698559+0000","last_fullsized":"2024-06-06T07:01:21.698559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698107+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T07:01:21.698107+0000","last_peered":"2024-06-06T07:01:21.698107+0000","last_clean":"2024-06-06T07:01:21.698107+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T07:01:21.698107+0000","last_undegraded":"2024-06-06T07:01:21.698107+0000","last_fullsized":"2024-06-06T07:01:21.698107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960014+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T07:01:21.960014+0000","last_peered":"2024-06-06T07:01:21.960014+0000","last_clean":"2024-06-06T07:01:21.960014+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T07:01:21.960014+0000","last_undegraded":"2024-06-06T07:01:21.960014+0000","last_fullsized":"2024-06-06T07:01:21.960014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954890+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T07:01:21.954890+0000","last_peered":"2024-06-06T07:01:21.954890+0000","last_clean":"2024-06-06T07:01:21.954890+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T07:01:21.954890+0000","last_undegraded":"2024-06-06T07:01:21.954890+0000","last_fullsized":"2024-06-06T07:01:21.954890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796557+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T07:01:21.796557+0000","last_peered":"2024-06-06T07:01:21.796557+0000","last_clean":"2024-06-06T07:01:21.796557+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T07:01:21.796557+0000","last_undegraded":"2024-06-06T07:01:21.796557+0000","last_fullsized":"2024-06-06T07:01:21.796557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960416+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T07:01:21.960416+0000","last_peered":"2024-06-06T07:01:21.960416+0000","last_clean":"2024-06-06T07:01:21.960416+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T07:01:21.960416+0000","last_undegraded":"2024-06-06T07:01:21.960416+0000","last_fullsized":"2024-06-06T07:01:21.960416+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697614+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T07:01:21.697614+0000","last_peered":"2024-06-06T07:01:21.697614+0000","last_clean":"2024-06-06T07:01:21.697614+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T07:01:21.697614+0000","last_undegraded":"2024-06-06T07:01:21.697614+0000","last_fullsized":"2024-06-06T07:01:21.697614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696430+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T07:01:21.696430+0000","last_peered":"2024-06-06T07:01:21.696430+0000","last_clean":"2024-06-06T07:01:21.696430+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T07:01:21.696430+0000","last_undegraded":"2024-06-06T07:01:21.696430+0000","last_fullsized":"2024-06-06T07:01:21.696430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796793+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T07:01:21.796793+0000","last_peered":"2024-06-06T07:01:21.796793+0000","last_clean":"2024-06-06T07:01:21.796793+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T07:01:21.796793+0000","last_undegraded":"2024-06-06T07:01:21.796793+0000","last_fullsized":"2024-06-06T07:01:21.796793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":422,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679687+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T07:01:22.679687+0000","last_peered":"2024-06-06T07:01:22.679687+0000","last_clean":"2024-06-06T07:01:22.679687+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T07:01:22.679687+0000","last_undegraded":"2024-06-06T07:01:22.679687+0000","last_fullsized":"2024-06-06T07:01:22.679687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":368,"num_read_kb":368,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955354+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T07:01:21.955354+0000","last_peered":"2024-06-06T07:01:21.955354+0000","last_clean":"2024-06-06T07:01:21.955354+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955354+0000","last_undegraded":"2024-06-06T07:01:21.955354+0000","last_fullsized":"2024-06-06T07:01:21.955354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696386+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T07:01:21.696386+0000","last_peered":"2024-06-06T07:01:21.696386+0000","last_clean":"2024-06-06T07:01:21.696386+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T07:01:21.696386+0000","last_undegraded":"2024-06-06T07:01:21.696386+0000","last_fullsized":"2024-06-06T07:01:21.696386+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951076+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T07:01:21.951076+0000","last_peered":"2024-06-06T07:01:21.951076+0000","last_clean":"2024-06-06T07:01:21.951076+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T07:01:21.951076+0000","last_undegraded":"2024-06-06T07:01:21.951076+0000","last_fullsized":"2024-06-06T07:01:21.951076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960384+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T07:01:21.960384+0000","last_peered":"2024-06-06T07:01:21.960384+0000","last_clean":"2024-06-06T07:01:21.960384+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T07:01:21.960384+0000","last_undegraded":"2024-06-06T07:01:21.960384+0000","last_fullsized":"2024-06-06T07:01:21.960384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950898+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T07:01:21.950898+0000","last_peered":"2024-06-06T07:01:21.950898+0000","last_clean":"2024-06-06T07:01:21.950898+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T07:01:21.950898+0000","last_undegraded":"2024-06-06T07:01:21.950898+0000","last_fullsized":"2024-06-06T07:01:21.950898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698689+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T07:01:21.698689+0000","last_peered":"2024-06-06T07:01:21.698689+0000","last_clean":"2024-06-06T07:01:21.698689+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T07:01:21.698689+0000","last_undegraded":"2024-06-06T07:01:21.698689+0000","last_fullsized":"2024-06-06T07:01:21.698689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955384+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T07:01:21.955384+0000","last_peered":"2024-06-06T07:01:21.955384+0000","last_clean":"2024-06-06T07:01:21.955384+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T07:01:21.955384+0000","last_undegraded":"2024-06-06T07:01:21.955384+0000","last_fullsized":"2024-06-06T07:01:21.955384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696457+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T07:01:21.696457+0000","last_peered":"2024-06-06T07:01:21.696457+0000","last_clean":"2024-06-06T07:01:21.696457+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T07:01:21.696457+0000","last_undegraded":"2024-06-06T07:01:21.696457+0000","last_fullsized":"2024-06-06T07:01:21.696457+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":469,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679951+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T07:01:22.679951+0000","last_peered":"2024-06-06T07:01:22.679951+0000","last_clean":"2024-06-06T07:01:22.679951+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T07:01:22.679951+0000","last_undegraded":"2024-06-06T07:01:22.679951+0000","last_fullsized":"2024-06-06T07:01:22.679951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68760,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":380,"num_read_kb":257,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797184+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T07:01:21.797184+0000","last_peered":"2024-06-06T07:01:21.797184+0000","last_clean":"2024-06-06T07:01:21.797184+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T07:01:21.797184+0000","last_undegraded":"2024-06-06T07:01:21.797184+0000","last_fullsized":"2024-06-06T07:01:21.797184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699244+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T07:01:21.699244+0000","last_peered":"2024-06-06T07:01:21.699244+0000","last_clean":"2024-06-06T07:01:21.699244+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T07:01:21.699244+0000","last_undegraded":"2024-06-06T07:01:21.699244+0000","last_fullsized":"2024-06-06T07:01:21.699244+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955406+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T07:01:21.955406+0000","last_peered":"2024-06-06T07:01:21.955406+0000","last_clean":"2024-06-06T07:01:21.955406+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T07:01:21.955406+0000","last_undegraded":"2024-06-06T07:01:21.955406+0000","last_fullsized":"2024-06-06T07:01:21.955406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954910+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T07:01:21.954910+0000","last_peered":"2024-06-06T07:01:21.954910+0000","last_clean":"2024-06-06T07:01:21.954910+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T07:01:21.954910+0000","last_undegraded":"2024-06-06T07:01:21.954910+0000","last_fullsized":"2024-06-06T07:01:21.954910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959844+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T07:01:21.959844+0000","last_peered":"2024-06-06T07:01:21.959844+0000","last_clean":"2024-06-06T07:01:21.959844+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T07:01:21.959844+0000","last_undegraded":"2024-06-06T07:01:21.959844+0000","last_fullsized":"2024-06-06T07:01:21.959844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697867+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T07:01:21.697867+0000","last_peered":"2024-06-06T07:01:21.697867+0000","last_clean":"2024-06-06T07:01:21.697867+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T07:01:21.697867+0000","last_undegraded":"2024-06-06T07:01:21.697867+0000","last_fullsized":"2024-06-06T07:01:21.697867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950431+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T07:01:21.950431+0000","last_peered":"2024-06-06T07:01:21.950431+0000","last_clean":"2024-06-06T07:01:21.950431+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T07:01:21.950431+0000","last_undegraded":"2024-06-06T07:01:21.950431+0000","last_fullsized":"2024-06-06T07:01:21.950431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697123+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T07:01:21.697123+0000","last_peered":"2024-06-06T07:01:21.697123+0000","last_clean":"2024-06-06T07:01:21.697123+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T07:01:21.697123+0000","last_undegraded":"2024-06-06T07:01:21.697123+0000","last_fullsized":"2024-06-06T07:01:21.697123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679778+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T07:01:22.679778+0000","last_peered":"2024-06-06T07:01:22.679778+0000","last_clean":"2024-06-06T07:01:22.679778+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T07:01:22.679778+0000","last_undegraded":"2024-06-06T07:01:22.679778+0000","last_fullsized":"2024-06-06T07:01:22.679778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2001,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949816+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T07:01:21.949816+0000","last_peered":"2024-06-06T07:01:21.949816+0000","last_clean":"2024-06-06T07:01:21.949816+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T07:01:21.949816+0000","last_undegraded":"2024-06-06T07:01:21.949816+0000","last_fullsized":"2024-06-06T07:01:21.949816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955616+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T07:01:21.955616+0000","last_peered":"2024-06-06T07:01:21.955616+0000","last_clean":"2024-06-06T07:01:21.955616+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T07:01:21.955616+0000","last_undegraded":"2024-06-06T07:01:21.955616+0000","last_fullsized":"2024-06-06T07:01:21.955616+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796849+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T07:01:21.796849+0000","last_peered":"2024-06-06T07:01:21.796849+0000","last_clean":"2024-06-06T07:01:21.796849+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T07:01:21.796849+0000","last_undegraded":"2024-06-06T07:01:21.796849+0000","last_fullsized":"2024-06-06T07:01:21.796849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697500+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T07:01:21.697500+0000","last_peered":"2024-06-06T07:01:21.697500+0000","last_clean":"2024-06-06T07:01:21.697500+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T07:01:21.697500+0000","last_undegraded":"2024-06-06T07:01:21.697500+0000","last_fullsized":"2024-06-06T07:01:21.697500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697474+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T07:01:21.697474+0000","last_peered":"2024-06-06T07:01:21.697474+0000","last_clean":"2024-06-06T07:01:21.697474+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T07:01:21.697474+0000","last_undegraded":"2024-06-06T07:01:21.697474+0000","last_fullsized":"2024-06-06T07:01:21.697474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797233+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T07:01:21.797233+0000","last_peered":"2024-06-06T07:01:21.797233+0000","last_clean":"2024-06-06T07:01:21.797233+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T07:01:21.797233+0000","last_undegraded":"2024-06-06T07:01:21.797233+0000","last_fullsized":"2024-06-06T07:01:21.797233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960037+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T07:01:21.960037+0000","last_peered":"2024-06-06T07:01:21.960037+0000","last_clean":"2024-06-06T07:01:21.960037+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T07:01:21.960037+0000","last_undegraded":"2024-06-06T07:01:21.960037+0000","last_fullsized":"2024-06-06T07:01:21.960037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797116+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T07:01:21.797116+0000","last_peered":"2024-06-06T07:01:21.797116+0000","last_clean":"2024-06-06T07:01:21.797116+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T07:01:21.797116+0000","last_undegraded":"2024-06-06T07:01:21.797116+0000","last_fullsized":"2024-06-06T07:01:21.797116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696557+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T07:01:21.696557+0000","last_peered":"2024-06-06T07:01:21.696557+0000","last_clean":"2024-06-06T07:01:21.696557+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T07:01:21.696557+0000","last_undegraded":"2024-06-06T07:01:21.696557+0000","last_fullsized":"2024-06-06T07:01:21.696557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698895+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.698895+0000","last_peered":"2024-06-06T07:01:21.698895+0000","last_clean":"2024-06-06T07:01:21.698895+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T07:01:21.698895+0000","last_undegraded":"2024-06-06T07:01:21.698895+0000","last_fullsized":"2024-06-06T07:01:21.698895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955129+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T07:01:21.955129+0000","last_peered":"2024-06-06T07:01:21.955129+0000","last_clean":"2024-06-06T07:01:21.955129+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T07:01:21.955129+0000","last_undegraded":"2024-06-06T07:01:21.955129+0000","last_fullsized":"2024-06-06T07:01:21.955129+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":241,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680327+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T07:01:22.680327+0000","last_peered":"2024-06-06T07:01:22.680327+0000","last_clean":"2024-06-06T07:01:22.680327+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T07:01:22.680327+0000","last_undegraded":"2024-06-06T07:01:22.680327+0000","last_fullsized":"2024-06-06T07:01:22.680327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698794+0000","last_change":"2024-06-06T06:58:53.571939+0000","last_active":"2024-06-06T07:01:21.698794+0000","last_peered":"2024-06-06T07:01:21.698794+0000","last_clean":"2024-06-06T07:01:21.698794+0000","last_became_active":"2024-06-06T06:58:53.571590+0000","last_became_peered":"2024-06-06T06:58:53.571590+0000","last_unstale":"2024-06-06T07:01:21.698794+0000","last_undegraded":"2024-06-06T07:01:21.698794+0000","last_fullsized":"2024-06-06T07:01:21.698794+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:10.858649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955640+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T07:01:21.955640+0000","last_peered":"2024-06-06T07:01:21.955640+0000","last_clean":"2024-06-06T07:01:21.955640+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T07:01:21.955640+0000","last_undegraded":"2024-06-06T07:01:21.955640+0000","last_fullsized":"2024-06-06T07:01:21.955640+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697837+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T07:01:21.697837+0000","last_peered":"2024-06-06T07:01:21.697837+0000","last_clean":"2024-06-06T07:01:21.697837+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T07:01:21.697837+0000","last_undegraded":"2024-06-06T07:01:21.697837+0000","last_fullsized":"2024-06-06T07:01:21.697837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679259+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T07:01:22.679259+0000","last_peered":"2024-06-06T07:01:22.679259+0000","last_clean":"2024-06-06T07:01:22.679259+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T07:01:22.679259+0000","last_undegraded":"2024-06-06T07:01:22.679259+0000","last_fullsized":"2024-06-06T07:01:22.679259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949771+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T07:01:21.949771+0000","last_peered":"2024-06-06T07:01:21.949771+0000","last_clean":"2024-06-06T07:01:21.949771+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T07:01:21.949771+0000","last_undegraded":"2024-06-06T07:01:21.949771+0000","last_fullsized":"2024-06-06T07:01:21.949771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797356+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T07:01:21.797356+0000","last_peered":"2024-06-06T07:01:21.797356+0000","last_clean":"2024-06-06T07:01:21.797356+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T07:01:21.797356+0000","last_undegraded":"2024-06-06T07:01:21.797356+0000","last_fullsized":"2024-06-06T07:01:21.797356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950845+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T07:01:21.950845+0000","last_peered":"2024-06-06T07:01:21.950845+0000","last_clean":"2024-06-06T07:01:21.950845+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T07:01:21.950845+0000","last_undegraded":"2024-06-06T07:01:21.950845+0000","last_fullsized":"2024-06-06T07:01:21.950845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":379,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949403+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T07:01:21.949403+0000","last_peered":"2024-06-06T07:01:21.949403+0000","last_clean":"2024-06-06T07:01:21.949403+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T07:01:21.949403+0000","last_undegraded":"2024-06-06T07:01:21.949403+0000","last_fullsized":"2024-06-06T07:01:21.949403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698791+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T07:01:21.698791+0000","last_peered":"2024-06-06T07:01:21.698791+0000","last_clean":"2024-06-06T07:01:21.698791+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T07:01:21.698791+0000","last_undegraded":"2024-06-06T07:01:21.698791+0000","last_fullsized":"2024-06-06T07:01:21.698791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954789+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T07:01:21.954789+0000","last_peered":"2024-06-06T07:01:21.954789+0000","last_clean":"2024-06-06T07:01:21.954789+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T07:01:21.954789+0000","last_undegraded":"2024-06-06T07:01:21.954789+0000","last_fullsized":"2024-06-06T07:01:21.954789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949695+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T07:01:21.949695+0000","last_peered":"2024-06-06T07:01:21.949695+0000","last_clean":"2024-06-06T07:01:21.949695+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T07:01:21.949695+0000","last_undegraded":"2024-06-06T07:01:21.949695+0000","last_fullsized":"2024-06-06T07:01:21.949695+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959816+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T07:01:21.959816+0000","last_peered":"2024-06-06T07:01:21.959816+0000","last_clean":"2024-06-06T07:01:21.959816+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T07:01:21.959816+0000","last_undegraded":"2024-06-06T07:01:21.959816+0000","last_fullsized":"2024-06-06T07:01:21.959816+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797392+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T07:01:21.797392+0000","last_peered":"2024-06-06T07:01:21.797392+0000","last_clean":"2024-06-06T07:01:21.797392+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T07:01:21.797392+0000","last_undegraded":"2024-06-06T07:01:21.797392+0000","last_fullsized":"2024-06-06T07:01:21.797392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955313+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T07:01:21.955313+0000","last_peered":"2024-06-06T07:01:21.955313+0000","last_clean":"2024-06-06T07:01:21.955313+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T07:01:21.955313+0000","last_undegraded":"2024-06-06T07:01:21.955313+0000","last_fullsized":"2024-06-06T07:01:21.955313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796559+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T07:01:21.796559+0000","last_peered":"2024-06-06T07:01:21.796559+0000","last_clean":"2024-06-06T07:01:21.796559+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T07:01:21.796559+0000","last_undegraded":"2024-06-06T07:01:21.796559+0000","last_fullsized":"2024-06-06T07:01:21.796559+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'52","reported_seq":123,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.909087+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T07:01:21.909087+0000","last_peered":"2024-06-06T07:01:21.909087+0000","last_clean":"2024-06-06T07:01:21.909087+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T07:01:21.909087+0000","last_undegraded":"2024-06-06T07:01:21.909087+0000","last_fullsized":"2024-06-06T07:01:21.909087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796688+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T07:01:21.796688+0000","last_peered":"2024-06-06T07:01:21.796688+0000","last_clean":"2024-06-06T07:01:21.796688+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T07:01:21.796688+0000","last_undegraded":"2024-06-06T07:01:21.796688+0000","last_fullsized":"2024-06-06T07:01:21.796688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954998+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T07:01:21.954998+0000","last_peered":"2024-06-06T07:01:21.954998+0000","last_clean":"2024-06-06T07:01:21.954998+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T07:01:21.954998+0000","last_undegraded":"2024-06-06T07:01:21.954998+0000","last_fullsized":"2024-06-06T07:01:21.954998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950264+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T07:01:21.950264+0000","last_peered":"2024-06-06T07:01:21.950264+0000","last_clean":"2024-06-06T07:01:21.950264+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T07:01:21.950264+0000","last_undegraded":"2024-06-06T07:01:21.950264+0000","last_fullsized":"2024-06-06T07:01:21.950264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960301+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T07:01:21.960301+0000","last_peered":"2024-06-06T07:01:21.960301+0000","last_clean":"2024-06-06T07:01:21.960301+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T07:01:21.960301+0000","last_undegraded":"2024-06-06T07:01:21.960301+0000","last_fullsized":"2024-06-06T07:01:21.960301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950031+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T07:01:21.950031+0000","last_peered":"2024-06-06T07:01:21.950031+0000","last_clean":"2024-06-06T07:01:21.950031+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T07:01:21.950031+0000","last_undegraded":"2024-06-06T07:01:21.950031+0000","last_fullsized":"2024-06-06T07:01:21.950031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954812+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T07:01:21.954812+0000","last_peered":"2024-06-06T07:01:21.954812+0000","last_clean":"2024-06-06T07:01:21.954812+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T07:01:21.954812+0000","last_undegraded":"2024-06-06T07:01:21.954812+0000","last_fullsized":"2024-06-06T07:01:21.954812+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796855+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T07:01:21.796855+0000","last_peered":"2024-06-06T07:01:21.796855+0000","last_clean":"2024-06-06T07:01:21.796855+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T07:01:21.796855+0000","last_undegraded":"2024-06-06T07:01:21.796855+0000","last_fullsized":"2024-06-06T07:01:21.796855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697002+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T07:01:21.697002+0000","last_peered":"2024-06-06T07:01:21.697002+0000","last_clean":"2024-06-06T07:01:21.697002+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T07:01:21.697002+0000","last_undegraded":"2024-06-06T07:01:21.697002+0000","last_fullsized":"2024-06-06T07:01:21.697002+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796885+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T07:01:21.796885+0000","last_peered":"2024-06-06T07:01:21.796885+0000","last_clean":"2024-06-06T07:01:21.796885+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T07:01:21.796885+0000","last_undegraded":"2024-06-06T07:01:21.796885+0000","last_fullsized":"2024-06-06T07:01:21.796885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959797+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T07:01:21.959797+0000","last_peered":"2024-06-06T07:01:21.959797+0000","last_clean":"2024-06-06T07:01:21.959797+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T07:01:21.959797+0000","last_undegraded":"2024-06-06T07:01:21.959797+0000","last_fullsized":"2024-06-06T07:01:21.959797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949681+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T07:01:21.949681+0000","last_peered":"2024-06-06T07:01:21.949681+0000","last_clean":"2024-06-06T07:01:21.949681+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T07:01:21.949681+0000","last_undegraded":"2024-06-06T07:01:21.949681+0000","last_fullsized":"2024-06-06T07:01:21.949681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.420884+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T07:01:22.420884+0000","last_peered":"2024-06-06T07:01:22.420884+0000","last_clean":"2024-06-06T07:01:22.420884+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T07:01:22.420884+0000","last_undegraded":"2024-06-06T07:01:22.420884+0000","last_fullsized":"2024-06-06T07:01:22.420884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797826+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T07:01:21.797826+0000","last_peered":"2024-06-06T07:01:21.797826+0000","last_clean":"2024-06-06T07:01:21.797826+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T07:01:21.797826+0000","last_undegraded":"2024-06-06T07:01:21.797826+0000","last_fullsized":"2024-06-06T07:01:21.797826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949910+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T07:01:21.949910+0000","last_peered":"2024-06-06T07:01:21.949910+0000","last_clean":"2024-06-06T07:01:21.949910+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T07:01:21.949910+0000","last_undegraded":"2024-06-06T07:01:21.949910+0000","last_fullsized":"2024-06-06T07:01:21.949910+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796968+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T07:01:21.796968+0000","last_peered":"2024-06-06T07:01:21.796968+0000","last_clean":"2024-06-06T07:01:21.796968+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T07:01:21.796968+0000","last_undegraded":"2024-06-06T07:01:21.796968+0000","last_fullsized":"2024-06-06T07:01:21.796968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950164+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T07:01:21.950164+0000","last_peered":"2024-06-06T07:01:21.950164+0000","last_clean":"2024-06-06T07:01:21.950164+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T07:01:21.950164+0000","last_undegraded":"2024-06-06T07:01:21.950164+0000","last_fullsized":"2024-06-06T07:01:21.950164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954886+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T07:01:21.954886+0000","last_peered":"2024-06-06T07:01:21.954886+0000","last_clean":"2024-06-06T07:01:21.954886+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T07:01:21.954886+0000","last_undegraded":"2024-06-06T07:01:21.954886+0000","last_fullsized":"2024-06-06T07:01:21.954886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680416+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T07:01:22.680416+0000","last_peered":"2024-06-06T07:01:22.680416+0000","last_clean":"2024-06-06T07:01:22.680416+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T07:01:22.680416+0000","last_undegraded":"2024-06-06T07:01:22.680416+0000","last_fullsized":"2024-06-06T07:01:22.680416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2583,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950755+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T07:01:21.950755+0000","last_peered":"2024-06-06T07:01:21.950755+0000","last_clean":"2024-06-06T07:01:21.950755+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T07:01:21.950755+0000","last_undegraded":"2024-06-06T07:01:21.950755+0000","last_fullsized":"2024-06-06T07:01:21.950755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959977+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T07:01:21.959977+0000","last_peered":"2024-06-06T07:01:21.959977+0000","last_clean":"2024-06-06T07:01:21.959977+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T07:01:21.959977+0000","last_undegraded":"2024-06-06T07:01:21.959977+0000","last_fullsized":"2024-06-06T07:01:21.959977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"101'41","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.904525+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T07:01:21.904525+0000","last_peered":"2024-06-06T07:01:21.904525+0000","last_clean":"2024-06-06T07:01:21.904525+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T07:01:21.904525+0000","last_undegraded":"2024-06-06T07:01:21.904525+0000","last_fullsized":"2024-06-06T07:01:21.904525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950160+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.950160+0000","last_peered":"2024-06-06T07:01:21.950160+0000","last_clean":"2024-06-06T07:01:21.950160+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T07:01:21.950160+0000","last_undegraded":"2024-06-06T07:01:21.950160+0000","last_fullsized":"2024-06-06T07:01:21.950160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697415+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T07:01:21.697415+0000","last_peered":"2024-06-06T07:01:21.697415+0000","last_clean":"2024-06-06T07:01:21.697415+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T07:01:21.697415+0000","last_undegraded":"2024-06-06T07:01:21.697415+0000","last_fullsized":"2024-06-06T07:01:21.697415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955485+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T07:01:21.955485+0000","last_peered":"2024-06-06T07:01:21.955485+0000","last_clean":"2024-06-06T07:01:21.955485+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T07:01:21.955485+0000","last_undegraded":"2024-06-06T07:01:21.955485+0000","last_fullsized":"2024-06-06T07:01:21.955485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951082+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T07:01:21.951082+0000","last_peered":"2024-06-06T07:01:21.951082+0000","last_clean":"2024-06-06T07:01:21.951082+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T07:01:21.951082+0000","last_undegraded":"2024-06-06T07:01:21.951082+0000","last_fullsized":"2024-06-06T07:01:21.951082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959867+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T07:01:21.959867+0000","last_peered":"2024-06-06T07:01:21.959867+0000","last_clean":"2024-06-06T07:01:21.959867+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T07:01:21.959867+0000","last_undegraded":"2024-06-06T07:01:21.959867+0000","last_fullsized":"2024-06-06T07:01:21.959867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955299+0000","last_change":"2024-06-06T06:58:53.478114+0000","last_active":"2024-06-06T07:01:21.955299+0000","last_peered":"2024-06-06T07:01:21.955299+0000","last_clean":"2024-06-06T07:01:21.955299+0000","last_became_active":"2024-06-06T06:58:53.477664+0000","last_became_peered":"2024-06-06T06:58:53.477664+0000","last_unstale":"2024-06-06T07:01:21.955299+0000","last_undegraded":"2024-06-06T07:01:21.955299+0000","last_fullsized":"2024-06-06T07:01:21.955299+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:01.541804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955202+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T07:01:21.955202+0000","last_peered":"2024-06-06T07:01:21.955202+0000","last_clean":"2024-06-06T07:01:21.955202+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T07:01:21.955202+0000","last_undegraded":"2024-06-06T07:01:21.955202+0000","last_fullsized":"2024-06-06T07:01:21.955202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960342+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T07:01:21.960342+0000","last_peered":"2024-06-06T07:01:21.960342+0000","last_clean":"2024-06-06T07:01:21.960342+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T07:01:21.960342+0000","last_undegraded":"2024-06-06T07:01:21.960342+0000","last_fullsized":"2024-06-06T07:01:21.960342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955336+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T07:01:21.955336+0000","last_peered":"2024-06-06T07:01:21.955336+0000","last_clean":"2024-06-06T07:01:21.955336+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T07:01:21.955336+0000","last_undegraded":"2024-06-06T07:01:21.955336+0000","last_fullsized":"2024-06-06T07:01:21.955336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960225+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T07:01:21.960225+0000","last_peered":"2024-06-06T07:01:21.960225+0000","last_clean":"2024-06-06T07:01:21.960225+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T07:01:21.960225+0000","last_undegraded":"2024-06-06T07:01:21.960225+0000","last_fullsized":"2024-06-06T07:01:21.960225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955704+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T07:01:21.955704+0000","last_peered":"2024-06-06T07:01:21.955704+0000","last_clean":"2024-06-06T07:01:21.955704+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T07:01:21.955704+0000","last_undegraded":"2024-06-06T07:01:21.955704+0000","last_fullsized":"2024-06-06T07:01:21.955704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696665+0000","last_change":"2024-06-06T06:58:53.478046+0000","last_active":"2024-06-06T07:01:21.696665+0000","last_peered":"2024-06-06T07:01:21.696665+0000","last_clean":"2024-06-06T07:01:21.696665+0000","last_became_active":"2024-06-06T06:58:53.477722+0000","last_became_peered":"2024-06-06T06:58:53.477722+0000","last_unstale":"2024-06-06T07:01:21.696665+0000","last_undegraded":"2024-06-06T07:01:21.696665+0000","last_fullsized":"2024-06-06T07:01:21.696665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:22.491510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697616+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T07:01:21.697616+0000","last_peered":"2024-06-06T07:01:21.697616+0000","last_clean":"2024-06-06T07:01:21.697616+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T07:01:21.697616+0000","last_undegraded":"2024-06-06T07:01:21.697616+0000","last_fullsized":"2024-06-06T07:01:21.697616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797013+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T07:01:21.797013+0000","last_peered":"2024-06-06T07:01:21.797013+0000","last_clean":"2024-06-06T07:01:21.797013+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T07:01:21.797013+0000","last_undegraded":"2024-06-06T07:01:21.797013+0000","last_fullsized":"2024-06-06T07:01:21.797013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955072+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T07:01:21.955072+0000","last_peered":"2024-06-06T07:01:21.955072+0000","last_clean":"2024-06-06T07:01:21.955072+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955072+0000","last_undegraded":"2024-06-06T07:01:21.955072+0000","last_fullsized":"2024-06-06T07:01:21.955072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955200+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T07:01:21.955200+0000","last_peered":"2024-06-06T07:01:21.955200+0000","last_clean":"2024-06-06T07:01:21.955200+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T07:01:21.955200+0000","last_undegraded":"2024-06-06T07:01:21.955200+0000","last_fullsized":"2024-06-06T07:01:21.955200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959980+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T07:01:21.959980+0000","last_peered":"2024-06-06T07:01:21.959980+0000","last_clean":"2024-06-06T07:01:21.959980+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T07:01:21.959980+0000","last_undegraded":"2024-06-06T07:01:21.959980+0000","last_fullsized":"2024-06-06T07:01:21.959980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108969,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604588,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1986,"num_read_kb":1114,"num_write":395,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1813764,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":584969,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612679,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1298432,"data_stored":1372145,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776204,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":980798,"data_compressed":54376,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037842,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":780472,"data_compressed":44750,"data_compressed_allocated":61440,"data_compressed_original":294912,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1183147,"data_compressed":71553,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":569558,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36508,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708644,"statfs":{"total":95995035648,"available":95957651456,"internally_reserved":0,"allocated":1196032,"data_stored":649662,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":571545,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36520,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708632,"statfs":{"total":95995035648,"available":95957639168,"internally_reserved":0,"allocated":1208320,"data_stored":649791,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1103514,"data_compressed":60532,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1138688,"data_stored":585981,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36544,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708608,"statfs":{"total":95995035648,"available":95957614592,"internally_reserved":0,"allocated":1224704,"data_stored":1307644,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135302,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":203509,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:01:24.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.974+0000 7f3156c4d640 1 -- 172.21.15.73:0/94069723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150100900 msgr2=0x7f3150100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.974+0000 7f3156c4d640 1 --2- 172.21.15.73:0/94069723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150100900 0x7f3150100d20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3144009900 tx=0x7f314402a410 comp rx=0 tx=0).stop 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 -- 172.21.15.73:0/94069723 shutdown_connections 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 --2- 172.21.15.73:0/94069723 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f3150108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 --2- 172.21.15.73:0/94069723 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3150101af0 0x7f3150101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 --2- 172.21.15.73:0/94069723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150100900 0x7f3150100d20 secure :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3144009900 tx=0x7f314402a410 comp rx=0 tx=0).stop 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 -- 172.21.15.73:0/94069723 >> 172.21.15.73:0/94069723 conn(0x7f31500fbe90 msgr2=0x7f31500fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 -- 172.21.15.73:0/94069723 shutdown_connections 2024-06-06T07:01:24.976 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 -- 172.21.15.73:0/94069723 wait complete. 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 Processor -- start 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.975+0000 7f3156c4d640 1 -- start start 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f315019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f315019f4d0 0x7f31501a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f315010c400 con 0x7f3150101af0 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f315010c280 con 0x7f315019f4d0 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f3156c4d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f315010c580 con 0x7f31501024b0 2024-06-06T07:01:24.979 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42978/0 (socket says 172.21.15.73:42978) 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 -- 172.21.15.73:0/3962111388 learned_addr learned my addr 172.21.15.73:0/3962111388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f314ffff640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f315019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31551ea640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f315019f4d0 0x7f31501a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 -- 172.21.15.73:0/3962111388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 msgr2=0x7f315019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f315019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 -- 172.21.15.73:0/3962111388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f315019f4d0 msgr2=0x7f31501a18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f315019f4d0 0x7f31501a18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 -- 172.21.15.73:0/3962111388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31501a20b0 con 0x7f3150101af0 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f314ffff640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f315019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f31549e9640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3144009230 tx=0x7f31440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:24.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.976+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f314403f030 con 0x7f3150101af0 2024-06-06T07:01:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.977+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3144003cb0 con 0x7f3150101af0 2024-06-06T07:01:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.977+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31501a22e0 con 0x7f3150101af0 2024-06-06T07:01:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.977+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3144035070 con 0x7f3150101af0 2024-06-06T07:01:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.977+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31501a2780 con 0x7f3150101af0 2024-06-06T07:01:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.978+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3150107de0 con 0x7f3150101af0 2024-06-06T07:01:24.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.979+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f3144042030 con 0x7f3150101af0 2024-06-06T07:01:24.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.979+0000 7f314dffb640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 0x7f31400a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:24.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.980+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f31440efc10 con 0x7f3150101af0 2024-06-06T07:01:24.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.980+0000 7f314ffff640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 0x7f31400a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:24.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.980+0000 7f314ffff640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 0x7f31400a51f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f313c005d00 tx=0x7f313c005c70 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:24.985 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:24.984+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3144049030 con 0x7f3150101af0 2024-06-06T07:01:25.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:25.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-06-06T07:01:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:24 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:25.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-06-06T07:01:25.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.052 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:24 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: mgrmap e22: x(active, since 2s), standbys: y, z 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:24 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:25.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.206+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3150061de0 con 0x7f31400a2d30 2024-06-06T07:01:25.217 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.216+0000 7f314dffb640 1 -- 172.21.15.73:0/3962111388 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339927 (secure 0 0 0) 0x7f3150061de0 con 0x7f31400a2d30 2024-06-06T07:01:25.221 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:25.223 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T07:01:25.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 msgr2=0x7f31400a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 0x7f31400a51f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f313c005d00 tx=0x7f313c005c70 comp rx=0 tx=0).stop 2024-06-06T07:01:25.224 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 msgr2=0x7f315006ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3144009230 tx=0x7f31440042f0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 shutdown_connections 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f315019f4d0 0x7f31501a18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f31400a2d30 0x7f31400a51f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f31501024b0 0x7f315019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 --2- 172.21.15.73:0/3962111388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3150101af0 0x7f315006ab90 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 >> 172.21.15.73:0/3962111388 conn(0x7f31500fbe90 msgr2=0x7f3150104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 shutdown_connections 2024-06-06T07:01:25.225 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.220+0000 7f3156c4d640 1 -- 172.21.15.73:0/3962111388 wait complete. 2024-06-06T07:01:25.232 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-06-06T07:01:23.708112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195190,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2122,"num_read_kb":1233,"num_write":578,"num_write_kb":4421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":956,"ondisk_log_size":956,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387960,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553864,"statfs":{"total":1151940427776,"available":1151543156736,"internally_reserved":0,"allocated":14139392,"data_stored":10339226,"data_compressed":592079,"data_compressed_allocated":802816,"data_compressed_original":4284416,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989506"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960251+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T07:01:21.960251+0000","last_peered":"2024-06-06T07:01:21.960251+0000","last_clean":"2024-06-06T07:01:21.960251+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T07:01:21.960251+0000","last_undegraded":"2024-06-06T07:01:21.960251+0000","last_fullsized":"2024-06-06T07:01:21.960251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960220+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T07:01:21.960220+0000","last_peered":"2024-06-06T07:01:21.960220+0000","last_clean":"2024-06-06T07:01:21.960220+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T07:01:21.960220+0000","last_undegraded":"2024-06-06T07:01:21.960220+0000","last_fullsized":"2024-06-06T07:01:21.960220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960212+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T07:01:21.960212+0000","last_peered":"2024-06-06T07:01:21.960212+0000","last_clean":"2024-06-06T07:01:21.960212+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T07:01:21.960212+0000","last_undegraded":"2024-06-06T07:01:21.960212+0000","last_fullsized":"2024-06-06T07:01:21.960212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950531+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T07:01:21.950531+0000","last_peered":"2024-06-06T07:01:21.950531+0000","last_clean":"2024-06-06T07:01:21.950531+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T07:01:21.950531+0000","last_undegraded":"2024-06-06T07:01:21.950531+0000","last_fullsized":"2024-06-06T07:01:21.950531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797379+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T07:01:21.797379+0000","last_peered":"2024-06-06T07:01:21.797379+0000","last_clean":"2024-06-06T07:01:21.797379+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T07:01:21.797379+0000","last_undegraded":"2024-06-06T07:01:21.797379+0000","last_fullsized":"2024-06-06T07:01:21.797379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698596+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T07:01:21.698596+0000","last_peered":"2024-06-06T07:01:21.698596+0000","last_clean":"2024-06-06T07:01:21.698596+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T07:01:21.698596+0000","last_undegraded":"2024-06-06T07:01:21.698596+0000","last_fullsized":"2024-06-06T07:01:21.698596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949896+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T07:01:21.949896+0000","last_peered":"2024-06-06T07:01:21.949896+0000","last_clean":"2024-06-06T07:01:21.949896+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T07:01:21.949896+0000","last_undegraded":"2024-06-06T07:01:21.949896+0000","last_fullsized":"2024-06-06T07:01:21.949896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796424+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T07:01:21.796424+0000","last_peered":"2024-06-06T07:01:21.796424+0000","last_clean":"2024-06-06T07:01:21.796424+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T07:01:21.796424+0000","last_undegraded":"2024-06-06T07:01:21.796424+0000","last_fullsized":"2024-06-06T07:01:21.796424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955413+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T07:01:21.955413+0000","last_peered":"2024-06-06T07:01:21.955413+0000","last_clean":"2024-06-06T07:01:21.955413+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T07:01:21.955413+0000","last_undegraded":"2024-06-06T07:01:21.955413+0000","last_fullsized":"2024-06-06T07:01:21.955413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699282+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T07:01:21.699282+0000","last_peered":"2024-06-06T07:01:21.699282+0000","last_clean":"2024-06-06T07:01:21.699282+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T07:01:21.699282+0000","last_undegraded":"2024-06-06T07:01:21.699282+0000","last_fullsized":"2024-06-06T07:01:21.699282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950661+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T07:01:21.950661+0000","last_peered":"2024-06-06T07:01:21.950661+0000","last_clean":"2024-06-06T07:01:21.950661+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T07:01:21.950661+0000","last_undegraded":"2024-06-06T07:01:21.950661+0000","last_fullsized":"2024-06-06T07:01:21.950661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950393+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T07:01:21.950393+0000","last_peered":"2024-06-06T07:01:21.950393+0000","last_clean":"2024-06-06T07:01:21.950393+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T07:01:21.950393+0000","last_undegraded":"2024-06-06T07:01:21.950393+0000","last_fullsized":"2024-06-06T07:01:21.950393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955249+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T07:01:21.955249+0000","last_peered":"2024-06-06T07:01:21.955249+0000","last_clean":"2024-06-06T07:01:21.955249+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T07:01:21.955249+0000","last_undegraded":"2024-06-06T07:01:21.955249+0000","last_fullsized":"2024-06-06T07:01:21.955249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959991+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T07:01:21.959991+0000","last_peered":"2024-06-06T07:01:21.959991+0000","last_clean":"2024-06-06T07:01:21.959991+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T07:01:21.959991+0000","last_undegraded":"2024-06-06T07:01:21.959991+0000","last_fullsized":"2024-06-06T07:01:21.959991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959909+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T07:01:21.959909+0000","last_peered":"2024-06-06T07:01:21.959909+0000","last_clean":"2024-06-06T07:01:21.959909+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T07:01:21.959909+0000","last_undegraded":"2024-06-06T07:01:21.959909+0000","last_fullsized":"2024-06-06T07:01:21.959909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954785+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T07:01:21.954785+0000","last_peered":"2024-06-06T07:01:21.954785+0000","last_clean":"2024-06-06T07:01:21.954785+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T07:01:21.954785+0000","last_undegraded":"2024-06-06T07:01:21.954785+0000","last_fullsized":"2024-06-06T07:01:21.954785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960031+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T07:01:21.960031+0000","last_peered":"2024-06-06T07:01:21.960031+0000","last_clean":"2024-06-06T07:01:21.960031+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T07:01:21.960031+0000","last_undegraded":"2024-06-06T07:01:21.960031+0000","last_fullsized":"2024-06-06T07:01:21.960031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696962+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T07:01:21.696962+0000","last_peered":"2024-06-06T07:01:21.696962+0000","last_clean":"2024-06-06T07:01:21.696962+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T07:01:21.696962+0000","last_undegraded":"2024-06-06T07:01:21.696962+0000","last_fullsized":"2024-06-06T07:01:21.696962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955638+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T07:01:21.955638+0000","last_peered":"2024-06-06T07:01:21.955638+0000","last_clean":"2024-06-06T07:01:21.955638+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T07:01:21.955638+0000","last_undegraded":"2024-06-06T07:01:21.955638+0000","last_fullsized":"2024-06-06T07:01:21.955638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955116+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T07:01:21.955116+0000","last_peered":"2024-06-06T07:01:21.955116+0000","last_clean":"2024-06-06T07:01:21.955116+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T07:01:21.955116+0000","last_undegraded":"2024-06-06T07:01:21.955116+0000","last_fullsized":"2024-06-06T07:01:21.955116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696861+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T07:01:21.696861+0000","last_peered":"2024-06-06T07:01:21.696861+0000","last_clean":"2024-06-06T07:01:21.696861+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T07:01:21.696861+0000","last_undegraded":"2024-06-06T07:01:21.696861+0000","last_fullsized":"2024-06-06T07:01:21.696861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798424+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T07:01:21.798424+0000","last_peered":"2024-06-06T07:01:21.798424+0000","last_clean":"2024-06-06T07:01:21.798424+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T07:01:21.798424+0000","last_undegraded":"2024-06-06T07:01:21.798424+0000","last_fullsized":"2024-06-06T07:01:21.798424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949488+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T07:01:21.949488+0000","last_peered":"2024-06-06T07:01:21.949488+0000","last_clean":"2024-06-06T07:01:21.949488+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T07:01:21.949488+0000","last_undegraded":"2024-06-06T07:01:21.949488+0000","last_fullsized":"2024-06-06T07:01:21.949488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798829+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T07:01:21.798829+0000","last_peered":"2024-06-06T07:01:21.798829+0000","last_clean":"2024-06-06T07:01:21.798829+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T07:01:21.798829+0000","last_undegraded":"2024-06-06T07:01:21.798829+0000","last_fullsized":"2024-06-06T07:01:21.798829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960215+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T07:01:21.960215+0000","last_peered":"2024-06-06T07:01:21.960215+0000","last_clean":"2024-06-06T07:01:21.960215+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T07:01:21.960215+0000","last_undegraded":"2024-06-06T07:01:21.960215+0000","last_fullsized":"2024-06-06T07:01:21.960215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950690+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T07:01:21.950690+0000","last_peered":"2024-06-06T07:01:21.950690+0000","last_clean":"2024-06-06T07:01:21.950690+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T07:01:21.950690+0000","last_undegraded":"2024-06-06T07:01:21.950690+0000","last_fullsized":"2024-06-06T07:01:21.950690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":33,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960125+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T07:01:21.960125+0000","last_peered":"2024-06-06T07:01:21.960125+0000","last_clean":"2024-06-06T07:01:21.960125+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T07:01:21.960125+0000","last_undegraded":"2024-06-06T07:01:21.960125+0000","last_fullsized":"2024-06-06T07:01:21.960125+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955097+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T07:01:21.955097+0000","last_peered":"2024-06-06T07:01:21.955097+0000","last_clean":"2024-06-06T07:01:21.955097+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T07:01:21.955097+0000","last_undegraded":"2024-06-06T07:01:21.955097+0000","last_fullsized":"2024-06-06T07:01:21.955097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949550+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T07:01:21.949550+0000","last_peered":"2024-06-06T07:01:21.949550+0000","last_clean":"2024-06-06T07:01:21.949550+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T07:01:21.949550+0000","last_undegraded":"2024-06-06T07:01:21.949550+0000","last_fullsized":"2024-06-06T07:01:21.949550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797295+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T07:01:21.797295+0000","last_peered":"2024-06-06T07:01:21.797295+0000","last_clean":"2024-06-06T07:01:21.797295+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T07:01:21.797295+0000","last_undegraded":"2024-06-06T07:01:21.797295+0000","last_fullsized":"2024-06-06T07:01:21.797295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698810+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T07:01:21.698810+0000","last_peered":"2024-06-06T07:01:21.698810+0000","last_clean":"2024-06-06T07:01:21.698810+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T07:01:21.698810+0000","last_undegraded":"2024-06-06T07:01:21.698810+0000","last_fullsized":"2024-06-06T07:01:21.698810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955250+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T07:01:21.955250+0000","last_peered":"2024-06-06T07:01:21.955250+0000","last_clean":"2024-06-06T07:01:21.955250+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T07:01:21.955250+0000","last_undegraded":"2024-06-06T07:01:21.955250+0000","last_fullsized":"2024-06-06T07:01:21.955250+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'7","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680167+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T07:01:22.680167+0000","last_peered":"2024-06-06T07:01:22.680167+0000","last_clean":"2024-06-06T07:01:22.680167+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T07:01:22.680167+0000","last_undegraded":"2024-06-06T07:01:22.680167+0000","last_fullsized":"2024-06-06T07:01:22.680167+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796975+0000","last_change":"2024-06-06T06:58:53.476855+0000","last_active":"2024-06-06T07:01:21.796975+0000","last_peered":"2024-06-06T07:01:21.796975+0000","last_clean":"2024-06-06T07:01:21.796975+0000","last_became_active":"2024-06-06T06:58:53.476098+0000","last_became_peered":"2024-06-06T06:58:53.476098+0000","last_unstale":"2024-06-06T07:01:21.796975+0000","last_undegraded":"2024-06-06T07:01:21.796975+0000","last_fullsized":"2024-06-06T07:01:21.796975+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:07:20.623344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955349+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T07:01:21.955349+0000","last_peered":"2024-06-06T07:01:21.955349+0000","last_clean":"2024-06-06T07:01:21.955349+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T07:01:21.955349+0000","last_undegraded":"2024-06-06T07:01:21.955349+0000","last_fullsized":"2024-06-06T07:01:21.955349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698559+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T07:01:21.698559+0000","last_peered":"2024-06-06T07:01:21.698559+0000","last_clean":"2024-06-06T07:01:21.698559+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T07:01:21.698559+0000","last_undegraded":"2024-06-06T07:01:21.698559+0000","last_fullsized":"2024-06-06T07:01:21.698559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698107+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T07:01:21.698107+0000","last_peered":"2024-06-06T07:01:21.698107+0000","last_clean":"2024-06-06T07:01:21.698107+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T07:01:21.698107+0000","last_undegraded":"2024-06-06T07:01:21.698107+0000","last_fullsized":"2024-06-06T07:01:21.698107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960014+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T07:01:21.960014+0000","last_peered":"2024-06-06T07:01:21.960014+0000","last_clean":"2024-06-06T07:01:21.960014+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T07:01:21.960014+0000","last_undegraded":"2024-06-06T07:01:21.960014+0000","last_fullsized":"2024-06-06T07:01:21.960014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954890+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T07:01:21.954890+0000","last_peered":"2024-06-06T07:01:21.954890+0000","last_clean":"2024-06-06T07:01:21.954890+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T07:01:21.954890+0000","last_undegraded":"2024-06-06T07:01:21.954890+0000","last_fullsized":"2024-06-06T07:01:21.954890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796557+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T07:01:21.796557+0000","last_peered":"2024-06-06T07:01:21.796557+0000","last_clean":"2024-06-06T07:01:21.796557+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T07:01:21.796557+0000","last_undegraded":"2024-06-06T07:01:21.796557+0000","last_fullsized":"2024-06-06T07:01:21.796557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960416+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T07:01:21.960416+0000","last_peered":"2024-06-06T07:01:21.960416+0000","last_clean":"2024-06-06T07:01:21.960416+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T07:01:21.960416+0000","last_undegraded":"2024-06-06T07:01:21.960416+0000","last_fullsized":"2024-06-06T07:01:21.960416+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697614+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T07:01:21.697614+0000","last_peered":"2024-06-06T07:01:21.697614+0000","last_clean":"2024-06-06T07:01:21.697614+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T07:01:21.697614+0000","last_undegraded":"2024-06-06T07:01:21.697614+0000","last_fullsized":"2024-06-06T07:01:21.697614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696430+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T07:01:21.696430+0000","last_peered":"2024-06-06T07:01:21.696430+0000","last_clean":"2024-06-06T07:01:21.696430+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T07:01:21.696430+0000","last_undegraded":"2024-06-06T07:01:21.696430+0000","last_fullsized":"2024-06-06T07:01:21.696430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796793+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T07:01:21.796793+0000","last_peered":"2024-06-06T07:01:21.796793+0000","last_clean":"2024-06-06T07:01:21.796793+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T07:01:21.796793+0000","last_undegraded":"2024-06-06T07:01:21.796793+0000","last_fullsized":"2024-06-06T07:01:21.796793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":422,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679687+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T07:01:22.679687+0000","last_peered":"2024-06-06T07:01:22.679687+0000","last_clean":"2024-06-06T07:01:22.679687+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T07:01:22.679687+0000","last_undegraded":"2024-06-06T07:01:22.679687+0000","last_fullsized":"2024-06-06T07:01:22.679687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":368,"num_read_kb":368,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955354+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T07:01:21.955354+0000","last_peered":"2024-06-06T07:01:21.955354+0000","last_clean":"2024-06-06T07:01:21.955354+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955354+0000","last_undegraded":"2024-06-06T07:01:21.955354+0000","last_fullsized":"2024-06-06T07:01:21.955354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696386+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T07:01:21.696386+0000","last_peered":"2024-06-06T07:01:21.696386+0000","last_clean":"2024-06-06T07:01:21.696386+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T07:01:21.696386+0000","last_undegraded":"2024-06-06T07:01:21.696386+0000","last_fullsized":"2024-06-06T07:01:21.696386+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951076+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T07:01:21.951076+0000","last_peered":"2024-06-06T07:01:21.951076+0000","last_clean":"2024-06-06T07:01:21.951076+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T07:01:21.951076+0000","last_undegraded":"2024-06-06T07:01:21.951076+0000","last_fullsized":"2024-06-06T07:01:21.951076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960384+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T07:01:21.960384+0000","last_peered":"2024-06-06T07:01:21.960384+0000","last_clean":"2024-06-06T07:01:21.960384+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T07:01:21.960384+0000","last_undegraded":"2024-06-06T07:01:21.960384+0000","last_fullsized":"2024-06-06T07:01:21.960384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950898+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T07:01:21.950898+0000","last_peered":"2024-06-06T07:01:21.950898+0000","last_clean":"2024-06-06T07:01:21.950898+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T07:01:21.950898+0000","last_undegraded":"2024-06-06T07:01:21.950898+0000","last_fullsized":"2024-06-06T07:01:21.950898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698689+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T07:01:21.698689+0000","last_peered":"2024-06-06T07:01:21.698689+0000","last_clean":"2024-06-06T07:01:21.698689+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T07:01:21.698689+0000","last_undegraded":"2024-06-06T07:01:21.698689+0000","last_fullsized":"2024-06-06T07:01:21.698689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955384+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T07:01:21.955384+0000","last_peered":"2024-06-06T07:01:21.955384+0000","last_clean":"2024-06-06T07:01:21.955384+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T07:01:21.955384+0000","last_undegraded":"2024-06-06T07:01:21.955384+0000","last_fullsized":"2024-06-06T07:01:21.955384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696457+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T07:01:21.696457+0000","last_peered":"2024-06-06T07:01:21.696457+0000","last_clean":"2024-06-06T07:01:21.696457+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T07:01:21.696457+0000","last_undegraded":"2024-06-06T07:01:21.696457+0000","last_fullsized":"2024-06-06T07:01:21.696457+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":469,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679951+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T07:01:22.679951+0000","last_peered":"2024-06-06T07:01:22.679951+0000","last_clean":"2024-06-06T07:01:22.679951+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T07:01:22.679951+0000","last_undegraded":"2024-06-06T07:01:22.679951+0000","last_fullsized":"2024-06-06T07:01:22.679951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68760,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":380,"num_read_kb":257,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797184+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T07:01:21.797184+0000","last_peered":"2024-06-06T07:01:21.797184+0000","last_clean":"2024-06-06T07:01:21.797184+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T07:01:21.797184+0000","last_undegraded":"2024-06-06T07:01:21.797184+0000","last_fullsized":"2024-06-06T07:01:21.797184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699244+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T07:01:21.699244+0000","last_peered":"2024-06-06T07:01:21.699244+0000","last_clean":"2024-06-06T07:01:21.699244+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T07:01:21.699244+0000","last_undegraded":"2024-06-06T07:01:21.699244+0000","last_fullsized":"2024-06-06T07:01:21.699244+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955406+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T07:01:21.955406+0000","last_peered":"2024-06-06T07:01:21.955406+0000","last_clean":"2024-06-06T07:01:21.955406+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T07:01:21.955406+0000","last_undegraded":"2024-06-06T07:01:21.955406+0000","last_fullsized":"2024-06-06T07:01:21.955406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954910+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T07:01:21.954910+0000","last_peered":"2024-06-06T07:01:21.954910+0000","last_clean":"2024-06-06T07:01:21.954910+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T07:01:21.954910+0000","last_undegraded":"2024-06-06T07:01:21.954910+0000","last_fullsized":"2024-06-06T07:01:21.954910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959844+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T07:01:21.959844+0000","last_peered":"2024-06-06T07:01:21.959844+0000","last_clean":"2024-06-06T07:01:21.959844+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T07:01:21.959844+0000","last_undegraded":"2024-06-06T07:01:21.959844+0000","last_fullsized":"2024-06-06T07:01:21.959844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697867+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T07:01:21.697867+0000","last_peered":"2024-06-06T07:01:21.697867+0000","last_clean":"2024-06-06T07:01:21.697867+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T07:01:21.697867+0000","last_undegraded":"2024-06-06T07:01:21.697867+0000","last_fullsized":"2024-06-06T07:01:21.697867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950431+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T07:01:21.950431+0000","last_peered":"2024-06-06T07:01:21.950431+0000","last_clean":"2024-06-06T07:01:21.950431+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T07:01:21.950431+0000","last_undegraded":"2024-06-06T07:01:21.950431+0000","last_fullsized":"2024-06-06T07:01:21.950431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697123+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T07:01:21.697123+0000","last_peered":"2024-06-06T07:01:21.697123+0000","last_clean":"2024-06-06T07:01:21.697123+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T07:01:21.697123+0000","last_undegraded":"2024-06-06T07:01:21.697123+0000","last_fullsized":"2024-06-06T07:01:21.697123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679778+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T07:01:22.679778+0000","last_peered":"2024-06-06T07:01:22.679778+0000","last_clean":"2024-06-06T07:01:22.679778+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T07:01:22.679778+0000","last_undegraded":"2024-06-06T07:01:22.679778+0000","last_fullsized":"2024-06-06T07:01:22.679778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2001,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949816+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T07:01:21.949816+0000","last_peered":"2024-06-06T07:01:21.949816+0000","last_clean":"2024-06-06T07:01:21.949816+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T07:01:21.949816+0000","last_undegraded":"2024-06-06T07:01:21.949816+0000","last_fullsized":"2024-06-06T07:01:21.949816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955616+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T07:01:21.955616+0000","last_peered":"2024-06-06T07:01:21.955616+0000","last_clean":"2024-06-06T07:01:21.955616+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T07:01:21.955616+0000","last_undegraded":"2024-06-06T07:01:21.955616+0000","last_fullsized":"2024-06-06T07:01:21.955616+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796849+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T07:01:21.796849+0000","last_peered":"2024-06-06T07:01:21.796849+0000","last_clean":"2024-06-06T07:01:21.796849+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T07:01:21.796849+0000","last_undegraded":"2024-06-06T07:01:21.796849+0000","last_fullsized":"2024-06-06T07:01:21.796849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697500+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T07:01:21.697500+0000","last_peered":"2024-06-06T07:01:21.697500+0000","last_clean":"2024-06-06T07:01:21.697500+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T07:01:21.697500+0000","last_undegraded":"2024-06-06T07:01:21.697500+0000","last_fullsized":"2024-06-06T07:01:21.697500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697474+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T07:01:21.697474+0000","last_peered":"2024-06-06T07:01:21.697474+0000","last_clean":"2024-06-06T07:01:21.697474+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T07:01:21.697474+0000","last_undegraded":"2024-06-06T07:01:21.697474+0000","last_fullsized":"2024-06-06T07:01:21.697474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797233+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T07:01:21.797233+0000","last_peered":"2024-06-06T07:01:21.797233+0000","last_clean":"2024-06-06T07:01:21.797233+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T07:01:21.797233+0000","last_undegraded":"2024-06-06T07:01:21.797233+0000","last_fullsized":"2024-06-06T07:01:21.797233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960037+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T07:01:21.960037+0000","last_peered":"2024-06-06T07:01:21.960037+0000","last_clean":"2024-06-06T07:01:21.960037+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T07:01:21.960037+0000","last_undegraded":"2024-06-06T07:01:21.960037+0000","last_fullsized":"2024-06-06T07:01:21.960037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797116+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T07:01:21.797116+0000","last_peered":"2024-06-06T07:01:21.797116+0000","last_clean":"2024-06-06T07:01:21.797116+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T07:01:21.797116+0000","last_undegraded":"2024-06-06T07:01:21.797116+0000","last_fullsized":"2024-06-06T07:01:21.797116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696557+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T07:01:21.696557+0000","last_peered":"2024-06-06T07:01:21.696557+0000","last_clean":"2024-06-06T07:01:21.696557+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T07:01:21.696557+0000","last_undegraded":"2024-06-06T07:01:21.696557+0000","last_fullsized":"2024-06-06T07:01:21.696557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698895+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.698895+0000","last_peered":"2024-06-06T07:01:21.698895+0000","last_clean":"2024-06-06T07:01:21.698895+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T07:01:21.698895+0000","last_undegraded":"2024-06-06T07:01:21.698895+0000","last_fullsized":"2024-06-06T07:01:21.698895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955129+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T07:01:21.955129+0000","last_peered":"2024-06-06T07:01:21.955129+0000","last_clean":"2024-06-06T07:01:21.955129+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T07:01:21.955129+0000","last_undegraded":"2024-06-06T07:01:21.955129+0000","last_fullsized":"2024-06-06T07:01:21.955129+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":241,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680327+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T07:01:22.680327+0000","last_peered":"2024-06-06T07:01:22.680327+0000","last_clean":"2024-06-06T07:01:22.680327+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T07:01:22.680327+0000","last_undegraded":"2024-06-06T07:01:22.680327+0000","last_fullsized":"2024-06-06T07:01:22.680327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698794+0000","last_change":"2024-06-06T06:58:53.571939+0000","last_active":"2024-06-06T07:01:21.698794+0000","last_peered":"2024-06-06T07:01:21.698794+0000","last_clean":"2024-06-06T07:01:21.698794+0000","last_became_active":"2024-06-06T06:58:53.571590+0000","last_became_peered":"2024-06-06T06:58:53.571590+0000","last_unstale":"2024-06-06T07:01:21.698794+0000","last_undegraded":"2024-06-06T07:01:21.698794+0000","last_fullsized":"2024-06-06T07:01:21.698794+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:10.858649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955640+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T07:01:21.955640+0000","last_peered":"2024-06-06T07:01:21.955640+0000","last_clean":"2024-06-06T07:01:21.955640+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T07:01:21.955640+0000","last_undegraded":"2024-06-06T07:01:21.955640+0000","last_fullsized":"2024-06-06T07:01:21.955640+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697837+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T07:01:21.697837+0000","last_peered":"2024-06-06T07:01:21.697837+0000","last_clean":"2024-06-06T07:01:21.697837+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T07:01:21.697837+0000","last_undegraded":"2024-06-06T07:01:21.697837+0000","last_fullsized":"2024-06-06T07:01:21.697837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679259+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T07:01:22.679259+0000","last_peered":"2024-06-06T07:01:22.679259+0000","last_clean":"2024-06-06T07:01:22.679259+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T07:01:22.679259+0000","last_undegraded":"2024-06-06T07:01:22.679259+0000","last_fullsized":"2024-06-06T07:01:22.679259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949771+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T07:01:21.949771+0000","last_peered":"2024-06-06T07:01:21.949771+0000","last_clean":"2024-06-06T07:01:21.949771+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T07:01:21.949771+0000","last_undegraded":"2024-06-06T07:01:21.949771+0000","last_fullsized":"2024-06-06T07:01:21.949771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797356+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T07:01:21.797356+0000","last_peered":"2024-06-06T07:01:21.797356+0000","last_clean":"2024-06-06T07:01:21.797356+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T07:01:21.797356+0000","last_undegraded":"2024-06-06T07:01:21.797356+0000","last_fullsized":"2024-06-06T07:01:21.797356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950845+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T07:01:21.950845+0000","last_peered":"2024-06-06T07:01:21.950845+0000","last_clean":"2024-06-06T07:01:21.950845+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T07:01:21.950845+0000","last_undegraded":"2024-06-06T07:01:21.950845+0000","last_fullsized":"2024-06-06T07:01:21.950845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":379,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949403+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T07:01:21.949403+0000","last_peered":"2024-06-06T07:01:21.949403+0000","last_clean":"2024-06-06T07:01:21.949403+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T07:01:21.949403+0000","last_undegraded":"2024-06-06T07:01:21.949403+0000","last_fullsized":"2024-06-06T07:01:21.949403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698791+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T07:01:21.698791+0000","last_peered":"2024-06-06T07:01:21.698791+0000","last_clean":"2024-06-06T07:01:21.698791+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T07:01:21.698791+0000","last_undegraded":"2024-06-06T07:01:21.698791+0000","last_fullsized":"2024-06-06T07:01:21.698791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954789+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T07:01:21.954789+0000","last_peered":"2024-06-06T07:01:21.954789+0000","last_clean":"2024-06-06T07:01:21.954789+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T07:01:21.954789+0000","last_undegraded":"2024-06-06T07:01:21.954789+0000","last_fullsized":"2024-06-06T07:01:21.954789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949695+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T07:01:21.949695+0000","last_peered":"2024-06-06T07:01:21.949695+0000","last_clean":"2024-06-06T07:01:21.949695+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T07:01:21.949695+0000","last_undegraded":"2024-06-06T07:01:21.949695+0000","last_fullsized":"2024-06-06T07:01:21.949695+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959816+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T07:01:21.959816+0000","last_peered":"2024-06-06T07:01:21.959816+0000","last_clean":"2024-06-06T07:01:21.959816+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T07:01:21.959816+0000","last_undegraded":"2024-06-06T07:01:21.959816+0000","last_fullsized":"2024-06-06T07:01:21.959816+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797392+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T07:01:21.797392+0000","last_peered":"2024-06-06T07:01:21.797392+0000","last_clean":"2024-06-06T07:01:21.797392+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T07:01:21.797392+0000","last_undegraded":"2024-06-06T07:01:21.797392+0000","last_fullsized":"2024-06-06T07:01:21.797392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955313+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T07:01:21.955313+0000","last_peered":"2024-06-06T07:01:21.955313+0000","last_clean":"2024-06-06T07:01:21.955313+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T07:01:21.955313+0000","last_undegraded":"2024-06-06T07:01:21.955313+0000","last_fullsized":"2024-06-06T07:01:21.955313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796559+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T07:01:21.796559+0000","last_peered":"2024-06-06T07:01:21.796559+0000","last_clean":"2024-06-06T07:01:21.796559+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T07:01:21.796559+0000","last_undegraded":"2024-06-06T07:01:21.796559+0000","last_fullsized":"2024-06-06T07:01:21.796559+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'52","reported_seq":123,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.909087+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T07:01:21.909087+0000","last_peered":"2024-06-06T07:01:21.909087+0000","last_clean":"2024-06-06T07:01:21.909087+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T07:01:21.909087+0000","last_undegraded":"2024-06-06T07:01:21.909087+0000","last_fullsized":"2024-06-06T07:01:21.909087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796688+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T07:01:21.796688+0000","last_peered":"2024-06-06T07:01:21.796688+0000","last_clean":"2024-06-06T07:01:21.796688+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T07:01:21.796688+0000","last_undegraded":"2024-06-06T07:01:21.796688+0000","last_fullsized":"2024-06-06T07:01:21.796688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954998+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T07:01:21.954998+0000","last_peered":"2024-06-06T07:01:21.954998+0000","last_clean":"2024-06-06T07:01:21.954998+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T07:01:21.954998+0000","last_undegraded":"2024-06-06T07:01:21.954998+0000","last_fullsized":"2024-06-06T07:01:21.954998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950264+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T07:01:21.950264+0000","last_peered":"2024-06-06T07:01:21.950264+0000","last_clean":"2024-06-06T07:01:21.950264+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T07:01:21.950264+0000","last_undegraded":"2024-06-06T07:01:21.950264+0000","last_fullsized":"2024-06-06T07:01:21.950264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960301+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T07:01:21.960301+0000","last_peered":"2024-06-06T07:01:21.960301+0000","last_clean":"2024-06-06T07:01:21.960301+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T07:01:21.960301+0000","last_undegraded":"2024-06-06T07:01:21.960301+0000","last_fullsized":"2024-06-06T07:01:21.960301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950031+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T07:01:21.950031+0000","last_peered":"2024-06-06T07:01:21.950031+0000","last_clean":"2024-06-06T07:01:21.950031+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T07:01:21.950031+0000","last_undegraded":"2024-06-06T07:01:21.950031+0000","last_fullsized":"2024-06-06T07:01:21.950031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954812+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T07:01:21.954812+0000","last_peered":"2024-06-06T07:01:21.954812+0000","last_clean":"2024-06-06T07:01:21.954812+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T07:01:21.954812+0000","last_undegraded":"2024-06-06T07:01:21.954812+0000","last_fullsized":"2024-06-06T07:01:21.954812+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796855+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T07:01:21.796855+0000","last_peered":"2024-06-06T07:01:21.796855+0000","last_clean":"2024-06-06T07:01:21.796855+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T07:01:21.796855+0000","last_undegraded":"2024-06-06T07:01:21.796855+0000","last_fullsized":"2024-06-06T07:01:21.796855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697002+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T07:01:21.697002+0000","last_peered":"2024-06-06T07:01:21.697002+0000","last_clean":"2024-06-06T07:01:21.697002+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T07:01:21.697002+0000","last_undegraded":"2024-06-06T07:01:21.697002+0000","last_fullsized":"2024-06-06T07:01:21.697002+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796885+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T07:01:21.796885+0000","last_peered":"2024-06-06T07:01:21.796885+0000","last_clean":"2024-06-06T07:01:21.796885+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T07:01:21.796885+0000","last_undegraded":"2024-06-06T07:01:21.796885+0000","last_fullsized":"2024-06-06T07:01:21.796885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959797+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T07:01:21.959797+0000","last_peered":"2024-06-06T07:01:21.959797+0000","last_clean":"2024-06-06T07:01:21.959797+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T07:01:21.959797+0000","last_undegraded":"2024-06-06T07:01:21.959797+0000","last_fullsized":"2024-06-06T07:01:21.959797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949681+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T07:01:21.949681+0000","last_peered":"2024-06-06T07:01:21.949681+0000","last_clean":"2024-06-06T07:01:21.949681+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T07:01:21.949681+0000","last_undegraded":"2024-06-06T07:01:21.949681+0000","last_fullsized":"2024-06-06T07:01:21.949681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.420884+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T07:01:22.420884+0000","last_peered":"2024-06-06T07:01:22.420884+0000","last_clean":"2024-06-06T07:01:22.420884+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T07:01:22.420884+0000","last_undegraded":"2024-06-06T07:01:22.420884+0000","last_fullsized":"2024-06-06T07:01:22.420884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797826+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T07:01:21.797826+0000","last_peered":"2024-06-06T07:01:21.797826+0000","last_clean":"2024-06-06T07:01:21.797826+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T07:01:21.797826+0000","last_undegraded":"2024-06-06T07:01:21.797826+0000","last_fullsized":"2024-06-06T07:01:21.797826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949910+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T07:01:21.949910+0000","last_peered":"2024-06-06T07:01:21.949910+0000","last_clean":"2024-06-06T07:01:21.949910+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T07:01:21.949910+0000","last_undegraded":"2024-06-06T07:01:21.949910+0000","last_fullsized":"2024-06-06T07:01:21.949910+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796968+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T07:01:21.796968+0000","last_peered":"2024-06-06T07:01:21.796968+0000","last_clean":"2024-06-06T07:01:21.796968+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T07:01:21.796968+0000","last_undegraded":"2024-06-06T07:01:21.796968+0000","last_fullsized":"2024-06-06T07:01:21.796968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950164+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T07:01:21.950164+0000","last_peered":"2024-06-06T07:01:21.950164+0000","last_clean":"2024-06-06T07:01:21.950164+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T07:01:21.950164+0000","last_undegraded":"2024-06-06T07:01:21.950164+0000","last_fullsized":"2024-06-06T07:01:21.950164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954886+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T07:01:21.954886+0000","last_peered":"2024-06-06T07:01:21.954886+0000","last_clean":"2024-06-06T07:01:21.954886+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T07:01:21.954886+0000","last_undegraded":"2024-06-06T07:01:21.954886+0000","last_fullsized":"2024-06-06T07:01:21.954886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680416+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T07:01:22.680416+0000","last_peered":"2024-06-06T07:01:22.680416+0000","last_clean":"2024-06-06T07:01:22.680416+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T07:01:22.680416+0000","last_undegraded":"2024-06-06T07:01:22.680416+0000","last_fullsized":"2024-06-06T07:01:22.680416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2583,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950755+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T07:01:21.950755+0000","last_peered":"2024-06-06T07:01:21.950755+0000","last_clean":"2024-06-06T07:01:21.950755+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T07:01:21.950755+0000","last_undegraded":"2024-06-06T07:01:21.950755+0000","last_fullsized":"2024-06-06T07:01:21.950755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959977+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T07:01:21.959977+0000","last_peered":"2024-06-06T07:01:21.959977+0000","last_clean":"2024-06-06T07:01:21.959977+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T07:01:21.959977+0000","last_undegraded":"2024-06-06T07:01:21.959977+0000","last_fullsized":"2024-06-06T07:01:21.959977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"101'41","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.904525+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T07:01:21.904525+0000","last_peered":"2024-06-06T07:01:21.904525+0000","last_clean":"2024-06-06T07:01:21.904525+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T07:01:21.904525+0000","last_undegraded":"2024-06-06T07:01:21.904525+0000","last_fullsized":"2024-06-06T07:01:21.904525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950160+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.950160+0000","last_peered":"2024-06-06T07:01:21.950160+0000","last_clean":"2024-06-06T07:01:21.950160+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T07:01:21.950160+0000","last_undegraded":"2024-06-06T07:01:21.950160+0000","last_fullsized":"2024-06-06T07:01:21.950160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697415+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T07:01:21.697415+0000","last_peered":"2024-06-06T07:01:21.697415+0000","last_clean":"2024-06-06T07:01:21.697415+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T07:01:21.697415+0000","last_undegraded":"2024-06-06T07:01:21.697415+0000","last_fullsized":"2024-06-06T07:01:21.697415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955485+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T07:01:21.955485+0000","last_peered":"2024-06-06T07:01:21.955485+0000","last_clean":"2024-06-06T07:01:21.955485+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T07:01:21.955485+0000","last_undegraded":"2024-06-06T07:01:21.955485+0000","last_fullsized":"2024-06-06T07:01:21.955485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951082+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T07:01:21.951082+0000","last_peered":"2024-06-06T07:01:21.951082+0000","last_clean":"2024-06-06T07:01:21.951082+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T07:01:21.951082+0000","last_undegraded":"2024-06-06T07:01:21.951082+0000","last_fullsized":"2024-06-06T07:01:21.951082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959867+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T07:01:21.959867+0000","last_peered":"2024-06-06T07:01:21.959867+0000","last_clean":"2024-06-06T07:01:21.959867+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T07:01:21.959867+0000","last_undegraded":"2024-06-06T07:01:21.959867+0000","last_fullsized":"2024-06-06T07:01:21.959867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955299+0000","last_change":"2024-06-06T06:58:53.478114+0000","last_active":"2024-06-06T07:01:21.955299+0000","last_peered":"2024-06-06T07:01:21.955299+0000","last_clean":"2024-06-06T07:01:21.955299+0000","last_became_active":"2024-06-06T06:58:53.477664+0000","last_became_peered":"2024-06-06T06:58:53.477664+0000","last_unstale":"2024-06-06T07:01:21.955299+0000","last_undegraded":"2024-06-06T07:01:21.955299+0000","last_fullsized":"2024-06-06T07:01:21.955299+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:01.541804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955202+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T07:01:21.955202+0000","last_peered":"2024-06-06T07:01:21.955202+0000","last_clean":"2024-06-06T07:01:21.955202+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T07:01:21.955202+0000","last_undegraded":"2024-06-06T07:01:21.955202+0000","last_fullsized":"2024-06-06T07:01:21.955202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960342+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T07:01:21.960342+0000","last_peered":"2024-06-06T07:01:21.960342+0000","last_clean":"2024-06-06T07:01:21.960342+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T07:01:21.960342+0000","last_undegraded":"2024-06-06T07:01:21.960342+0000","last_fullsized":"2024-06-06T07:01:21.960342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955336+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T07:01:21.955336+0000","last_peered":"2024-06-06T07:01:21.955336+0000","last_clean":"2024-06-06T07:01:21.955336+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T07:01:21.955336+0000","last_undegraded":"2024-06-06T07:01:21.955336+0000","last_fullsized":"2024-06-06T07:01:21.955336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960225+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T07:01:21.960225+0000","last_peered":"2024-06-06T07:01:21.960225+0000","last_clean":"2024-06-06T07:01:21.960225+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T07:01:21.960225+0000","last_undegraded":"2024-06-06T07:01:21.960225+0000","last_fullsized":"2024-06-06T07:01:21.960225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955704+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T07:01:21.955704+0000","last_peered":"2024-06-06T07:01:21.955704+0000","last_clean":"2024-06-06T07:01:21.955704+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T07:01:21.955704+0000","last_undegraded":"2024-06-06T07:01:21.955704+0000","last_fullsized":"2024-06-06T07:01:21.955704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696665+0000","last_change":"2024-06-06T06:58:53.478046+0000","last_active":"2024-06-06T07:01:21.696665+0000","last_peered":"2024-06-06T07:01:21.696665+0000","last_clean":"2024-06-06T07:01:21.696665+0000","last_became_active":"2024-06-06T06:58:53.477722+0000","last_became_peered":"2024-06-06T06:58:53.477722+0000","last_unstale":"2024-06-06T07:01:21.696665+0000","last_undegraded":"2024-06-06T07:01:21.696665+0000","last_fullsized":"2024-06-06T07:01:21.696665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:22.491510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697616+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T07:01:21.697616+0000","last_peered":"2024-06-06T07:01:21.697616+0000","last_clean":"2024-06-06T07:01:21.697616+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T07:01:21.697616+0000","last_undegraded":"2024-06-06T07:01:21.697616+0000","last_fullsized":"2024-06-06T07:01:21.697616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797013+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T07:01:21.797013+0000","last_peered":"2024-06-06T07:01:21.797013+0000","last_clean":"2024-06-06T07:01:21.797013+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T07:01:21.797013+0000","last_undegraded":"2024-06-06T07:01:21.797013+0000","last_fullsized":"2024-06-06T07:01:21.797013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955072+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T07:01:21.955072+0000","last_peered":"2024-06-06T07:01:21.955072+0000","last_clean":"2024-06-06T07:01:21.955072+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955072+0000","last_undegraded":"2024-06-06T07:01:21.955072+0000","last_fullsized":"2024-06-06T07:01:21.955072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955200+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T07:01:21.955200+0000","last_peered":"2024-06-06T07:01:21.955200+0000","last_clean":"2024-06-06T07:01:21.955200+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T07:01:21.955200+0000","last_undegraded":"2024-06-06T07:01:21.955200+0000","last_fullsized":"2024-06-06T07:01:21.955200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959980+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T07:01:21.959980+0000","last_peered":"2024-06-06T07:01:21.959980+0000","last_clean":"2024-06-06T07:01:21.959980+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T07:01:21.959980+0000","last_undegraded":"2024-06-06T07:01:21.959980+0000","last_fullsized":"2024-06-06T07:01:21.959980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108969,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604588,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1986,"num_read_kb":1114,"num_write":395,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1813764,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":584969,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612679,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1298432,"data_stored":1372145,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776204,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":980798,"data_compressed":54376,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037842,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":780472,"data_compressed":44750,"data_compressed_allocated":61440,"data_compressed_original":294912,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1183147,"data_compressed":71553,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":569558,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36508,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708644,"statfs":{"total":95995035648,"available":95957651456,"internally_reserved":0,"allocated":1196032,"data_stored":649662,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":571545,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36520,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708632,"statfs":{"total":95995035648,"available":95957639168,"internally_reserved":0,"allocated":1208320,"data_stored":649791,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1103514,"data_compressed":60532,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1138688,"data_stored":585981,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36544,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708608,"statfs":{"total":95995035648,"available":95957614592,"internally_reserved":0,"allocated":1224704,"data_stored":1307644,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135302,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":203509,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:01:25.238 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-06T07:01:25.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 -- 172.21.15.73:0/2650735878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80101b50 msgr2=0x7f1d80101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/2650735878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80101b50 0x7f1d80101fd0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f1d6800d910 tx=0x7f1d6802c380 comp rx=0 tx=0).stop 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 -- 172.21.15.73:0/2650735878 shutdown_connections 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/2650735878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 0x7f1d80108da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/2650735878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80101b50 0x7f1d80101fd0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/2650735878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80100960 0x7f1d80100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 -- 172.21.15.73:0/2650735878 >> 172.21.15.73:0/2650735878 conn(0x7f1d800fbef0 msgr2=0x7f1d800fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 -- 172.21.15.73:0/2650735878 shutdown_connections 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.399+0000 7f1d85b7e640 1 -- 172.21.15.73:0/2650735878 wait complete. 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 Processor -- start 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 -- start start 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80101b50 0x7f1d80195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 0x7f1d8019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010b320 con 0x7f1d80100960 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010b1a0 con 0x7f1d80102510 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d85b7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010b4a0 con 0x7f1d80101b50 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:25.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42998/0 (socket says 172.21.15.73:42998) 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 -- 172.21.15.73:0/3948075124 learned_addr learned my addr 172.21.15.73:0/3948075124 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d77fff640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80101b50 0x7f1d80195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7ffff640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 0x7f1d8019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 -- 172.21.15.73:0/3948075124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80101b50 msgr2=0x7f1d80195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80101b50 0x7f1d80195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 -- 172.21.15.73:0/3948075124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 msgr2=0x7f1d8019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 0x7f1d8019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.400+0000 7f1d7f7fe640 1 -- 172.21.15.73:0/3948075124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d8019d610 con 0x7f1d80100960 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d7f7fe640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000e9c0 tx=0x7f1d7000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000cd80 con 0x7f1d80100960 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d700035f0 con 0x7f1d80100960 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d8019d900 con 0x7f1d80100960 2024-06-06T07:01:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d70010690 con 0x7f1d80100960 2024-06-06T07:01:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.401+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8019de20 con 0x7f1d80100960 2024-06-06T07:01:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.402+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d80068c10 con 0x7f1d80100960 2024-06-06T07:01:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.403+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) ==== 137953+0+0 (secure 0 0 0) 0x7f1d700037a0 con 0x7f1d80100960 2024-06-06T07:01:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.404+0000 7f1d7d7fa640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 0x7f1d6c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.404+0000 7f1d77fff640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 0x7f1d6c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.404+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f1d700cfc50 con 0x7f1d80100960 2024-06-06T07:01:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.404+0000 7f1d77fff640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 0x7f1d6c0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68005d50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:25.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.408+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d700d5020 con 0x7f1d80100960 2024-06-06T07:01:25.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.591+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1d80061de0 con 0x7f1d6c0a2d00 2024-06-06T07:01:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.601+0000 7f1d7d7fa640 1 -- 172.21.15.73:0/3948075124 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339927 (secure 0 0 0) 0x7f1d80061de0 con 0x7f1d6c0a2d00 2024-06-06T07:01:25.606 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:25.608 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 msgr2=0x7f1d6c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 0x7f1d6c0a51c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1d68006010 tx=0x7f1d68005d50 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 msgr2=0x7f1d801959f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f1d7000e9c0 tx=0x7f1d7000ee80 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 shutdown_connections 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1d80102510 0x7f1d8019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1d6c0a2d00 0x7f1d6c0a51c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1d80101b50 0x7f1d80195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 --2- 172.21.15.73:0/3948075124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1d80100960 0x7f1d801959f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 >> 172.21.15.73:0/3948075124 conn(0x7f1d800fbef0 msgr2=0x7f1d80104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 shutdown_connections 2024-06-06T07:01:25.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:25.605+0000 7f1d85b7e640 1 -- 172.21.15.73:0/3948075124 wait complete. 2024-06-06T07:01:25.618 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-06-06T07:01:23.708112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195190,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2122,"num_read_kb":1233,"num_write":578,"num_write_kb":4421,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":956,"ondisk_log_size":956,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":387960,"kb_used_data":13808,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124553864,"statfs":{"total":1151940427776,"available":1151543156736,"internally_reserved":0,"allocated":14139392,"data_stored":10339226,"data_compressed":592079,"data_compressed_allocated":802816,"data_compressed_original":4284416,"omap_allocated":19142,"internal_metadata":382973242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989506"},"pg_stats":[{"pgid":"3.3f","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960251+0000","last_change":"2024-06-06T06:57:07.556040+0000","last_active":"2024-06-06T07:01:21.960251+0000","last_peered":"2024-06-06T07:01:21.960251+0000","last_clean":"2024-06-06T07:01:21.960251+0000","last_became_active":"2024-06-06T06:57:07.555627+0000","last_became_peered":"2024-06-06T06:57:07.555627+0000","last_unstale":"2024-06-06T07:01:21.960251+0000","last_undegraded":"2024-06-06T07:01:21.960251+0000","last_fullsized":"2024-06-06T07:01:21.960251+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:27.472975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"100'4","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960220+0000","last_change":"2024-06-06T06:57:07.548342+0000","last_active":"2024-06-06T07:01:21.960220+0000","last_peered":"2024-06-06T07:01:21.960220+0000","last_clean":"2024-06-06T07:01:21.960220+0000","last_became_active":"2024-06-06T06:57:07.547842+0000","last_became_peered":"2024-06-06T06:57:07.547842+0000","last_unstale":"2024-06-06T07:01:21.960220+0000","last_undegraded":"2024-06-06T07:01:21.960220+0000","last_fullsized":"2024-06-06T07:01:21.960220+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:27:45.197154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960212+0000","last_change":"2024-06-06T06:57:07.550586+0000","last_active":"2024-06-06T07:01:21.960212+0000","last_peered":"2024-06-06T07:01:21.960212+0000","last_clean":"2024-06-06T07:01:21.960212+0000","last_became_active":"2024-06-06T06:57:07.550269+0000","last_became_peered":"2024-06-06T06:57:07.550269+0000","last_unstale":"2024-06-06T07:01:21.960212+0000","last_undegraded":"2024-06-06T07:01:21.960212+0000","last_fullsized":"2024-06-06T07:01:21.960212+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:52:39.465282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950531+0000","last_change":"2024-06-06T06:57:07.547659+0000","last_active":"2024-06-06T07:01:21.950531+0000","last_peered":"2024-06-06T07:01:21.950531+0000","last_clean":"2024-06-06T07:01:21.950531+0000","last_became_active":"2024-06-06T06:57:07.546925+0000","last_became_peered":"2024-06-06T06:57:07.546925+0000","last_unstale":"2024-06-06T07:01:21.950531+0000","last_undegraded":"2024-06-06T07:01:21.950531+0000","last_fullsized":"2024-06-06T07:01:21.950531+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:12:01.384384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797379+0000","last_change":"2024-06-06T06:57:07.546142+0000","last_active":"2024-06-06T07:01:21.797379+0000","last_peered":"2024-06-06T07:01:21.797379+0000","last_clean":"2024-06-06T07:01:21.797379+0000","last_became_active":"2024-06-06T06:57:07.545757+0000","last_became_peered":"2024-06-06T06:57:07.545757+0000","last_unstale":"2024-06-06T07:01:21.797379+0000","last_undegraded":"2024-06-06T07:01:21.797379+0000","last_fullsized":"2024-06-06T07:01:21.797379+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:08.352925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"100'6","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698596+0000","last_change":"2024-06-06T06:57:52.405561+0000","last_active":"2024-06-06T07:01:21.698596+0000","last_peered":"2024-06-06T07:01:21.698596+0000","last_clean":"2024-06-06T07:01:21.698596+0000","last_became_active":"2024-06-06T06:57:52.405148+0000","last_became_peered":"2024-06-06T06:57:52.405148+0000","last_unstale":"2024-06-06T07:01:21.698596+0000","last_undegraded":"2024-06-06T07:01:21.698596+0000","last_fullsized":"2024-06-06T07:01:21.698596+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:50:12.750217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,8],"acting":[10,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949896+0000","last_change":"2024-06-06T06:57:07.549981+0000","last_active":"2024-06-06T07:01:21.949896+0000","last_peered":"2024-06-06T07:01:21.949896+0000","last_clean":"2024-06-06T07:01:21.949896+0000","last_became_active":"2024-06-06T06:57:07.549491+0000","last_became_peered":"2024-06-06T06:57:07.549491+0000","last_unstale":"2024-06-06T07:01:21.949896+0000","last_undegraded":"2024-06-06T07:01:21.949896+0000","last_fullsized":"2024-06-06T07:01:21.949896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:05:59.088476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796424+0000","last_change":"2024-06-06T06:57:07.556993+0000","last_active":"2024-06-06T07:01:21.796424+0000","last_peered":"2024-06-06T07:01:21.796424+0000","last_clean":"2024-06-06T07:01:21.796424+0000","last_became_active":"2024-06-06T06:57:07.556642+0000","last_became_peered":"2024-06-06T06:57:07.556642+0000","last_unstale":"2024-06-06T07:01:21.796424+0000","last_undegraded":"2024-06-06T07:01:21.796424+0000","last_fullsized":"2024-06-06T07:01:21.796424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:39:09.257809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955413+0000","last_change":"2024-06-06T06:57:07.559222+0000","last_active":"2024-06-06T07:01:21.955413+0000","last_peered":"2024-06-06T07:01:21.955413+0000","last_clean":"2024-06-06T07:01:21.955413+0000","last_became_active":"2024-06-06T06:57:07.558776+0000","last_became_peered":"2024-06-06T06:57:07.558776+0000","last_unstale":"2024-06-06T07:01:21.955413+0000","last_undegraded":"2024-06-06T07:01:21.955413+0000","last_fullsized":"2024-06-06T07:01:21.955413+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:28:23.479312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"100'16","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699282+0000","last_change":"2024-06-06T06:57:07.558309+0000","last_active":"2024-06-06T07:01:21.699282+0000","last_peered":"2024-06-06T07:01:21.699282+0000","last_clean":"2024-06-06T07:01:21.699282+0000","last_became_active":"2024-06-06T06:57:07.557973+0000","last_became_peered":"2024-06-06T06:57:07.557973+0000","last_unstale":"2024-06-06T07:01:21.699282+0000","last_undegraded":"2024-06-06T07:01:21.699282+0000","last_fullsized":"2024-06-06T07:01:21.699282+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:13:57.441726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950661+0000","last_change":"2024-06-06T06:57:07.553923+0000","last_active":"2024-06-06T07:01:21.950661+0000","last_peered":"2024-06-06T07:01:21.950661+0000","last_clean":"2024-06-06T07:01:21.950661+0000","last_became_active":"2024-06-06T06:57:07.552504+0000","last_became_peered":"2024-06-06T06:57:07.552504+0000","last_unstale":"2024-06-06T07:01:21.950661+0000","last_undegraded":"2024-06-06T07:01:21.950661+0000","last_fullsized":"2024-06-06T07:01:21.950661+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:06:17.468660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950393+0000","last_change":"2024-06-06T06:57:07.550130+0000","last_active":"2024-06-06T07:01:21.950393+0000","last_peered":"2024-06-06T07:01:21.950393+0000","last_clean":"2024-06-06T07:01:21.950393+0000","last_became_active":"2024-06-06T06:57:07.548221+0000","last_became_peered":"2024-06-06T06:57:07.548221+0000","last_unstale":"2024-06-06T07:01:21.950393+0000","last_undegraded":"2024-06-06T07:01:21.950393+0000","last_fullsized":"2024-06-06T07:01:21.950393+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:54:39.016080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955249+0000","last_change":"2024-06-06T06:57:07.563255+0000","last_active":"2024-06-06T07:01:21.955249+0000","last_peered":"2024-06-06T07:01:21.955249+0000","last_clean":"2024-06-06T07:01:21.955249+0000","last_became_active":"2024-06-06T06:57:07.562925+0000","last_became_peered":"2024-06-06T06:57:07.562925+0000","last_unstale":"2024-06-06T07:01:21.955249+0000","last_undegraded":"2024-06-06T07:01:21.955249+0000","last_fullsized":"2024-06-06T07:01:21.955249+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:32.251931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959991+0000","last_change":"2024-06-06T06:57:07.547486+0000","last_active":"2024-06-06T07:01:21.959991+0000","last_peered":"2024-06-06T07:01:21.959991+0000","last_clean":"2024-06-06T07:01:21.959991+0000","last_became_active":"2024-06-06T06:57:07.547214+0000","last_became_peered":"2024-06-06T06:57:07.547214+0000","last_unstale":"2024-06-06T07:01:21.959991+0000","last_undegraded":"2024-06-06T07:01:21.959991+0000","last_fullsized":"2024-06-06T07:01:21.959991+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:44.182719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959909+0000","last_change":"2024-06-06T06:57:07.551214+0000","last_active":"2024-06-06T07:01:21.959909+0000","last_peered":"2024-06-06T07:01:21.959909+0000","last_clean":"2024-06-06T07:01:21.959909+0000","last_became_active":"2024-06-06T06:57:07.550127+0000","last_became_peered":"2024-06-06T06:57:07.550127+0000","last_unstale":"2024-06-06T07:01:21.959909+0000","last_undegraded":"2024-06-06T07:01:21.959909+0000","last_fullsized":"2024-06-06T07:01:21.959909+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:15:50.894490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954785+0000","last_change":"2024-06-06T06:57:07.553545+0000","last_active":"2024-06-06T07:01:21.954785+0000","last_peered":"2024-06-06T07:01:21.954785+0000","last_clean":"2024-06-06T07:01:21.954785+0000","last_became_active":"2024-06-06T06:57:07.553204+0000","last_became_peered":"2024-06-06T06:57:07.553204+0000","last_unstale":"2024-06-06T07:01:21.954785+0000","last_undegraded":"2024-06-06T07:01:21.954785+0000","last_fullsized":"2024-06-06T07:01:21.954785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:42:53.921314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960031+0000","last_change":"2024-06-06T06:57:07.560847+0000","last_active":"2024-06-06T07:01:21.960031+0000","last_peered":"2024-06-06T07:01:21.960031+0000","last_clean":"2024-06-06T07:01:21.960031+0000","last_became_active":"2024-06-06T06:57:07.560374+0000","last_became_peered":"2024-06-06T06:57:07.560374+0000","last_unstale":"2024-06-06T07:01:21.960031+0000","last_undegraded":"2024-06-06T07:01:21.960031+0000","last_fullsized":"2024-06-06T07:01:21.960031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:30:08.395935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696962+0000","last_change":"2024-06-06T06:57:07.547998+0000","last_active":"2024-06-06T07:01:21.696962+0000","last_peered":"2024-06-06T07:01:21.696962+0000","last_clean":"2024-06-06T07:01:21.696962+0000","last_became_active":"2024-06-06T06:57:07.547684+0000","last_became_peered":"2024-06-06T06:57:07.547684+0000","last_unstale":"2024-06-06T07:01:21.696962+0000","last_undegraded":"2024-06-06T07:01:21.696962+0000","last_fullsized":"2024-06-06T07:01:21.696962+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:52:07.929759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955638+0000","last_change":"2024-06-06T06:57:07.553564+0000","last_active":"2024-06-06T07:01:21.955638+0000","last_peered":"2024-06-06T07:01:21.955638+0000","last_clean":"2024-06-06T07:01:21.955638+0000","last_became_active":"2024-06-06T06:57:07.553201+0000","last_became_peered":"2024-06-06T06:57:07.553201+0000","last_unstale":"2024-06-06T07:01:21.955638+0000","last_undegraded":"2024-06-06T07:01:21.955638+0000","last_fullsized":"2024-06-06T07:01:21.955638+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:23:19.484178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955116+0000","last_change":"2024-06-06T06:57:18.374155+0000","last_active":"2024-06-06T07:01:21.955116+0000","last_peered":"2024-06-06T07:01:21.955116+0000","last_clean":"2024-06-06T07:01:21.955116+0000","last_became_active":"2024-06-06T06:57:18.372569+0000","last_became_peered":"2024-06-06T06:57:18.372569+0000","last_unstale":"2024-06-06T07:01:21.955116+0000","last_undegraded":"2024-06-06T07:01:21.955116+0000","last_fullsized":"2024-06-06T07:01:21.955116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:48:28.723815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"100'14","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696861+0000","last_change":"2024-06-06T06:57:04.371774+0000","last_active":"2024-06-06T07:01:21.696861+0000","last_peered":"2024-06-06T07:01:21.696861+0000","last_clean":"2024-06-06T07:01:21.696861+0000","last_became_active":"2024-06-06T06:57:04.371513+0000","last_became_peered":"2024-06-06T06:57:04.371513+0000","last_unstale":"2024-06-06T07:01:21.696861+0000","last_undegraded":"2024-06-06T07:01:21.696861+0000","last_fullsized":"2024-06-06T07:01:21.696861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:12:26.418061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798424+0000","last_change":"2024-06-06T06:57:07.553241+0000","last_active":"2024-06-06T07:01:21.798424+0000","last_peered":"2024-06-06T07:01:21.798424+0000","last_clean":"2024-06-06T07:01:21.798424+0000","last_became_active":"2024-06-06T06:57:07.552922+0000","last_became_peered":"2024-06-06T06:57:07.552922+0000","last_unstale":"2024-06-06T07:01:21.798424+0000","last_undegraded":"2024-06-06T07:01:21.798424+0000","last_fullsized":"2024-06-06T07:01:21.798424+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:12.449743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949488+0000","last_change":"2024-06-06T06:57:18.365715+0000","last_active":"2024-06-06T07:01:21.949488+0000","last_peered":"2024-06-06T07:01:21.949488+0000","last_clean":"2024-06-06T07:01:21.949488+0000","last_became_active":"2024-06-06T06:57:18.365274+0000","last_became_peered":"2024-06-06T06:57:18.365274+0000","last_unstale":"2024-06-06T07:01:21.949488+0000","last_undegraded":"2024-06-06T07:01:21.949488+0000","last_fullsized":"2024-06-06T07:01:21.949488+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:36.327916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"98'2","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.798829+0000","last_change":"2024-06-06T06:57:04.435587+0000","last_active":"2024-06-06T07:01:21.798829+0000","last_peered":"2024-06-06T07:01:21.798829+0000","last_clean":"2024-06-06T07:01:21.798829+0000","last_became_active":"2024-06-06T06:57:04.434081+0000","last_became_peered":"2024-06-06T06:57:04.434081+0000","last_unstale":"2024-06-06T07:01:21.798829+0000","last_undegraded":"2024-06-06T07:01:21.798829+0000","last_fullsized":"2024-06-06T07:01:21.798829+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:31:14.321815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960215+0000","last_change":"2024-06-06T06:57:07.545329+0000","last_active":"2024-06-06T07:01:21.960215+0000","last_peered":"2024-06-06T07:01:21.960215+0000","last_clean":"2024-06-06T07:01:21.960215+0000","last_became_active":"2024-06-06T06:57:07.545005+0000","last_became_peered":"2024-06-06T06:57:07.545005+0000","last_unstale":"2024-06-06T07:01:21.960215+0000","last_undegraded":"2024-06-06T07:01:21.960215+0000","last_fullsized":"2024-06-06T07:01:21.960215+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:57:26.317278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950690+0000","last_change":"2024-06-06T06:57:52.406751+0000","last_active":"2024-06-06T07:01:21.950690+0000","last_peered":"2024-06-06T07:01:21.950690+0000","last_clean":"2024-06-06T07:01:21.950690+0000","last_became_active":"2024-06-06T06:57:52.406270+0000","last_became_peered":"2024-06-06T06:57:52.406270+0000","last_unstale":"2024-06-06T07:01:21.950690+0000","last_undegraded":"2024-06-06T07:01:21.950690+0000","last_fullsized":"2024-06-06T07:01:21.950690+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:45:29.842508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,8,5],"acting":[4,3,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"98'12","reported_seq":33,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960125+0000","last_change":"2024-06-06T06:57:52.408099+0000","last_active":"2024-06-06T07:01:21.960125+0000","last_peered":"2024-06-06T07:01:21.960125+0000","last_clean":"2024-06-06T07:01:21.960125+0000","last_became_active":"2024-06-06T06:57:52.401823+0000","last_became_peered":"2024-06-06T06:57:52.401823+0000","last_unstale":"2024-06-06T07:01:21.960125+0000","last_undegraded":"2024-06-06T07:01:21.960125+0000","last_fullsized":"2024-06-06T07:01:21.960125+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:26:30.569845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955097+0000","last_change":"2024-06-06T06:57:07.562913+0000","last_active":"2024-06-06T07:01:21.955097+0000","last_peered":"2024-06-06T07:01:21.955097+0000","last_clean":"2024-06-06T07:01:21.955097+0000","last_became_active":"2024-06-06T06:57:07.562564+0000","last_became_peered":"2024-06-06T06:57:07.562564+0000","last_unstale":"2024-06-06T07:01:21.955097+0000","last_undegraded":"2024-06-06T07:01:21.955097+0000","last_fullsized":"2024-06-06T07:01:21.955097+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:38.707354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949550+0000","last_change":"2024-06-06T06:57:18.363778+0000","last_active":"2024-06-06T07:01:21.949550+0000","last_peered":"2024-06-06T07:01:21.949550+0000","last_clean":"2024-06-06T07:01:21.949550+0000","last_became_active":"2024-06-06T06:57:18.363275+0000","last_became_peered":"2024-06-06T06:57:18.363275+0000","last_unstale":"2024-06-06T07:01:21.949550+0000","last_undegraded":"2024-06-06T07:01:21.949550+0000","last_fullsized":"2024-06-06T07:01:21.949550+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:04:16.101687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"98'5","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797295+0000","last_change":"2024-06-06T06:57:04.371376+0000","last_active":"2024-06-06T07:01:21.797295+0000","last_peered":"2024-06-06T07:01:21.797295+0000","last_clean":"2024-06-06T07:01:21.797295+0000","last_became_active":"2024-06-06T06:57:04.371023+0000","last_became_peered":"2024-06-06T06:57:04.371023+0000","last_unstale":"2024-06-06T07:01:21.797295+0000","last_undegraded":"2024-06-06T07:01:21.797295+0000","last_fullsized":"2024-06-06T07:01:21.797295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:20:57.181225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698810+0000","last_change":"2024-06-06T06:57:07.554679+0000","last_active":"2024-06-06T07:01:21.698810+0000","last_peered":"2024-06-06T07:01:21.698810+0000","last_clean":"2024-06-06T07:01:21.698810+0000","last_became_active":"2024-06-06T06:57:07.554342+0000","last_became_peered":"2024-06-06T06:57:07.554342+0000","last_unstale":"2024-06-06T07:01:21.698810+0000","last_undegraded":"2024-06-06T07:01:21.698810+0000","last_fullsized":"2024-06-06T07:01:21.698810+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:09:34.197311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955250+0000","last_change":"2024-06-06T06:57:18.372139+0000","last_active":"2024-06-06T07:01:21.955250+0000","last_peered":"2024-06-06T07:01:21.955250+0000","last_clean":"2024-06-06T07:01:21.955250+0000","last_became_active":"2024-06-06T06:57:18.371677+0000","last_became_peered":"2024-06-06T06:57:18.371677+0000","last_unstale":"2024-06-06T07:01:21.955250+0000","last_undegraded":"2024-06-06T07:01:21.955250+0000","last_fullsized":"2024-06-06T07:01:21.955250+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:23:32.182354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"101'7","reported_seq":247,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680167+0000","last_change":"2024-06-06T06:57:52.403476+0000","last_active":"2024-06-06T07:01:22.680167+0000","last_peered":"2024-06-06T07:01:22.680167+0000","last_clean":"2024-06-06T07:01:22.680167+0000","last_became_active":"2024-06-06T06:57:52.403055+0000","last_became_peered":"2024-06-06T06:57:52.403055+0000","last_unstale":"2024-06-06T07:01:22.680167+0000","last_undegraded":"2024-06-06T07:01:22.680167+0000","last_fullsized":"2024-06-06T07:01:22.680167+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:04:51.547941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1504,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"100'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796975+0000","last_change":"2024-06-06T06:58:53.476855+0000","last_active":"2024-06-06T07:01:21.796975+0000","last_peered":"2024-06-06T07:01:21.796975+0000","last_clean":"2024-06-06T07:01:21.796975+0000","last_became_active":"2024-06-06T06:58:53.476098+0000","last_became_peered":"2024-06-06T06:58:53.476098+0000","last_unstale":"2024-06-06T07:01:21.796975+0000","last_undegraded":"2024-06-06T07:01:21.796975+0000","last_fullsized":"2024-06-06T07:01:21.796975+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:07:20.623344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,7],"acting":[8,9,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955349+0000","last_change":"2024-06-06T06:57:52.432492+0000","last_active":"2024-06-06T07:01:21.955349+0000","last_peered":"2024-06-06T07:01:21.955349+0000","last_clean":"2024-06-06T07:01:21.955349+0000","last_became_active":"2024-06-06T06:57:52.432062+0000","last_became_peered":"2024-06-06T06:57:52.432062+0000","last_unstale":"2024-06-06T07:01:21.955349+0000","last_undegraded":"2024-06-06T07:01:21.955349+0000","last_fullsized":"2024-06-06T07:01:21.955349+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:31:45.062193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,8,5],"acting":[2,6,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698559+0000","last_change":"2024-06-06T06:57:04.366003+0000","last_active":"2024-06-06T07:01:21.698559+0000","last_peered":"2024-06-06T07:01:21.698559+0000","last_clean":"2024-06-06T07:01:21.698559+0000","last_became_active":"2024-06-06T06:57:04.365577+0000","last_became_peered":"2024-06-06T06:57:04.365577+0000","last_unstale":"2024-06-06T07:01:21.698559+0000","last_undegraded":"2024-06-06T07:01:21.698559+0000","last_fullsized":"2024-06-06T07:01:21.698559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:17:29.587625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"100'8","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698107+0000","last_change":"2024-06-06T06:57:07.556140+0000","last_active":"2024-06-06T07:01:21.698107+0000","last_peered":"2024-06-06T07:01:21.698107+0000","last_clean":"2024-06-06T07:01:21.698107+0000","last_became_active":"2024-06-06T06:57:07.554283+0000","last_became_peered":"2024-06-06T06:57:07.554283+0000","last_unstale":"2024-06-06T07:01:21.698107+0000","last_undegraded":"2024-06-06T07:01:21.698107+0000","last_fullsized":"2024-06-06T07:01:21.698107+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:13:00.901439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960014+0000","last_change":"2024-06-06T06:57:18.375102+0000","last_active":"2024-06-06T07:01:21.960014+0000","last_peered":"2024-06-06T07:01:21.960014+0000","last_clean":"2024-06-06T07:01:21.960014+0000","last_became_active":"2024-06-06T06:57:18.374672+0000","last_became_peered":"2024-06-06T06:57:18.374672+0000","last_unstale":"2024-06-06T07:01:21.960014+0000","last_undegraded":"2024-06-06T07:01:21.960014+0000","last_fullsized":"2024-06-06T07:01:21.960014+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:18:14.662332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954890+0000","last_change":"2024-06-06T06:57:04.369939+0000","last_active":"2024-06-06T07:01:21.954890+0000","last_peered":"2024-06-06T07:01:21.954890+0000","last_clean":"2024-06-06T07:01:21.954890+0000","last_became_active":"2024-06-06T06:57:04.369560+0000","last_became_peered":"2024-06-06T06:57:04.369560+0000","last_unstale":"2024-06-06T07:01:21.954890+0000","last_undegraded":"2024-06-06T07:01:21.954890+0000","last_fullsized":"2024-06-06T07:01:21.954890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:44:54.762934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796557+0000","last_change":"2024-06-06T06:57:07.544146+0000","last_active":"2024-06-06T07:01:21.796557+0000","last_peered":"2024-06-06T07:01:21.796557+0000","last_clean":"2024-06-06T07:01:21.796557+0000","last_became_active":"2024-06-06T06:57:07.543700+0000","last_became_peered":"2024-06-06T06:57:07.543700+0000","last_unstale":"2024-06-06T07:01:21.796557+0000","last_undegraded":"2024-06-06T07:01:21.796557+0000","last_fullsized":"2024-06-06T07:01:21.796557+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:17:52.391628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960416+0000","last_change":"2024-06-06T06:57:18.373381+0000","last_active":"2024-06-06T07:01:21.960416+0000","last_peered":"2024-06-06T07:01:21.960416+0000","last_clean":"2024-06-06T07:01:21.960416+0000","last_became_active":"2024-06-06T06:57:18.372600+0000","last_became_peered":"2024-06-06T06:57:18.372600+0000","last_unstale":"2024-06-06T07:01:21.960416+0000","last_undegraded":"2024-06-06T07:01:21.960416+0000","last_fullsized":"2024-06-06T07:01:21.960416+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:41:38.359165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"98'3","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697614+0000","last_change":"2024-06-06T06:57:04.373478+0000","last_active":"2024-06-06T07:01:21.697614+0000","last_peered":"2024-06-06T07:01:21.697614+0000","last_clean":"2024-06-06T07:01:21.697614+0000","last_became_active":"2024-06-06T06:57:04.373169+0000","last_became_peered":"2024-06-06T06:57:04.373169+0000","last_unstale":"2024-06-06T07:01:21.697614+0000","last_undegraded":"2024-06-06T07:01:21.697614+0000","last_fullsized":"2024-06-06T07:01:21.697614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:10:56.930022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696430+0000","last_change":"2024-06-06T06:57:07.549072+0000","last_active":"2024-06-06T07:01:21.696430+0000","last_peered":"2024-06-06T07:01:21.696430+0000","last_clean":"2024-06-06T07:01:21.696430+0000","last_became_active":"2024-06-06T06:57:07.548479+0000","last_became_peered":"2024-06-06T06:57:07.548479+0000","last_unstale":"2024-06-06T07:01:21.696430+0000","last_undegraded":"2024-06-06T07:01:21.696430+0000","last_fullsized":"2024-06-06T07:01:21.696430+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:03:29.463870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796793+0000","last_change":"2024-06-06T06:57:18.371030+0000","last_active":"2024-06-06T07:01:21.796793+0000","last_peered":"2024-06-06T07:01:21.796793+0000","last_clean":"2024-06-06T07:01:21.796793+0000","last_became_active":"2024-06-06T06:57:18.370398+0000","last_became_peered":"2024-06-06T06:57:18.370398+0000","last_unstale":"2024-06-06T07:01:21.796793+0000","last_undegraded":"2024-06-06T07:01:21.796793+0000","last_fullsized":"2024-06-06T07:01:21.796793+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:45:31.484461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"100'7","reported_seq":422,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679687+0000","last_change":"2024-06-06T06:57:04.367128+0000","last_active":"2024-06-06T07:01:22.679687+0000","last_peered":"2024-06-06T07:01:22.679687+0000","last_clean":"2024-06-06T07:01:22.679687+0000","last_became_active":"2024-06-06T06:57:04.366749+0000","last_became_peered":"2024-06-06T06:57:04.366749+0000","last_unstale":"2024-06-06T07:01:22.679687+0000","last_undegraded":"2024-06-06T07:01:22.679687+0000","last_fullsized":"2024-06-06T07:01:22.679687+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:10:20.650271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":368,"num_read_kb":368,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955354+0000","last_change":"2024-06-06T06:57:07.563114+0000","last_active":"2024-06-06T07:01:21.955354+0000","last_peered":"2024-06-06T07:01:21.955354+0000","last_clean":"2024-06-06T07:01:21.955354+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955354+0000","last_undegraded":"2024-06-06T07:01:21.955354+0000","last_fullsized":"2024-06-06T07:01:21.955354+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:16:29.982159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696386+0000","last_change":"2024-06-06T06:57:18.374280+0000","last_active":"2024-06-06T07:01:21.696386+0000","last_peered":"2024-06-06T07:01:21.696386+0000","last_clean":"2024-06-06T07:01:21.696386+0000","last_became_active":"2024-06-06T06:57:18.373831+0000","last_became_peered":"2024-06-06T06:57:18.373831+0000","last_unstale":"2024-06-06T07:01:21.696386+0000","last_undegraded":"2024-06-06T07:01:21.696386+0000","last_fullsized":"2024-06-06T07:01:21.696386+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:17:24.223753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"98'4","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951076+0000","last_change":"2024-06-06T06:57:04.372216+0000","last_active":"2024-06-06T07:01:21.951076+0000","last_peered":"2024-06-06T07:01:21.951076+0000","last_clean":"2024-06-06T07:01:21.951076+0000","last_became_active":"2024-06-06T06:57:04.371829+0000","last_became_peered":"2024-06-06T06:57:04.371829+0000","last_unstale":"2024-06-06T07:01:21.951076+0000","last_undegraded":"2024-06-06T07:01:21.951076+0000","last_fullsized":"2024-06-06T07:01:21.951076+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:52:54.201786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"100'9","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960384+0000","last_change":"2024-06-06T06:57:07.552635+0000","last_active":"2024-06-06T07:01:21.960384+0000","last_peered":"2024-06-06T07:01:21.960384+0000","last_clean":"2024-06-06T07:01:21.960384+0000","last_became_active":"2024-06-06T06:57:07.551972+0000","last_became_peered":"2024-06-06T06:57:07.551972+0000","last_unstale":"2024-06-06T07:01:21.960384+0000","last_undegraded":"2024-06-06T07:01:21.960384+0000","last_fullsized":"2024-06-06T07:01:21.960384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:21:28.124336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950898+0000","last_change":"2024-06-06T06:57:52.404541+0000","last_active":"2024-06-06T07:01:21.950898+0000","last_peered":"2024-06-06T07:01:21.950898+0000","last_clean":"2024-06-06T07:01:21.950898+0000","last_became_active":"2024-06-06T06:57:52.404010+0000","last_became_peered":"2024-06-06T06:57:52.404010+0000","last_unstale":"2024-06-06T07:01:21.950898+0000","last_undegraded":"2024-06-06T07:01:21.950898+0000","last_fullsized":"2024-06-06T07:01:21.950898+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:41:06.272914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,10],"acting":[4,1,2,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698689+0000","last_change":"2024-06-06T06:57:04.435797+0000","last_active":"2024-06-06T07:01:21.698689+0000","last_peered":"2024-06-06T07:01:21.698689+0000","last_clean":"2024-06-06T07:01:21.698689+0000","last_became_active":"2024-06-06T06:57:04.435455+0000","last_became_peered":"2024-06-06T06:57:04.435455+0000","last_unstale":"2024-06-06T07:01:21.698689+0000","last_undegraded":"2024-06-06T07:01:21.698689+0000","last_fullsized":"2024-06-06T07:01:21.698689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:18:47.263169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955384+0000","last_change":"2024-06-06T06:57:07.558427+0000","last_active":"2024-06-06T07:01:21.955384+0000","last_peered":"2024-06-06T07:01:21.955384+0000","last_clean":"2024-06-06T07:01:21.955384+0000","last_became_active":"2024-06-06T06:57:07.558086+0000","last_became_peered":"2024-06-06T06:57:07.558086+0000","last_unstale":"2024-06-06T07:01:21.955384+0000","last_undegraded":"2024-06-06T07:01:21.955384+0000","last_fullsized":"2024-06-06T07:01:21.955384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:32:13.879006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696457+0000","last_change":"2024-06-06T06:57:18.370783+0000","last_active":"2024-06-06T07:01:21.696457+0000","last_peered":"2024-06-06T07:01:21.696457+0000","last_clean":"2024-06-06T07:01:21.696457+0000","last_became_active":"2024-06-06T06:57:18.370363+0000","last_became_peered":"2024-06-06T06:57:18.370363+0000","last_unstale":"2024-06-06T07:01:21.696457+0000","last_undegraded":"2024-06-06T07:01:21.696457+0000","last_fullsized":"2024-06-06T07:01:21.696457+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:43:42.412787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"101'42","reported_seq":469,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679951+0000","last_change":"2024-06-06T06:57:04.371876+0000","last_active":"2024-06-06T07:01:22.679951+0000","last_peered":"2024-06-06T07:01:22.679951+0000","last_clean":"2024-06-06T07:01:22.679951+0000","last_became_active":"2024-06-06T06:57:04.371486+0000","last_became_peered":"2024-06-06T06:57:04.371486+0000","last_unstale":"2024-06-06T07:01:22.679951+0000","last_undegraded":"2024-06-06T07:01:22.679951+0000","last_fullsized":"2024-06-06T07:01:22.679951+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:31:59.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68760,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":380,"num_read_kb":257,"num_write":44,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797184+0000","last_change":"2024-06-06T06:57:07.544437+0000","last_active":"2024-06-06T07:01:21.797184+0000","last_peered":"2024-06-06T07:01:21.797184+0000","last_clean":"2024-06-06T07:01:21.797184+0000","last_became_active":"2024-06-06T06:57:07.544080+0000","last_became_peered":"2024-06-06T06:57:07.544080+0000","last_unstale":"2024-06-06T07:01:21.797184+0000","last_undegraded":"2024-06-06T07:01:21.797184+0000","last_fullsized":"2024-06-06T07:01:21.797184+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:40.555874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.699244+0000","last_change":"2024-06-06T06:57:18.368647+0000","last_active":"2024-06-06T07:01:21.699244+0000","last_peered":"2024-06-06T07:01:21.699244+0000","last_clean":"2024-06-06T07:01:21.699244+0000","last_became_active":"2024-06-06T06:57:18.367507+0000","last_became_peered":"2024-06-06T06:57:18.367507+0000","last_unstale":"2024-06-06T07:01:21.699244+0000","last_undegraded":"2024-06-06T07:01:21.699244+0000","last_fullsized":"2024-06-06T07:01:21.699244+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:01.122728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"98'9","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955406+0000","last_change":"2024-06-06T06:57:04.370969+0000","last_active":"2024-06-06T07:01:21.955406+0000","last_peered":"2024-06-06T07:01:21.955406+0000","last_clean":"2024-06-06T07:01:21.955406+0000","last_became_active":"2024-06-06T06:57:04.370586+0000","last_became_peered":"2024-06-06T06:57:04.370586+0000","last_unstale":"2024-06-06T07:01:21.955406+0000","last_undegraded":"2024-06-06T07:01:21.955406+0000","last_fullsized":"2024-06-06T07:01:21.955406+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:48:31.599110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954910+0000","last_change":"2024-06-06T06:57:07.552295+0000","last_active":"2024-06-06T07:01:21.954910+0000","last_peered":"2024-06-06T07:01:21.954910+0000","last_clean":"2024-06-06T07:01:21.954910+0000","last_became_active":"2024-06-06T06:57:07.551942+0000","last_became_peered":"2024-06-06T06:57:07.551942+0000","last_unstale":"2024-06-06T07:01:21.954910+0000","last_undegraded":"2024-06-06T07:01:21.954910+0000","last_fullsized":"2024-06-06T07:01:21.954910+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:17:57.568092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959844+0000","last_change":"2024-06-06T06:57:18.371092+0000","last_active":"2024-06-06T07:01:21.959844+0000","last_peered":"2024-06-06T07:01:21.959844+0000","last_clean":"2024-06-06T07:01:21.959844+0000","last_became_active":"2024-06-06T06:57:18.370517+0000","last_became_peered":"2024-06-06T06:57:18.370517+0000","last_unstale":"2024-06-06T07:01:21.959844+0000","last_undegraded":"2024-06-06T07:01:21.959844+0000","last_fullsized":"2024-06-06T07:01:21.959844+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:51:02.398428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697867+0000","last_change":"2024-06-06T06:57:04.369049+0000","last_active":"2024-06-06T07:01:21.697867+0000","last_peered":"2024-06-06T07:01:21.697867+0000","last_clean":"2024-06-06T07:01:21.697867+0000","last_became_active":"2024-06-06T06:57:04.368712+0000","last_became_peered":"2024-06-06T06:57:04.368712+0000","last_unstale":"2024-06-06T07:01:21.697867+0000","last_undegraded":"2024-06-06T07:01:21.697867+0000","last_fullsized":"2024-06-06T07:01:21.697867+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:58.487871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950431+0000","last_change":"2024-06-06T06:57:07.550747+0000","last_active":"2024-06-06T07:01:21.950431+0000","last_peered":"2024-06-06T07:01:21.950431+0000","last_clean":"2024-06-06T07:01:21.950431+0000","last_became_active":"2024-06-06T06:57:07.550072+0000","last_became_peered":"2024-06-06T06:57:07.550072+0000","last_unstale":"2024-06-06T07:01:21.950431+0000","last_undegraded":"2024-06-06T07:01:21.950431+0000","last_fullsized":"2024-06-06T07:01:21.950431+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:04:45.470107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697123+0000","last_change":"2024-06-06T06:57:18.374257+0000","last_active":"2024-06-06T07:01:21.697123+0000","last_peered":"2024-06-06T07:01:21.697123+0000","last_clean":"2024-06-06T07:01:21.697123+0000","last_became_active":"2024-06-06T06:57:18.373875+0000","last_became_peered":"2024-06-06T06:57:18.373875+0000","last_unstale":"2024-06-06T07:01:21.697123+0000","last_undegraded":"2024-06-06T07:01:21.697123+0000","last_fullsized":"2024-06-06T07:01:21.697123+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:24:19.348115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679778+0000","last_change":"2024-06-06T06:57:04.375274+0000","last_active":"2024-06-06T07:01:22.679778+0000","last_peered":"2024-06-06T07:01:22.679778+0000","last_clean":"2024-06-06T07:01:22.679778+0000","last_became_active":"2024-06-06T06:57:04.374780+0000","last_became_peered":"2024-06-06T06:57:04.374780+0000","last_unstale":"2024-06-06T07:01:22.679778+0000","last_undegraded":"2024-06-06T07:01:22.679778+0000","last_fullsized":"2024-06-06T07:01:22.679778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:49:58.691846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2001,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949816+0000","last_change":"2024-06-06T06:57:07.557670+0000","last_active":"2024-06-06T07:01:21.949816+0000","last_peered":"2024-06-06T07:01:21.949816+0000","last_clean":"2024-06-06T07:01:21.949816+0000","last_became_active":"2024-06-06T06:57:07.556546+0000","last_became_peered":"2024-06-06T06:57:07.556546+0000","last_unstale":"2024-06-06T07:01:21.949816+0000","last_undegraded":"2024-06-06T07:01:21.949816+0000","last_fullsized":"2024-06-06T07:01:21.949816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:59:59.662297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955616+0000","last_change":"2024-06-06T06:57:18.368616+0000","last_active":"2024-06-06T07:01:21.955616+0000","last_peered":"2024-06-06T07:01:21.955616+0000","last_clean":"2024-06-06T07:01:21.955616+0000","last_became_active":"2024-06-06T06:57:18.368240+0000","last_became_peered":"2024-06-06T06:57:18.368240+0000","last_unstale":"2024-06-06T07:01:21.955616+0000","last_undegraded":"2024-06-06T07:01:21.955616+0000","last_fullsized":"2024-06-06T07:01:21.955616+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:21.349630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"98'17","reported_seq":65,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796849+0000","last_change":"2024-06-06T06:57:04.434749+0000","last_active":"2024-06-06T07:01:21.796849+0000","last_peered":"2024-06-06T07:01:21.796849+0000","last_clean":"2024-06-06T07:01:21.796849+0000","last_became_active":"2024-06-06T06:57:04.434402+0000","last_became_peered":"2024-06-06T06:57:04.434402+0000","last_unstale":"2024-06-06T07:01:21.796849+0000","last_undegraded":"2024-06-06T07:01:21.796849+0000","last_fullsized":"2024-06-06T07:01:21.796849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:44.501248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697500+0000","last_change":"2024-06-06T06:57:07.555685+0000","last_active":"2024-06-06T07:01:21.697500+0000","last_peered":"2024-06-06T07:01:21.697500+0000","last_clean":"2024-06-06T07:01:21.697500+0000","last_became_active":"2024-06-06T06:57:07.555288+0000","last_became_peered":"2024-06-06T06:57:07.555288+0000","last_unstale":"2024-06-06T07:01:21.697500+0000","last_undegraded":"2024-06-06T07:01:21.697500+0000","last_fullsized":"2024-06-06T07:01:21.697500+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:20:32.630778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697474+0000","last_change":"2024-06-06T06:57:18.374636+0000","last_active":"2024-06-06T07:01:21.697474+0000","last_peered":"2024-06-06T07:01:21.697474+0000","last_clean":"2024-06-06T07:01:21.697474+0000","last_became_active":"2024-06-06T06:57:18.374211+0000","last_became_peered":"2024-06-06T06:57:18.374211+0000","last_unstale":"2024-06-06T07:01:21.697474+0000","last_undegraded":"2024-06-06T07:01:21.697474+0000","last_fullsized":"2024-06-06T07:01:21.697474+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:54:00.391028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"100'3","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797233+0000","last_change":"2024-06-06T06:57:04.369734+0000","last_active":"2024-06-06T07:01:21.797233+0000","last_peered":"2024-06-06T07:01:21.797233+0000","last_clean":"2024-06-06T07:01:21.797233+0000","last_became_active":"2024-06-06T06:57:04.369430+0000","last_became_peered":"2024-06-06T06:57:04.369430+0000","last_unstale":"2024-06-06T07:01:21.797233+0000","last_undegraded":"2024-06-06T07:01:21.797233+0000","last_fullsized":"2024-06-06T07:01:21.797233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:23:15.554269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"100'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960037+0000","last_change":"2024-06-06T06:57:52.432469+0000","last_active":"2024-06-06T07:01:21.960037+0000","last_peered":"2024-06-06T07:01:21.960037+0000","last_clean":"2024-06-06T07:01:21.960037+0000","last_became_active":"2024-06-06T06:57:52.432051+0000","last_became_peered":"2024-06-06T06:57:52.432051+0000","last_unstale":"2024-06-06T07:01:21.960037+0000","last_undegraded":"2024-06-06T07:01:21.960037+0000","last_fullsized":"2024-06-06T07:01:21.960037+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:41:21.940054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797116+0000","last_change":"2024-06-06T06:57:18.425354+0000","last_active":"2024-06-06T07:01:21.797116+0000","last_peered":"2024-06-06T07:01:21.797116+0000","last_clean":"2024-06-06T07:01:21.797116+0000","last_became_active":"2024-06-06T06:57:18.424703+0000","last_became_peered":"2024-06-06T06:57:18.424703+0000","last_unstale":"2024-06-06T07:01:21.797116+0000","last_undegraded":"2024-06-06T07:01:21.797116+0000","last_fullsized":"2024-06-06T07:01:21.797116+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T12:06:18.180822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"98'1","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696557+0000","last_change":"2024-06-06T06:57:04.434972+0000","last_active":"2024-06-06T07:01:21.696557+0000","last_peered":"2024-06-06T07:01:21.696557+0000","last_clean":"2024-06-06T07:01:21.696557+0000","last_became_active":"2024-06-06T06:57:04.434649+0000","last_became_peered":"2024-06-06T06:57:04.434649+0000","last_unstale":"2024-06-06T07:01:21.696557+0000","last_undegraded":"2024-06-06T07:01:21.696557+0000","last_fullsized":"2024-06-06T07:01:21.696557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:05:14.861416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"100'4","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698895+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.698895+0000","last_peered":"2024-06-06T07:01:21.698895+0000","last_clean":"2024-06-06T07:01:21.698895+0000","last_became_active":"2024-06-06T06:57:07.551108+0000","last_became_peered":"2024-06-06T06:57:07.551108+0000","last_unstale":"2024-06-06T07:01:21.698895+0000","last_undegraded":"2024-06-06T07:01:21.698895+0000","last_fullsized":"2024-06-06T07:01:21.698895+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:32:55.470163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955129+0000","last_change":"2024-06-06T06:57:18.372690+0000","last_active":"2024-06-06T07:01:21.955129+0000","last_peered":"2024-06-06T07:01:21.955129+0000","last_clean":"2024-06-06T07:01:21.955129+0000","last_became_active":"2024-06-06T06:57:18.372306+0000","last_became_peered":"2024-06-06T06:57:18.372306+0000","last_unstale":"2024-06-06T07:01:21.955129+0000","last_undegraded":"2024-06-06T07:01:21.955129+0000","last_fullsized":"2024-06-06T07:01:21.955129+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:55:46.597620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"101'7","reported_seq":241,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680327+0000","last_change":"2024-06-06T06:57:04.372772+0000","last_active":"2024-06-06T07:01:22.680327+0000","last_peered":"2024-06-06T07:01:22.680327+0000","last_clean":"2024-06-06T07:01:22.680327+0000","last_became_active":"2024-06-06T06:57:04.372317+0000","last_became_peered":"2024-06-06T06:57:04.372317+0000","last_unstale":"2024-06-06T07:01:22.680327+0000","last_undegraded":"2024-06-06T07:01:22.680327+0000","last_fullsized":"2024-06-06T07:01:22.680327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:37:27.944669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1470,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"100'7","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698794+0000","last_change":"2024-06-06T06:58:53.571939+0000","last_active":"2024-06-06T07:01:21.698794+0000","last_peered":"2024-06-06T07:01:21.698794+0000","last_clean":"2024-06-06T07:01:21.698794+0000","last_became_active":"2024-06-06T06:58:53.571590+0000","last_became_peered":"2024-06-06T06:58:53.571590+0000","last_unstale":"2024-06-06T07:01:21.698794+0000","last_undegraded":"2024-06-06T07:01:21.698794+0000","last_fullsized":"2024-06-06T07:01:21.698794+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:10.858649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955640+0000","last_change":"2024-06-06T06:57:18.424958+0000","last_active":"2024-06-06T07:01:21.955640+0000","last_peered":"2024-06-06T07:01:21.955640+0000","last_clean":"2024-06-06T07:01:21.955640+0000","last_became_active":"2024-06-06T06:57:18.424524+0000","last_became_peered":"2024-06-06T06:57:18.424524+0000","last_unstale":"2024-06-06T07:01:21.955640+0000","last_undegraded":"2024-06-06T07:01:21.955640+0000","last_fullsized":"2024-06-06T07:01:21.955640+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:59:13.098537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697837+0000","last_change":"2024-06-06T06:57:07.560173+0000","last_active":"2024-06-06T07:01:21.697837+0000","last_peered":"2024-06-06T07:01:21.697837+0000","last_clean":"2024-06-06T07:01:21.697837+0000","last_became_active":"2024-06-06T06:57:07.559562+0000","last_became_peered":"2024-06-06T06:57:07.559562+0000","last_unstale":"2024-06-06T07:01:21.697837+0000","last_undegraded":"2024-06-06T07:01:21.697837+0000","last_fullsized":"2024-06-06T07:01:21.697837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:30:05.389321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"100'7","reported_seq":240,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.679259+0000","last_change":"2024-06-06T06:57:04.434352+0000","last_active":"2024-06-06T07:01:22.679259+0000","last_peered":"2024-06-06T07:01:22.679259+0000","last_clean":"2024-06-06T07:01:22.679259+0000","last_became_active":"2024-06-06T06:57:04.433969+0000","last_became_peered":"2024-06-06T06:57:04.433969+0000","last_unstale":"2024-06-06T07:01:22.679259+0000","last_undegraded":"2024-06-06T07:01:22.679259+0000","last_fullsized":"2024-06-06T07:01:22.679259+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:33:46.553605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":185,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949771+0000","last_change":"2024-06-06T06:57:18.370056+0000","last_active":"2024-06-06T07:01:21.949771+0000","last_peered":"2024-06-06T07:01:21.949771+0000","last_clean":"2024-06-06T07:01:21.949771+0000","last_became_active":"2024-06-06T06:57:18.369469+0000","last_became_peered":"2024-06-06T06:57:18.369469+0000","last_unstale":"2024-06-06T07:01:21.949771+0000","last_undegraded":"2024-06-06T07:01:21.949771+0000","last_fullsized":"2024-06-06T07:01:21.949771+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:06:55.942651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"100'8","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797356+0000","last_change":"2024-06-06T06:57:07.558081+0000","last_active":"2024-06-06T07:01:21.797356+0000","last_peered":"2024-06-06T07:01:21.797356+0000","last_clean":"2024-06-06T07:01:21.797356+0000","last_became_active":"2024-06-06T06:57:07.557619+0000","last_became_peered":"2024-06-06T06:57:07.557619+0000","last_unstale":"2024-06-06T07:01:21.797356+0000","last_undegraded":"2024-06-06T07:01:21.797356+0000","last_fullsized":"2024-06-06T07:01:21.797356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:09:01.903737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"98'6","reported_seq":56,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950845+0000","last_change":"2024-06-06T06:57:04.369923+0000","last_active":"2024-06-06T07:01:21.950845+0000","last_peered":"2024-06-06T07:01:21.950845+0000","last_clean":"2024-06-06T07:01:21.950845+0000","last_became_active":"2024-06-06T06:57:04.369561+0000","last_became_peered":"2024-06-06T06:57:04.369561+0000","last_unstale":"2024-06-06T07:01:21.950845+0000","last_undegraded":"2024-06-06T07:01:21.950845+0000","last_fullsized":"2024-06-06T07:01:21.950845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:18:32.969867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"23'137","reported_seq":379,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949403+0000","last_change":"2024-06-06T06:55:42.787325+0000","last_active":"2024-06-06T07:01:21.949403+0000","last_peered":"2024-06-06T07:01:21.949403+0000","last_clean":"2024-06-06T07:01:21.949403+0000","last_became_active":"2024-06-06T06:55:42.786706+0000","last_became_peered":"2024-06-06T06:55:42.786706+0000","last_unstale":"2024-06-06T07:01:21.949403+0000","last_undegraded":"2024-06-06T07:01:21.949403+0000","last_fullsized":"2024-06-06T07:01:21.949403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:52:17.944463+0000","last_clean_scrub_stamp":"2024-06-06T06:52:17.944463+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:25:08.279511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.698791+0000","last_change":"2024-06-06T06:57:18.425360+0000","last_active":"2024-06-06T07:01:21.698791+0000","last_peered":"2024-06-06T07:01:21.698791+0000","last_clean":"2024-06-06T07:01:21.698791+0000","last_became_active":"2024-06-06T06:57:18.424890+0000","last_became_peered":"2024-06-06T06:57:18.424890+0000","last_unstale":"2024-06-06T07:01:21.698791+0000","last_undegraded":"2024-06-06T07:01:21.698791+0000","last_fullsized":"2024-06-06T07:01:21.698791+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:16:32.810859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"100'5","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954789+0000","last_change":"2024-06-06T06:57:07.555040+0000","last_active":"2024-06-06T07:01:21.954789+0000","last_peered":"2024-06-06T07:01:21.954789+0000","last_clean":"2024-06-06T07:01:21.954789+0000","last_became_active":"2024-06-06T06:57:07.554601+0000","last_became_peered":"2024-06-06T06:57:07.554601+0000","last_unstale":"2024-06-06T07:01:21.954789+0000","last_undegraded":"2024-06-06T07:01:21.954789+0000","last_fullsized":"2024-06-06T07:01:21.954789+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:58:58.907326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"98'4","reported_seq":57,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949695+0000","last_change":"2024-06-06T06:57:52.402678+0000","last_active":"2024-06-06T07:01:21.949695+0000","last_peered":"2024-06-06T07:01:21.949695+0000","last_clean":"2024-06-06T07:01:21.949695+0000","last_became_active":"2024-06-06T06:57:52.402262+0000","last_became_peered":"2024-06-06T06:57:52.402262+0000","last_unstale":"2024-06-06T07:01:21.949695+0000","last_undegraded":"2024-06-06T07:01:21.949695+0000","last_fullsized":"2024-06-06T07:01:21.949695+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:07:18.772893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959816+0000","last_change":"2024-06-06T06:57:18.376162+0000","last_active":"2024-06-06T07:01:21.959816+0000","last_peered":"2024-06-06T07:01:21.959816+0000","last_clean":"2024-06-06T07:01:21.959816+0000","last_became_active":"2024-06-06T06:57:18.375715+0000","last_became_peered":"2024-06-06T06:57:18.375715+0000","last_unstale":"2024-06-06T07:01:21.959816+0000","last_undegraded":"2024-06-06T07:01:21.959816+0000","last_fullsized":"2024-06-06T07:01:21.959816+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:55:08.517417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"98'4","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797392+0000","last_change":"2024-06-06T06:57:04.434909+0000","last_active":"2024-06-06T07:01:21.797392+0000","last_peered":"2024-06-06T07:01:21.797392+0000","last_clean":"2024-06-06T07:01:21.797392+0000","last_became_active":"2024-06-06T06:57:04.434514+0000","last_became_peered":"2024-06-06T06:57:04.434514+0000","last_unstale":"2024-06-06T07:01:21.797392+0000","last_undegraded":"2024-06-06T07:01:21.797392+0000","last_fullsized":"2024-06-06T07:01:21.797392+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:12.366300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955313+0000","last_change":"2024-06-06T06:57:07.554128+0000","last_active":"2024-06-06T07:01:21.955313+0000","last_peered":"2024-06-06T07:01:21.955313+0000","last_clean":"2024-06-06T07:01:21.955313+0000","last_became_active":"2024-06-06T06:57:07.553785+0000","last_became_peered":"2024-06-06T06:57:07.553785+0000","last_unstale":"2024-06-06T07:01:21.955313+0000","last_undegraded":"2024-06-06T07:01:21.955313+0000","last_fullsized":"2024-06-06T07:01:21.955313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:37:27.077316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796559+0000","last_change":"2024-06-06T06:57:18.424810+0000","last_active":"2024-06-06T07:01:21.796559+0000","last_peered":"2024-06-06T07:01:21.796559+0000","last_clean":"2024-06-06T07:01:21.796559+0000","last_became_active":"2024-06-06T06:57:18.424431+0000","last_became_peered":"2024-06-06T06:57:18.424431+0000","last_unstale":"2024-06-06T07:01:21.796559+0000","last_undegraded":"2024-06-06T07:01:21.796559+0000","last_fullsized":"2024-06-06T07:01:21.796559+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:30:40.621563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"101'52","reported_seq":123,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.909087+0000","last_change":"2024-06-06T06:57:04.369552+0000","last_active":"2024-06-06T07:01:21.909087+0000","last_peered":"2024-06-06T07:01:21.909087+0000","last_clean":"2024-06-06T07:01:21.909087+0000","last_became_active":"2024-06-06T06:57:04.369210+0000","last_became_peered":"2024-06-06T06:57:04.369210+0000","last_unstale":"2024-06-06T07:01:21.909087+0000","last_undegraded":"2024-06-06T07:01:21.909087+0000","last_fullsized":"2024-06-06T07:01:21.909087+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:52:56.287658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":92,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"100'5","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796688+0000","last_change":"2024-06-06T06:57:07.547324+0000","last_active":"2024-06-06T07:01:21.796688+0000","last_peered":"2024-06-06T07:01:21.796688+0000","last_clean":"2024-06-06T07:01:21.796688+0000","last_became_active":"2024-06-06T06:57:07.546996+0000","last_became_peered":"2024-06-06T06:57:07.546996+0000","last_unstale":"2024-06-06T07:01:21.796688+0000","last_undegraded":"2024-06-06T07:01:21.796688+0000","last_fullsized":"2024-06-06T07:01:21.796688+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:42:05.017651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954998+0000","last_change":"2024-06-06T06:57:18.370707+0000","last_active":"2024-06-06T07:01:21.954998+0000","last_peered":"2024-06-06T07:01:21.954998+0000","last_clean":"2024-06-06T07:01:21.954998+0000","last_became_active":"2024-06-06T06:57:18.370191+0000","last_became_peered":"2024-06-06T06:57:18.370191+0000","last_unstale":"2024-06-06T07:01:21.954998+0000","last_undegraded":"2024-06-06T07:01:21.954998+0000","last_fullsized":"2024-06-06T07:01:21.954998+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:17:07.097748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950264+0000","last_change":"2024-06-06T06:57:04.728965+0000","last_active":"2024-06-06T07:01:21.950264+0000","last_peered":"2024-06-06T07:01:21.950264+0000","last_clean":"2024-06-06T07:01:21.950264+0000","last_became_active":"2024-06-06T06:57:04.728601+0000","last_became_peered":"2024-06-06T06:57:04.728601+0000","last_unstale":"2024-06-06T07:01:21.950264+0000","last_undegraded":"2024-06-06T07:01:21.950264+0000","last_fullsized":"2024-06-06T07:01:21.950264+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T18:16:29.824796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"100'7","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960301+0000","last_change":"2024-06-06T06:57:07.551542+0000","last_active":"2024-06-06T07:01:21.960301+0000","last_peered":"2024-06-06T07:01:21.960301+0000","last_clean":"2024-06-06T07:01:21.960301+0000","last_became_active":"2024-06-06T06:57:07.550569+0000","last_became_peered":"2024-06-06T06:57:07.550569+0000","last_unstale":"2024-06-06T07:01:21.960301+0000","last_undegraded":"2024-06-06T07:01:21.960301+0000","last_fullsized":"2024-06-06T07:01:21.960301+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:02:51.545218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950031+0000","last_change":"2024-06-06T06:57:18.425629+0000","last_active":"2024-06-06T07:01:21.950031+0000","last_peered":"2024-06-06T07:01:21.950031+0000","last_clean":"2024-06-06T07:01:21.950031+0000","last_became_active":"2024-06-06T06:57:18.425054+0000","last_became_peered":"2024-06-06T06:57:18.425054+0000","last_unstale":"2024-06-06T07:01:21.950031+0000","last_undegraded":"2024-06-06T07:01:21.950031+0000","last_fullsized":"2024-06-06T07:01:21.950031+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:37:05.971895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"98'2","reported_seq":55,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954812+0000","last_change":"2024-06-06T06:57:52.431417+0000","last_active":"2024-06-06T07:01:21.954812+0000","last_peered":"2024-06-06T07:01:21.954812+0000","last_clean":"2024-06-06T07:01:21.954812+0000","last_became_active":"2024-06-06T06:57:52.431144+0000","last_became_peered":"2024-06-06T06:57:52.431144+0000","last_unstale":"2024-06-06T07:01:21.954812+0000","last_undegraded":"2024-06-06T07:01:21.954812+0000","last_fullsized":"2024-06-06T07:01:21.954812+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:14:25.508378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"100'2","reported_seq":43,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796855+0000","last_change":"2024-06-06T06:57:07.548127+0000","last_active":"2024-06-06T07:01:21.796855+0000","last_peered":"2024-06-06T07:01:21.796855+0000","last_clean":"2024-06-06T07:01:21.796855+0000","last_became_active":"2024-06-06T06:57:07.547751+0000","last_became_peered":"2024-06-06T06:57:07.547751+0000","last_unstale":"2024-06-06T07:01:21.796855+0000","last_undegraded":"2024-06-06T07:01:21.796855+0000","last_fullsized":"2024-06-06T07:01:21.796855+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:15:34.410418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":38,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697002+0000","last_change":"2024-06-06T06:57:52.405259+0000","last_active":"2024-06-06T07:01:21.697002+0000","last_peered":"2024-06-06T07:01:21.697002+0000","last_clean":"2024-06-06T07:01:21.697002+0000","last_became_active":"2024-06-06T06:57:52.404869+0000","last_became_peered":"2024-06-06T06:57:52.404869+0000","last_unstale":"2024-06-06T07:01:21.697002+0000","last_undegraded":"2024-06-06T07:01:21.697002+0000","last_fullsized":"2024-06-06T07:01:21.697002+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:55:06.889318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"98'7","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796885+0000","last_change":"2024-06-06T06:57:04.370166+0000","last_active":"2024-06-06T07:01:21.796885+0000","last_peered":"2024-06-06T07:01:21.796885+0000","last_clean":"2024-06-06T07:01:21.796885+0000","last_became_active":"2024-06-06T06:57:04.369114+0000","last_became_peered":"2024-06-06T06:57:04.369114+0000","last_unstale":"2024-06-06T07:01:21.796885+0000","last_undegraded":"2024-06-06T07:01:21.796885+0000","last_fullsized":"2024-06-06T07:01:21.796885+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:24:12.773387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"100'9","reported_seq":50,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959797+0000","last_change":"2024-06-06T06:57:07.550573+0000","last_active":"2024-06-06T07:01:21.959797+0000","last_peered":"2024-06-06T07:01:21.959797+0000","last_clean":"2024-06-06T07:01:21.959797+0000","last_became_active":"2024-06-06T06:57:07.550147+0000","last_became_peered":"2024-06-06T06:57:07.550147+0000","last_unstale":"2024-06-06T07:01:21.959797+0000","last_undegraded":"2024-06-06T07:01:21.959797+0000","last_fullsized":"2024-06-06T07:01:21.959797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T10:42:20.212971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"98'1","reported_seq":32,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949681+0000","last_change":"2024-06-06T06:57:18.365667+0000","last_active":"2024-06-06T07:01:21.949681+0000","last_peered":"2024-06-06T07:01:21.949681+0000","last_clean":"2024-06-06T07:01:21.949681+0000","last_became_active":"2024-06-06T06:57:18.365186+0000","last_became_peered":"2024-06-06T06:57:18.365186+0000","last_unstale":"2024-06-06T07:01:21.949681+0000","last_undegraded":"2024-06-06T07:01:21.949681+0000","last_fullsized":"2024-06-06T07:01:21.949681+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:28:19.550694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"101'7","reported_seq":53,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.420884+0000","last_change":"2024-06-06T06:57:04.370003+0000","last_active":"2024-06-06T07:01:22.420884+0000","last_peered":"2024-06-06T07:01:22.420884+0000","last_clean":"2024-06-06T07:01:22.420884+0000","last_became_active":"2024-06-06T06:57:04.369693+0000","last_became_peered":"2024-06-06T06:57:04.369693+0000","last_unstale":"2024-06-06T07:01:22.420884+0000","last_undegraded":"2024-06-06T07:01:22.420884+0000","last_fullsized":"2024-06-06T07:01:22.420884+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:47:15.945510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797826+0000","last_change":"2024-06-06T06:57:07.558465+0000","last_active":"2024-06-06T07:01:21.797826+0000","last_peered":"2024-06-06T07:01:21.797826+0000","last_clean":"2024-06-06T07:01:21.797826+0000","last_became_active":"2024-06-06T06:57:07.558073+0000","last_became_peered":"2024-06-06T06:57:07.558073+0000","last_unstale":"2024-06-06T07:01:21.797826+0000","last_undegraded":"2024-06-06T07:01:21.797826+0000","last_fullsized":"2024-06-06T07:01:21.797826+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:33:02.302141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.949910+0000","last_change":"2024-06-06T06:57:18.425579+0000","last_active":"2024-06-06T07:01:21.949910+0000","last_peered":"2024-06-06T07:01:21.949910+0000","last_clean":"2024-06-06T07:01:21.949910+0000","last_became_active":"2024-06-06T06:57:18.425018+0000","last_became_peered":"2024-06-06T06:57:18.425018+0000","last_unstale":"2024-06-06T07:01:21.949910+0000","last_undegraded":"2024-06-06T07:01:21.949910+0000","last_fullsized":"2024-06-06T07:01:21.949910+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:17:00.791625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"98'4","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.796968+0000","last_change":"2024-06-06T06:57:04.370112+0000","last_active":"2024-06-06T07:01:21.796968+0000","last_peered":"2024-06-06T07:01:21.796968+0000","last_clean":"2024-06-06T07:01:21.796968+0000","last_became_active":"2024-06-06T06:57:04.369794+0000","last_became_peered":"2024-06-06T06:57:04.369794+0000","last_unstale":"2024-06-06T07:01:21.796968+0000","last_undegraded":"2024-06-06T07:01:21.796968+0000","last_fullsized":"2024-06-06T07:01:21.796968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:26:24.046409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"100'5","reported_seq":44,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950164+0000","last_change":"2024-06-06T06:57:07.555359+0000","last_active":"2024-06-06T07:01:21.950164+0000","last_peered":"2024-06-06T07:01:21.950164+0000","last_clean":"2024-06-06T07:01:21.950164+0000","last_became_active":"2024-06-06T06:57:07.554900+0000","last_became_peered":"2024-06-06T06:57:07.554900+0000","last_unstale":"2024-06-06T07:01:21.950164+0000","last_undegraded":"2024-06-06T07:01:21.950164+0000","last_fullsized":"2024-06-06T07:01:21.950164+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:00:59.805631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.954886+0000","last_change":"2024-06-06T06:57:18.371416+0000","last_active":"2024-06-06T07:01:21.954886+0000","last_peered":"2024-06-06T07:01:21.954886+0000","last_clean":"2024-06-06T07:01:21.954886+0000","last_became_active":"2024-06-06T06:57:18.370807+0000","last_became_peered":"2024-06-06T06:57:18.370807+0000","last_unstale":"2024-06-06T07:01:21.954886+0000","last_undegraded":"2024-06-06T07:01:21.954886+0000","last_fullsized":"2024-06-06T07:01:21.954886+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:07:47.616975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"101'9","reported_seq":245,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:22.680416+0000","last_change":"2024-06-06T06:57:04.728969+0000","last_active":"2024-06-06T07:01:22.680416+0000","last_peered":"2024-06-06T07:01:22.680416+0000","last_clean":"2024-06-06T07:01:22.680416+0000","last_became_active":"2024-06-06T06:57:04.728463+0000","last_became_peered":"2024-06-06T06:57:04.728463+0000","last_unstale":"2024-06-06T07:01:22.680416+0000","last_undegraded":"2024-06-06T07:01:22.680416+0000","last_fullsized":"2024-06-06T07:01:22.680416+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:00:59.483862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2583,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950755+0000","last_change":"2024-06-06T06:57:07.554817+0000","last_active":"2024-06-06T07:01:21.950755+0000","last_peered":"2024-06-06T07:01:21.950755+0000","last_clean":"2024-06-06T07:01:21.950755+0000","last_became_active":"2024-06-06T06:57:07.553947+0000","last_became_peered":"2024-06-06T06:57:07.553947+0000","last_unstale":"2024-06-06T07:01:21.950755+0000","last_undegraded":"2024-06-06T07:01:21.950755+0000","last_fullsized":"2024-06-06T07:01:21.950755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:22:28.712585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959977+0000","last_change":"2024-06-06T06:57:18.373442+0000","last_active":"2024-06-06T07:01:21.959977+0000","last_peered":"2024-06-06T07:01:21.959977+0000","last_clean":"2024-06-06T07:01:21.959977+0000","last_became_active":"2024-06-06T06:57:18.373072+0000","last_became_peered":"2024-06-06T06:57:18.373072+0000","last_unstale":"2024-06-06T07:01:21.959977+0000","last_undegraded":"2024-06-06T07:01:21.959977+0000","last_fullsized":"2024-06-06T07:01:21.959977+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:11:32.150153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"101'41","reported_seq":95,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.904525+0000","last_change":"2024-06-06T06:57:04.366775+0000","last_active":"2024-06-06T07:01:21.904525+0000","last_peered":"2024-06-06T07:01:21.904525+0000","last_clean":"2024-06-06T07:01:21.904525+0000","last_became_active":"2024-06-06T06:57:04.366244+0000","last_became_peered":"2024-06-06T06:57:04.366244+0000","last_unstale":"2024-06-06T07:01:21.904525+0000","last_undegraded":"2024-06-06T07:01:21.904525+0000","last_fullsized":"2024-06-06T07:01:21.904525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:27:52.094478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.950160+0000","last_change":"2024-06-06T06:57:07.551367+0000","last_active":"2024-06-06T07:01:21.950160+0000","last_peered":"2024-06-06T07:01:21.950160+0000","last_clean":"2024-06-06T07:01:21.950160+0000","last_became_active":"2024-06-06T06:57:07.551025+0000","last_became_peered":"2024-06-06T06:57:07.551025+0000","last_unstale":"2024-06-06T07:01:21.950160+0000","last_undegraded":"2024-06-06T07:01:21.950160+0000","last_fullsized":"2024-06-06T07:01:21.950160+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:00:06.818694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697415+0000","last_change":"2024-06-06T06:57:18.368461+0000","last_active":"2024-06-06T07:01:21.697415+0000","last_peered":"2024-06-06T07:01:21.697415+0000","last_clean":"2024-06-06T07:01:21.697415+0000","last_became_active":"2024-06-06T06:57:18.367829+0000","last_became_peered":"2024-06-06T06:57:18.367829+0000","last_unstale":"2024-06-06T07:01:21.697415+0000","last_undegraded":"2024-06-06T07:01:21.697415+0000","last_fullsized":"2024-06-06T07:01:21.697415+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":92,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:17.349878+0000","last_clean_scrub_stamp":"2024-06-06T06:57:17.349878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:41:09.332855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"98'5","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955485+0000","last_change":"2024-06-06T06:57:04.368798+0000","last_active":"2024-06-06T07:01:21.955485+0000","last_peered":"2024-06-06T07:01:21.955485+0000","last_clean":"2024-06-06T07:01:21.955485+0000","last_became_active":"2024-06-06T06:57:04.368261+0000","last_became_peered":"2024-06-06T06:57:04.368261+0000","last_unstale":"2024-06-06T07:01:21.955485+0000","last_undegraded":"2024-06-06T07:01:21.955485+0000","last_fullsized":"2024-06-06T07:01:21.955485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:03.349075+0000","last_clean_scrub_stamp":"2024-06-06T06:57:03.349075+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T17:41:25.238061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"100'9","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.951082+0000","last_change":"2024-06-06T06:57:07.550696+0000","last_active":"2024-06-06T07:01:21.951082+0000","last_peered":"2024-06-06T07:01:21.951082+0000","last_clean":"2024-06-06T07:01:21.951082+0000","last_became_active":"2024-06-06T06:57:07.550354+0000","last_became_peered":"2024-06-06T06:57:07.550354+0000","last_unstale":"2024-06-06T07:01:21.951082+0000","last_undegraded":"2024-06-06T07:01:21.951082+0000","last_fullsized":"2024-06-06T07:01:21.951082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:42:30.674102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"100'9","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959867+0000","last_change":"2024-06-06T06:57:07.557539+0000","last_active":"2024-06-06T07:01:21.959867+0000","last_peered":"2024-06-06T07:01:21.959867+0000","last_clean":"2024-06-06T07:01:21.959867+0000","last_became_active":"2024-06-06T06:57:07.557189+0000","last_became_peered":"2024-06-06T06:57:07.557189+0000","last_unstale":"2024-06-06T07:01:21.959867+0000","last_undegraded":"2024-06-06T07:01:21.959867+0000","last_fullsized":"2024-06-06T07:01:21.959867+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:38:33.410650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"100'12","reported_seq":58,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955299+0000","last_change":"2024-06-06T06:58:53.478114+0000","last_active":"2024-06-06T07:01:21.955299+0000","last_peered":"2024-06-06T07:01:21.955299+0000","last_clean":"2024-06-06T07:01:21.955299+0000","last_became_active":"2024-06-06T06:58:53.477664+0000","last_became_peered":"2024-06-06T06:58:53.477664+0000","last_unstale":"2024-06-06T07:01:21.955299+0000","last_undegraded":"2024-06-06T07:01:21.955299+0000","last_fullsized":"2024-06-06T07:01:21.955299+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:49:01.541804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"100'8","reported_seq":48,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955202+0000","last_change":"2024-06-06T06:57:07.550639+0000","last_active":"2024-06-06T07:01:21.955202+0000","last_peered":"2024-06-06T07:01:21.955202+0000","last_clean":"2024-06-06T07:01:21.955202+0000","last_became_active":"2024-06-06T06:57:07.550272+0000","last_became_peered":"2024-06-06T06:57:07.550272+0000","last_unstale":"2024-06-06T07:01:21.955202+0000","last_undegraded":"2024-06-06T07:01:21.955202+0000","last_fullsized":"2024-06-06T07:01:21.955202+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:42:41.471123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"100'7","reported_seq":46,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960342+0000","last_change":"2024-06-06T06:57:07.551287+0000","last_active":"2024-06-06T07:01:21.960342+0000","last_peered":"2024-06-06T07:01:21.960342+0000","last_clean":"2024-06-06T07:01:21.960342+0000","last_became_active":"2024-06-06T06:57:07.550957+0000","last_became_peered":"2024-06-06T06:57:07.550957+0000","last_unstale":"2024-06-06T07:01:21.960342+0000","last_undegraded":"2024-06-06T07:01:21.960342+0000","last_fullsized":"2024-06-06T07:01:21.960342+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T07:16:00.858063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"100'4","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955336+0000","last_change":"2024-06-06T06:57:07.553727+0000","last_active":"2024-06-06T07:01:21.955336+0000","last_peered":"2024-06-06T07:01:21.955336+0000","last_clean":"2024-06-06T07:01:21.955336+0000","last_became_active":"2024-06-06T06:57:07.553352+0000","last_became_peered":"2024-06-06T06:57:07.553352+0000","last_unstale":"2024-06-06T07:01:21.955336+0000","last_undegraded":"2024-06-06T07:01:21.955336+0000","last_fullsized":"2024-06-06T07:01:21.955336+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T15:41:40.569514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"100'6","reported_seq":47,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.960225+0000","last_change":"2024-06-06T06:57:07.549235+0000","last_active":"2024-06-06T07:01:21.960225+0000","last_peered":"2024-06-06T07:01:21.960225+0000","last_clean":"2024-06-06T07:01:21.960225+0000","last_became_active":"2024-06-06T06:57:07.548863+0000","last_became_peered":"2024-06-06T06:57:07.548863+0000","last_unstale":"2024-06-06T07:01:21.960225+0000","last_undegraded":"2024-06-06T07:01:21.960225+0000","last_fullsized":"2024-06-06T07:01:21.960225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:10:06.055760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"100'15","reported_seq":54,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955704+0000","last_change":"2024-06-06T06:57:07.554911+0000","last_active":"2024-06-06T07:01:21.955704+0000","last_peered":"2024-06-06T07:01:21.955704+0000","last_clean":"2024-06-06T07:01:21.955704+0000","last_became_active":"2024-06-06T06:57:07.553733+0000","last_became_peered":"2024-06-06T06:57:07.553733+0000","last_unstale":"2024-06-06T07:01:21.955704+0000","last_undegraded":"2024-06-06T07:01:21.955704+0000","last_fullsized":"2024-06-06T07:01:21.955704+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:13:41.556053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"100'2","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.696665+0000","last_change":"2024-06-06T06:58:53.478046+0000","last_active":"2024-06-06T07:01:21.696665+0000","last_peered":"2024-06-06T07:01:21.696665+0000","last_clean":"2024-06-06T07:01:21.696665+0000","last_became_active":"2024-06-06T06:58:53.477722+0000","last_became_peered":"2024-06-06T06:58:53.477722+0000","last_unstale":"2024-06-06T07:01:21.696665+0000","last_undegraded":"2024-06-06T07:01:21.696665+0000","last_fullsized":"2024-06-06T07:01:21.696665+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T16:46:22.491510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.28","version":"100'6","reported_seq":45,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.697616+0000","last_change":"2024-06-06T06:57:07.556348+0000","last_active":"2024-06-06T07:01:21.697616+0000","last_peered":"2024-06-06T07:01:21.697616+0000","last_clean":"2024-06-06T07:01:21.697616+0000","last_became_active":"2024-06-06T06:57:07.555145+0000","last_became_peered":"2024-06-06T06:57:07.555145+0000","last_unstale":"2024-06-06T07:01:21.697616+0000","last_undegraded":"2024-06-06T07:01:21.697616+0000","last_fullsized":"2024-06-06T07:01:21.697616+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T08:40:18.683901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.797013+0000","last_change":"2024-06-06T06:57:07.553527+0000","last_active":"2024-06-06T07:01:21.797013+0000","last_peered":"2024-06-06T07:01:21.797013+0000","last_clean":"2024-06-06T07:01:21.797013+0000","last_became_active":"2024-06-06T06:57:07.553167+0000","last_became_peered":"2024-06-06T06:57:07.553167+0000","last_unstale":"2024-06-06T07:01:21.797013+0000","last_undegraded":"2024-06-06T07:01:21.797013+0000","last_fullsized":"2024-06-06T07:01:21.797013+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T13:38:00.442689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"100'10","reported_seq":51,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955072+0000","last_change":"2024-06-06T06:57:07.564426+0000","last_active":"2024-06-06T07:01:21.955072+0000","last_peered":"2024-06-06T07:01:21.955072+0000","last_clean":"2024-06-06T07:01:21.955072+0000","last_became_active":"2024-06-06T06:57:07.562736+0000","last_became_peered":"2024-06-06T06:57:07.562736+0000","last_unstale":"2024-06-06T07:01:21.955072+0000","last_undegraded":"2024-06-06T07:01:21.955072+0000","last_fullsized":"2024-06-06T07:01:21.955072+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T14:28:51.207716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"100'11","reported_seq":52,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.955200+0000","last_change":"2024-06-06T06:57:07.556480+0000","last_active":"2024-06-06T07:01:21.955200+0000","last_peered":"2024-06-06T07:01:21.955200+0000","last_clean":"2024-06-06T07:01:21.955200+0000","last_became_active":"2024-06-06T06:57:07.556134+0000","last_became_peered":"2024-06-06T06:57:07.556134+0000","last_unstale":"2024-06-06T07:01:21.955200+0000","last_undegraded":"2024-06-06T07:01:21.955200+0000","last_fullsized":"2024-06-06T07:01:21.955200+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T11:46:05.119484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"100'8","reported_seq":49,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-06-06T07:01:21.959980+0000","last_change":"2024-06-06T06:57:07.549202+0000","last_active":"2024-06-06T07:01:21.959980+0000","last_peered":"2024-06-06T07:01:21.959980+0000","last_clean":"2024-06-06T07:01:21.959980+0000","last_became_active":"2024-06-06T06:57:07.548794+0000","last_became_peered":"2024-06-06T06:57:07.548794+0000","last_unstale":"2024-06-06T07:01:21.959980+0000","last_undegraded":"2024-06-06T07:01:21.959980+0000","last_fullsized":"2024-06-06T07:01:21.959980+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-06T06:57:06.525913+0000","last_clean_scrub_stamp":"2024-06-06T06:57:06.525913+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-07T09:43:50.130942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108969,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604588,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1986,"num_read_kb":1114,"num_write":395,"num_write_kb":1481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1813764,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":317,"ondisk_log_size":317,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351042,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36436,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708716,"statfs":{"total":95995035648,"available":95957725184,"internally_reserved":0,"allocated":1122304,"data_stored":584969,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612679,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28284,"kb_used_data":1268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716868,"statfs":{"total":95995035648,"available":95966072832,"internally_reserved":0,"allocated":1298432,"data_stored":1372145,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776204,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1175552,"data_stored":980798,"data_compressed":54376,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037842,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1128,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1155072,"data_stored":780472,"data_compressed":44750,"data_compressed_allocated":61440,"data_compressed_original":294912,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299479,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28188,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716964,"statfs":{"total":95995035648,"available":95966171136,"internally_reserved":0,"allocated":1196032,"data_stored":1183147,"data_compressed":71553,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863463004,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1110016,"data_stored":569558,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36508,"kb_used_data":1168,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708644,"statfs":{"total":95995035648,"available":95957651456,"internally_reserved":0,"allocated":1196032,"data_stored":649662,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712868,"statfs":{"total":95995035648,"available":95961976832,"internally_reserved":0,"allocated":1134592,"data_stored":571545,"data_compressed":32265,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36520,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708632,"statfs":{"total":95995035648,"available":95957639168,"internally_reserved":0,"allocated":1208320,"data_stored":649791,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149802,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32328,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712824,"statfs":{"total":95995035648,"available":95961931776,"internally_reserved":0,"allocated":1179648,"data_stored":1103514,"data_compressed":60532,"data_compressed_allocated":81920,"data_compressed_original":614400,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888166,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1138688,"data_stored":585981,"data_compressed":35230,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724640,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36544,"kb_used_data":1196,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708608,"statfs":{"total":95995035648,"available":95957614592,"internally_reserved":0,"allocated":1224704,"data_stored":1307644,"data_compressed":77709,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36323,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":71394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":71523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135302,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":394716,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":203509,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-06T07:01:25.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:01:25.912 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:25 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:25 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:01:26.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:26.349 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:26.349 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:25 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:27.727 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:27.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 -- 172.21.15.73:0/4246473804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac107400 msgr2=0x7f83ac1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 --2- 172.21.15.73:0/4246473804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac107400 0x7f83ac1077e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f839c009f90 tx=0x7f839c00b330 comp rx=0 tx=0).stop 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 -- 172.21.15.73:0/4246473804 shutdown_connections 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 --2- 172.21.15.73:0/4246473804 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac101d50 0x7f83ac10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 --2- 172.21.15.73:0/4246473804 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac107400 0x7f83ac1077e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:27.834 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 --2- 172.21.15.73:0/4246473804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101390 0x7f83ac101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:27.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 -- 172.21.15.73:0/4246473804 >> 172.21.15.73:0/4246473804 conn(0x7f83ac078340 msgr2=0x7f83ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:27.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 -- 172.21.15.73:0/4246473804 shutdown_connections 2024-06-06T07:01:27.835 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.833+0000 7f83b119b700 1 -- 172.21.15.73:0/4246473804 wait complete. 2024-06-06T07:01:27.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 Processor -- start 2024-06-06T07:01:27.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 -- start start 2024-06-06T07:01:27.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 0x7f83ac19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:27.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:27.837 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 0x7f83ac19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac111550 con 0x7f83ac101d50 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1113d0 con 0x7f83ac101390 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83b119b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83ac1116d0 con 0x7f83ac107400 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83aa59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43006/0 (socket says 172.21.15.73:43006) 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83aa59c700 1 -- 172.21.15.73:0/1663233013 learned_addr learned my addr 172.21.15.73:0/1663233013 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.834+0000 7f83ab59e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 0x7f83ac19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aad9d700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 0x7f83ac19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 -- 172.21.15.73:0/1663233013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 msgr2=0x7f83ac19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 0x7f83ac19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 -- 172.21.15.73:0/1663233013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 msgr2=0x7f83ac19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 0x7f83ac19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:27.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 -- 172.21.15.73:0/1663233013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ac19fd20 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aad9d700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 0x7f83ac19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83ab59e700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 0x7f83ac19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83aa59c700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f839400ec10 tx=0x7f839400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839400cd40 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ac1a0010 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83940044d0 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8394010690 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.835+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ac1a7890 con 0x7f83ac101d50 2024-06-06T07:01:27.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.836+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ac069180 con 0x7f83ac101d50 2024-06-06T07:01:27.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.837+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8394003e50 con 0x7f83ac101d50 2024-06-06T07:01:27.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.838+0000 7f83a3fff700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 0x7f83980a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:27.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.838+0000 7f83aad9d700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 0x7f83980a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:27.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.839+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f83940cfe10 con 0x7f83ac101d50 2024-06-06T07:01:27.840 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.839+0000 7f83aad9d700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 0x7f83980a52e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f839c009790 tx=0x7f839c009700 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:27.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:27.842+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f839409c4b0 con 0x7f83ac101d50 2024-06-06T07:01:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='client.15393 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-06T07:01:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-06T07:01:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: mgrmap e23: x(active, since 4s), standbys: y, z 2024-06-06T07:01:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:27.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:28.091 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.091+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83ac19c070 con 0x7f83ac101d50 2024-06-06T07:01:28.092 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.091+0000 7f83a3fff700 1 -- 172.21.15.73:0/1663233013 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11932 (secure 0 0 0) 0x7f839409bdb0 con 0x7f83ac101d50 2024-06-06T07:01:28.092 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:28.092 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T07:01:19.680600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24419,24433,24421,24404,24415]},"id":1}]} 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 msgr2=0x7f83980a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 0x7f83980a52e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f839c009790 tx=0x7f839c009700 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 msgr2=0x7f83ac19b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f839400ec10 tx=0x7f839400ef20 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 shutdown_connections 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83980a2e20 0x7f83980a52e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83ac107400 0x7f83ac19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83ac101390 0x7f83ac19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 --2- 172.21.15.73:0/1663233013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83ac101d50 0x7f83ac19b250 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 >> 172.21.15.73:0/1663233013 conn(0x7f83ac078340 msgr2=0x7f83ac06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 shutdown_connections 2024-06-06T07:01:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:28.094+0000 7f83b119b700 1 -- 172.21.15.73:0/1663233013 wait complete. 2024-06-06T07:01:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 30 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:28 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1663233013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:28.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:28.804 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.805 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:28 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1663233013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:01:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:01:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi073:/etc/ceph/ceph.conf 2024-06-06T07:01:28.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi136:/etc/ceph/ceph.conf 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi170:/etc/ceph/ceph.conf 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi136:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:28.849 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:28 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1663233013' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:01:29.044 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:29 smithi073 ceph-mon[32303]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:29 smithi073 ceph-mon[32303]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:29 smithi073 ceph-mon[32303]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:29 smithi073 ceph-mon[32303]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s 2024-06-06T07:01:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:29 smithi136 ceph-mon[35901]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:29 smithi136 ceph-mon[35901]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:29 smithi136 ceph-mon[35901]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:29 smithi136 ceph-mon[35901]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s 2024-06-06T07:01:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:29 smithi170 ceph-mon[36108]: Updating smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:29 smithi170 ceph-mon[36108]: Updating smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:29 smithi170 ceph-mon[36108]: Updating smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-06T07:01:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:29 smithi170 ceph-mon[36108]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 5 op/s 2024-06-06T07:01:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.848+0000 7f078c9c7700 1 -- 172.21.15.73:0/1899201303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0788101d70 msgr2=0x7f078810e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.848+0000 7f078c9c7700 1 --2- 172.21.15.73:0/1899201303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0788101d70 0x7f078810e250 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f078000b0a0 tx=0x7f078000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.848+0000 7f078c9c7700 1 -- 172.21.15.73:0/1899201303 shutdown_connections 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 --2- 172.21.15.73:0/1899201303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788107410 0x7f07881077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 --2- 172.21.15.73:0/1899201303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0788101d70 0x7f078810e250 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 --2- 172.21.15.73:0/1899201303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f07881013d0 0x7f0788101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 -- 172.21.15.73:0/1899201303 >> 172.21.15.73:0/1899201303 conn(0x7f07880782c0 msgr2=0x7f07880786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 -- 172.21.15.73:0/1899201303 shutdown_connections 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 -- 172.21.15.73:0/1899201303 wait complete. 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 Processor -- start 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.849+0000 7f078c9c7700 1 -- start start 2024-06-06T07:01:30.852 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788101d70 0x7f078819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 0x7f0788198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f07881126b0 con 0x7f07881013d0 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0788112530 con 0x7f0788107410 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0788112830 con 0x7f0788101d70 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43026/0 (socket says 172.21.15.73:43026) 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 -- 172.21.15.73:0/2576391892 learned_addr learned my addr 172.21.15.73:0/2576391892 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f0785d9b700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788101d70 0x7f078819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f0786d9d700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 0x7f0788198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 -- 172.21.15.73:0/2576391892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788101d70 msgr2=0x7f078819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788101d70 0x7f078819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.853 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 -- 172.21.15.73:0/2576391892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 msgr2=0x7f0788198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 0x7f0788198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 -- 172.21.15.73:0/2576391892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0788199190 con 0x7f07881013d0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f0786d9d700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 0x7f0788198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078659c700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f077800d980 tx=0x7f077800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778017070 con 0x7f07881013d0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0788199480 con 0x7f07881013d0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0778005020 con 0x7f07881013d0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0778009b50 con 0x7f07881013d0 2024-06-06T07:01:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.850+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07881a56f0 con 0x7f07881013d0 2024-06-06T07:01:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.851+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0788102ac0 con 0x7f07881013d0 2024-06-06T07:01:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.852+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f0778009cf0 con 0x7f07881013d0 2024-06-06T07:01:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.853+0000 7f077f7fe700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 0x7f07740a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.853+0000 7f0785d9b700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 0x7f07740a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.853+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f07780cf870 con 0x7f07881013d0 2024-06-06T07:01:30.856 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.854+0000 7f0785d9b700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 0x7f07740a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0788078ae0 tx=0x7f077000a300 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:30.858 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:30.858+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0778020020 con 0x7f07881013d0 2024-06-06T07:01:31.055 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.054+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f078802dca0 con 0x7f07881013d0 2024-06-06T07:01:31.056 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.056+0000 7f077f7fe700 1 -- 172.21.15.73:0/2576391892 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+25192 (secure 0 0 0) 0x7f077809bf70 con 0x7f07881013d0 2024-06-06T07:01:31.057 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:01:31.057 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":101,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T07:01:21.691401+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/793833713":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1478961120":"2024-06-07T07:01:21.691366+0000","172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1988924098":"2024-06-07T07:01:21.691366+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/3113057040":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1117851147":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/793833713":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:01:31.058 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 msgr2=0x7f07740a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 0x7f07740a52e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0788078ae0 tx=0x7f077000a300 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 msgr2=0x7f078819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f077800d980 tx=0x7f077800dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 shutdown_connections 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f07740a2e20 0x7f07740a52e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0788107410 0x7f0788198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f07881013d0 0x7f078819e820 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 --2- 172.21.15.73:0/2576391892 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0788101d70 0x7f078819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 >> 172.21.15.73:0/2576391892 conn(0x7f07880782c0 msgr2=0x7f0788100480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 shutdown_connections 2024-06-06T07:01:31.059 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.058+0000 7f078c9c7700 1 -- 172.21.15.73:0/2576391892 wait complete. 2024-06-06T07:01:31.718 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:31 smithi073 ceph-mon[32303]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-06T07:01:31.719 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:31 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2576391892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:31.720 DEBUG:tasks.fwd_scrub:Yielding 2024-06-06T07:01:31.720 INFO:teuthology.run_tasks:Running task exec... 2024-06-06T07:01:31.730 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:01:31.731 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:01:31.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-06-06T07:01:31.731 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-06T07:01:31.731 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:01:31.732 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:01:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:31 smithi136 ceph-mon[35901]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-06T07:01:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:31 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2576391892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:31.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 -- 172.21.15.73:0/2359972861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c102c50 msgr2=0x7f474c103030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 --2- 172.21.15.73:0/2359972861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c102c50 0x7f474c103030 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f473c009900 tx=0x7f473c02a480 comp rx=0 tx=0).stop 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 -- 172.21.15.73:0/2359972861 shutdown_connections 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 --2- 172.21.15.73:0/2359972861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c10bf90 0x7f474c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 --2- 172.21.15.73:0/2359972861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 0x7f474c10ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 --2- 172.21.15.73:0/2359972861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c102c50 0x7f474c103030 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 -- 172.21.15.73:0/2359972861 >> 172.21.15.73:0/2359972861 conn(0x7f474c077a20 msgr2=0x7f474c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 -- 172.21.15.73:0/2359972861 shutdown_connections 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.837+0000 7f475084d640 1 -- 172.21.15.73:0/2359972861 wait complete. 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 Processor -- start 2024-06-06T07:01:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 -- start start 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c102c50 0x7f474c198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 0x7f474c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f474c111c10 con 0x7f474c10bf90 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f474c111a90 con 0x7f474c102c50 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f475084d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f474c111d90 con 0x7f474c103570 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.838+0000 7f474ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43042/0 (socket says 172.21.15.73:43042) 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 -- 172.21.15.73:0/1590476643 learned_addr learned my addr 172.21.15.73:0/1590476643 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474a59c640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c102c50 0x7f474c198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f4741d9b640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 0x7f474c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 -- 172.21.15.73:0/1590476643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 msgr2=0x7f474c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 0x7f474c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 -- 172.21.15.73:0/1590476643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c102c50 msgr2=0x7f474c198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c102c50 0x7f474c198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 -- 172.21.15.73:0/1590476643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c19d180 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f474ad9d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f474400eaa0 tx=0x7f474400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f474400cdb0 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4744005020 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c19d470 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47440116d0 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.839+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c1a4d00 con 0x7f474c10bf90 2024-06-06T07:01:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.840+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f474c04f760 con 0x7f474c10bf90 2024-06-06T07:01:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.841+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f47440051d0 con 0x7f474c10bf90 2024-06-06T07:01:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.842+0000 7f4743fff640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 0x7f47380a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.842+0000 7f474a59c640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 0x7f47380a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.842+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f47440cfaa0 con 0x7f474c10bf90 2024-06-06T07:01:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.843+0000 7f474a59c640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 0x7f47380a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f473c031440 tx=0x7f473c005b50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:31 smithi170 ceph-mon[36108]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-06-06T07:01:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:31 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2576391892' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:01:31.847 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:31.847+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4744016030 con 0x7f474c10bf90 2024-06-06T07:01:31.984 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:32.045 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.044+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f474c106b80 con 0x7f474c10bf90 2024-06-06T07:01:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.048+0000 7f4743fff640 1 -- 172.21.15.73:0/1590476643 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v31)=0 v31) ==== 129+0+0 (secure 0 0 0) 0x7f474409bb30 con 0x7f474c10bf90 2024-06-06T07:01:32.051 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.050+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 msgr2=0x7f47380a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.050+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 0x7f47380a5300 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f473c031440 tx=0x7f473c005b50 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.050+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 msgr2=0x7f474c19ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.050+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f474400eaa0 tx=0x7f474400ef60 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 shutdown_connections 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f474c10bf90 0x7f474c19ca50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47380a2e40 0x7f47380a5300 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f474c103570 0x7f474c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 --2- 172.21.15.73:0/1590476643 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f474c102c50 0x7f474c198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 >> 172.21.15.73:0/1590476643 conn(0x7f474c077a20 msgr2=0x7f474c0fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 shutdown_connections 2024-06-06T07:01:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.051+0000 7f475084d640 1 -- 172.21.15.73:0/1590476643 wait complete. 2024-06-06T07:01:32.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 -- 172.21.15.73:0/1382484729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b01008d0 msgr2=0x7fa1b0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/1382484729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b01008d0 0x7fa1b0100d50 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009930 tx=0x7fa1a002a420 comp rx=0 tx=0).stop 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 -- 172.21.15.73:0/1382484729 shutdown_connections 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/1382484729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/1382484729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b01008d0 0x7fa1b0100d50 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/1382484729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b0106890 0x7fa1b0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 -- 172.21.15.73:0/1382484729 >> 172.21.15.73:0/1382484729 conn(0x7fa1b0077a20 msgr2=0x7fa1b0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 -- 172.21.15.73:0/1382484729 shutdown_connections 2024-06-06T07:01:32.172 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.168+0000 7fa1b6f14640 1 -- 172.21.15.73:0/1382484729 wait complete. 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 Processor -- start 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 -- start start 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b01008d0 0x7fa1b0196f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b01974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110b10 con 0x7fa1b0106890 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110990 con 0x7fa1b0101290 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b6f14640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b0110c90 con 0x7fa1b01008d0 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b54b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b54b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43050/0 (socket says 172.21.15.73:43050) 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1b54b1640 1 -- 172.21.15.73:0/3285016420 learned_addr learned my addr 172.21.15.73:0/3285016420 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.169+0000 7fa1a7fff640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b01974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.173 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b4cb0640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b01008d0 0x7fa1b0196f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 -- 172.21.15.73:0/3285016420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b01008d0 msgr2=0x7fa1b0196f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b01008d0 0x7fa1b0196f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 -- 172.21.15.73:0/3285016420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 msgr2=0x7fa1b01974d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b01974d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 -- 172.21.15.73:0/3285016420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b019bf80 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1a7fff640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b01974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b54b1640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800eb10 tx=0x7fa1a800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8004b20 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1a8004cc0 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a8010690 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b019c270 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.170+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b01a3b00 con 0x7fa1b0106890 2024-06-06T07:01:32.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.171+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b004f760 con 0x7fa1b0106890 2024-06-06T07:01:32.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.172+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fa1a8005020 con 0x7fa1b0106890 2024-06-06T07:01:32.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.173+0000 7fa1a5ffb640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 0x7fa1980a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.173+0000 7fa1b4cb0640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 0x7fa1980a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.173+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fa1a80d00a0 con 0x7fa1b0106890 2024-06-06T07:01:32.175 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.173+0000 7fa1b4cb0640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 0x7fa1980a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00b9c0 tx=0x7fa1ac005c50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.177 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.177+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1a809c7a0 con 0x7fa1b0106890 2024-06-06T07:01:32.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.360+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fa1b002d2e0 con 0x7fa1b0106890 2024-06-06T07:01:32.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.364+0000 7fa1a5ffb640 1 -- 172.21.15.73:0/3285016420 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v32)=0 v32) ==== 138+0+0 (secure 0 0 0) 0x7fa1a809c0a0 con 0x7fa1b0106890 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 msgr2=0x7fa1980a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 0x7fa1980a5300 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac00b9c0 tx=0x7fa1ac005c50 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 msgr2=0x7fa1b019b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa1a800eb10 tx=0x7fa1a800c2d0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 shutdown_connections 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1b0106890 0x7fa1b019b850 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa1b0101290 0x7fa1b01974d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa1980a2e40 0x7fa1980a5300 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 --2- 172.21.15.73:0/3285016420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa1b01008d0 0x7fa1b0196f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 >> 172.21.15.73:0/3285016420 conn(0x7fa1b0077a20 msgr2=0x7fa1b01033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 shutdown_connections 2024-06-06T07:01:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.367+0000 7fa1b6f14640 1 -- 172.21.15.73:0/3285016420 wait complete. 2024-06-06T07:01:32.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-06-06T07:01:32.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 -- 172.21.15.73:0/3344497668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 msgr2=0x7ff08410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 --2- 172.21.15.73:0/3344497668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff08410e880 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff07c009cd0 tx=0x7ff07c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 -- 172.21.15.73:0/3344497668 shutdown_connections 2024-06-06T07:01:32.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 --2- 172.21.15.73:0/3344497668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff08410e880 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 --2- 172.21.15.73:0/3344497668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841019b0 0x7ff084101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 --2- 172.21.15.73:0/3344497668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841079c0 0x7ff084107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 -- 172.21.15.73:0/3344497668 >> 172.21.15.73:0/3344497668 conn(0x7ff084078e30 msgr2=0x7ff084079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 -- 172.21.15.73:0/3344497668 shutdown_connections 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.490+0000 7ff088e9e640 1 -- 172.21.15.73:0/3344497668 wait complete. 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 Processor -- start 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 -- start start 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 0x7ff084198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841079c0 0x7ff08419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff084111c50 con 0x7ff084102370 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff084111ad0 con 0x7ff0841019b0 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff088e9e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff084111dd0 con 0x7ff0841079c0 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff07bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43066/0 (socket says 172.21.15.73:43066) 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff07bfff640 1 -- 172.21.15.73:0/1261703596 learned_addr learned my addr 172.21.15.73:0/1261703596 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff08259c640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 0x7ff084198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff07bfff640 1 -- 172.21.15.73:0/1261703596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841079c0 msgr2=0x7ff08419c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.491+0000 7ff082d9d640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841079c0 0x7ff08419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07bfff640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841079c0 0x7ff08419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07bfff640 1 -- 172.21.15.73:0/1261703596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 msgr2=0x7ff084198050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07bfff640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 0x7ff084198050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07bfff640 1 -- 172.21.15.73:0/1261703596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff08419cff0 con 0x7ff084102370 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff08259c640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 0x7ff084198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07bfff640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c920 tx=0x7ff06c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c007c70 con 0x7ff084102370 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff08419d2e0 con 0x7ff084102370 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff06c007e10 con 0x7ff084102370 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c018820 con 0x7ff084102370 2024-06-06T07:01:32.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.492+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0841a4b70 con 0x7ff084102370 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.493+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff08404f760 con 0x7ff084102370 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.494+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ff06c0189c0 con 0x7ff084102370 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.495+0000 7ff07b7fe640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.495+0000 7ff08259c640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.495+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7ff06c0cfba0 con 0x7ff084102370 2024-06-06T07:01:32.497 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.495+0000 7ff08259c640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 0x7ff0700a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff074005fd0 tx=0x7ff074005850 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.499+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff06c014030 con 0x7ff084102370 2024-06-06T07:01:32.682 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.681+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7ff0841a4f10 con 0x7ff084102370 2024-06-06T07:01:32.686 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.685+0000 7ff07b7fe640 1 -- 172.21.15.73:0/1261703596 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v33)=0 v33) ==== 133+0+0 (secure 0 0 0) 0x7ff06c00de20 con 0x7ff084102370 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 msgr2=0x7ff0700a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 0x7ff0700a5300 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff074005fd0 tx=0x7ff074005850 comp rx=0 tx=0).stop 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 msgr2=0x7ff084198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff06c00c920 tx=0x7ff06c00cde0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 shutdown_connections 2024-06-06T07:01:32.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0700a2e40 0x7ff0700a5300 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff0841079c0 0x7ff08419c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff084102370 0x7ff084198590 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 --2- 172.21.15.73:0/1261703596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff0841019b0 0x7ff084198050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 >> 172.21.15.73:0/1261703596 conn(0x7ff084078e30 msgr2=0x7ff0840ff160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 shutdown_connections 2024-06-06T07:01:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.688+0000 7ff088e9e640 1 -- 172.21.15.73:0/1261703596 wait complete. 2024-06-06T07:01:32.702 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-06-06T07:01:32.814 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- 172.21.15.73:0/2831530142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541068b0 msgr2=0x7ffa54106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2831530142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541068b0 0x7ffa54106c90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009930 tx=0x7ffa4402a420 comp rx=0 tx=0).stop 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- 172.21.15.73:0/2831530142 shutdown_connections 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2831530142 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 0x7ffa5410d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2831530142 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541008a0 0x7ffa54100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2831530142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541068b0 0x7ffa54106c90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- 172.21.15.73:0/2831530142 >> 172.21.15.73:0/2831530142 conn(0x7ffa540779c0 msgr2=0x7ffa54077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- 172.21.15.73:0/2831530142 shutdown_connections 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- 172.21.15.73:0/2831530142 wait complete. 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 Processor -- start 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.812+0000 7ffa5a815640 1 -- start start 2024-06-06T07:01:32.815 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 0x7ffa54197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541068b0 0x7ffa5419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54110b70 con 0x7ffa541008a0 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541109f0 con 0x7ffa54101260 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54110cf0 con 0x7ffa541068b0 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43088/0 (socket says 172.21.15.73:43088) 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 -- 172.21.15.73:0/2638048150 learned_addr learned my addr 172.21.15.73:0/2638048150 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa58db2640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541068b0 0x7ffa5419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa537fe640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 0x7ffa54197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 -- 172.21.15.73:0/2638048150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541068b0 msgr2=0x7ffa5419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541068b0 0x7ffa5419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 -- 172.21.15.73:0/2638048150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 msgr2=0x7ffa54197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 0x7ffa54197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 -- 172.21.15.73:0/2638048150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419c0c0 con 0x7ffa541008a0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa53fff640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa44004770 tx=0x7ffa44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44035050 con 0x7ffa541008a0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa44003aa0 con 0x7ffa541008a0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419c350 con 0x7ffa541008a0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44038ae0 con 0x7ffa541008a0 2024-06-06T07:01:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.813+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa54069c40 con 0x7ffa541008a0 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.814+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54101fe0 con 0x7ffa541008a0 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.816+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7ffa44003c50 con 0x7ffa541008a0 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.816+0000 7ffa517fa640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 0x7ffa3c0a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.816+0000 7ffa537fe640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 0x7ffa3c0a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.816+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7ffa440f1c90 con 0x7ffa541008a0 2024-06-06T07:01:32.818 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.817+0000 7ffa537fe640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 0x7ffa3c0a5300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa54078100 tx=0x7ffa4800b390 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:32.820+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffa440be390 con 0x7ffa541008a0 2024-06-06T07:01:33.022 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.021+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7ffa54069ea0 con 0x7ffa3c0a2e40 2024-06-06T07:01:33.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.083+0000 7ffa517fa640 1 -- 172.21.15.73:0/2638048150 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffa54069ea0 con 0x7ffa3c0a2e40 2024-06-06T07:01:33.084 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path / 2024-06-06T07:01:33.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 msgr2=0x7ffa3c0a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 0x7ffa3c0a5300 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa54078100 tx=0x7ffa4800b390 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 msgr2=0x7ffa541970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ffa44004770 tx=0x7ffa44004270 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 shutdown_connections 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa3c0a2e40 0x7ffa3c0a5300 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa541068b0 0x7ffa5419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa54101260 0x7ffa54197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 --2- 172.21.15.73:0/2638048150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa541008a0 0x7ffa541970d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.085+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 >> 172.21.15.73:0/2638048150 conn(0x7ffa540779c0 msgr2=0x7ffa54103510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.086+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 shutdown_connections 2024-06-06T07:01:33.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.086+0000 7ffa5a815640 1 -- 172.21.15.73:0/2638048150 wait complete. 2024-06-06T07:01:33.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-06-06T07:01:33.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 -- 172.21.15.73:0/1335457675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8568100880 msgr2=0x7f856810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 --2- 172.21.15.73:0/1335457675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8568100880 0x7f856810e850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8560009cd0 tx=0x7f856002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.208 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 -- 172.21.15.73:0/1335457675 shutdown_connections 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 --2- 172.21.15.73:0/1335457675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8568100880 0x7f856810e850 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 --2- 172.21.15.73:0/1335457675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 0x7f8568100340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 --2- 172.21.15.73:0/1335457675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85681069a0 0x7f8568106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 -- 172.21.15.73:0/1335457675 >> 172.21.15.73:0/1335457675 conn(0x7f8568076e30 msgr2=0x7f85680fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 -- 172.21.15.73:0/1335457675 shutdown_connections 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.205+0000 7f856f82a640 1 -- 172.21.15.73:0/1335457675 wait complete. 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 Processor -- start 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 -- start start 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 0x7f85681981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 0x7f85681986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111d60 con 0x7f85681069a0 2024-06-06T07:01:33.209 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111be0 con 0x7f8568100880 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856f82a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8568111ee0 con 0x7f85680ffec0 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43108/0 (socket says 172.21.15.73:43108) 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 -- 172.21.15.73:0/2416121074 learned_addr learned my addr 172.21.15.73:0/2416121074 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856d5c6640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 0x7f85681981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 -- 172.21.15.73:0/2416121074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 msgr2=0x7f85681981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 0x7f85681981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 -- 172.21.15.73:0/2416121074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 msgr2=0x7f85681986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856cdc5640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 0x7f85681986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 0x7f85681986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856ddc7640 1 -- 172.21.15.73:0/2416121074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f856819d140 con 0x7f85681069a0 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.206+0000 7f856cdc5640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 0x7f85681986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:33.210 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f856ddc7640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8560005cc0 tx=0x7f8560003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560004240 con 0x7f85681069a0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f856819d3d0 con 0x7f85681069a0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85600305a0 con 0x7f85681069a0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600466c0 con 0x7f85681069a0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.207+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85681a4cc0 con 0x7f85681069a0 2024-06-06T07:01:33.211 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.208+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f856804f760 con 0x7f85681069a0 2024-06-06T07:01:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.209+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f8560030ac0 con 0x7f85681069a0 2024-06-06T07:01:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.209+0000 7f855e7fc640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 0x7f85540a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.210+0000 7f856d5c6640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 0x7f85540a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.210+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f856002e030 con 0x7f85681069a0 2024-06-06T07:01:33.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.210+0000 7f856d5c6640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 0x7f85540a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f856400abe0 tx=0x7f856400a2d0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.214 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.214+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f856004b060 con 0x7f85681069a0 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1590476643' entity='client.admin' 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3285016420' entity='client.admin' 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1261703596' entity='client.admin' 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.303 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.317 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1590476643' entity='client.admin' 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3285016420' entity='client.admin' 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.318 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1261703596' entity='client.admin' 2024-06-06T07:01:33.319 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.319 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.319 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.319 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1590476643' entity='client.admin' 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3285016420' entity='client.admin' 2024-06-06T07:01:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1261703596' entity='client.admin' 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:01:33.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:01:33.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.416+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f8568061de0 con 0x7f85540a2e40 2024-06-06T07:01:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.488+0000 7f855e7fc640 1 -- 172.21.15.73:0/2416121074 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8568061de0 con 0x7f85540a2e40 2024-06-06T07:01:33.489 INFO:teuthology.orchestra.run.smithi073.stdout:Retention added to path / 2024-06-06T07:01:33.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 msgr2=0x7f85540a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 0x7f85540a5300 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f856400abe0 tx=0x7f856400a2d0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 msgr2=0x7f856819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8560005cc0 tx=0x7f8560003ba0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 shutdown_connections 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85681069a0 0x7f856819ca10 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8568100880 0x7f85681986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85540a2e40 0x7f85540a5300 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 --2- 172.21.15.73:0/2416121074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85680ffec0 0x7f85681981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.490+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 >> 172.21.15.73:0/2416121074 conn(0x7f8568076e30 msgr2=0x7f85681048a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.491+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 shutdown_connections 2024-06-06T07:01:33.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.491+0000 7f856f82a640 1 -- 172.21.15.73:0/2416121074 wait complete. 2024-06-06T07:01:33.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1564485799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 msgr2=0x7f9d1c107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1564485799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c107d80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009930 tx=0x7f9d1802a420 comp rx=0 tx=0).stop 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1564485799 shutdown_connections 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1564485799 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c102350 0x7f9d1c10e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1564485799 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c101990 0x7f9d1c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1564485799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c107d80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1564485799 >> 172.21.15.73:0/1564485799 conn(0x7f9d1c0fd460 msgr2=0x7f9d1c0ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1564485799 shutdown_connections 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1564485799 wait complete. 2024-06-06T07:01:33.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 Processor -- start 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 -- start start 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.607+0000 7f9d23ba2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c101990 0x7f9d1c198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 0x7f9d1c1985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111c60 con 0x7f9d1c1079a0 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111ae0 con 0x7f9d1c102350 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c111de0 con 0x7f9d1c101990 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43124/0 (socket says 172.21.15.73:43124) 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 -- 172.21.15.73:0/1666721943 learned_addr learned my addr 172.21.15.73:0/1666721943 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2193e640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c101990 0x7f9d1c198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 -- 172.21.15.73:0/1666721943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c101990 msgr2=0x7f9d1c198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2113d640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 0x7f9d1c1985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c101990 0x7f9d1c198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 -- 172.21.15.73:0/1666721943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 msgr2=0x7f9d1c1985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 0x7f9d1c1985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 -- 172.21.15.73:0/1666721943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c19d050 con 0x7f9d1c1079a0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2113d640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 0x7f9d1c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d2213f640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eaa0 tx=0x7f9d1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400cdb0 con 0x7f9d1c1079a0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d14005020 con 0x7f9d1c1079a0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d140116d0 con 0x7f9d1c1079a0 2024-06-06T07:01:33.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c19d340 con 0x7f9d1c1079a0 2024-06-06T07:01:33.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.608+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c1a4bd0 con 0x7f9d1c1079a0 2024-06-06T07:01:33.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.609+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c04f760 con 0x7f9d1c1079a0 2024-06-06T07:01:33.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.610+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f9d140051d0 con 0x7f9d1c1079a0 2024-06-06T07:01:33.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.611+0000 7f9d12ffd640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 0x7f9d080a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.611+0000 7f9d2193e640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 0x7f9d080a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.611+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f9d140cfa50 con 0x7f9d1c1079a0 2024-06-06T07:01:33.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.611+0000 7f9d2193e640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 0x7f9d080a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d18000bc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.615+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9d14016030 con 0x7f9d1c1079a0 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 -- 172.21.15.73:0/1418454831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 msgr2=0x7f05fc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 --2- 172.21.15.73:0/1418454831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc101840 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec009ca0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 -- 172.21.15.73:0/1418454831 shutdown_connections 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 --2- 172.21.15.73:0/1418454831 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc101d80 0x7f05fc10e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 --2- 172.21.15.73:0/1418454831 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc107430 0x7f05fc107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 --2- 172.21.15.73:0/1418454831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc101840 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 -- 172.21.15.73:0/1418454831 >> 172.21.15.73:0/1418454831 conn(0x7f05fc0783d0 msgr2=0x7f05fc0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 -- 172.21.15.73:0/1418454831 shutdown_connections 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.746+0000 7f0603297700 1 -- 172.21.15.73:0/1418454831 wait complete. 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 Processor -- start 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 -- start start 2024-06-06T07:01:33.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc101d80 0x7f05fc199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc107430 0x7f05fc1a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc111580 con 0x7f05fc1013c0 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc111400 con 0x7f05fc101d80 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0603297700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc111700 con 0x7f05fc107430 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43128/0 (socket says 172.21.15.73:43128) 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 -- 172.21.15.73:0/612457753 learned_addr learned my addr 172.21.15.73:0/612457753 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0600832700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc101d80 0x7f05fc199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601834700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc107430 0x7f05fc1a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 -- 172.21.15.73:0/612457753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc107430 msgr2=0x7f05fc1a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc107430 0x7f05fc1a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 -- 172.21.15.73:0/612457753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc101d80 msgr2=0x7f05fc199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc101d80 0x7f05fc199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 -- 172.21.15.73:0/612457753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05fc1a4fa0 con 0x7f05fc1013c0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.747+0000 7f0601033700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f05f800d980 tx=0x7f05f800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.748+0000 7f05f27fc700 1 -- 172.21.15.73:0/612457753 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8016070 con 0x7f05fc1013c0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.748+0000 7f05f27fc700 1 -- 172.21.15.73:0/612457753 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f8004830 con 0x7f05fc1013c0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.748+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05fc1a51d0 con 0x7f05fc1013c0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.748+0000 7f05f27fc700 1 -- 172.21.15.73:0/612457753 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f8005530 con 0x7f05fc1013c0 2024-06-06T07:01:33.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.748+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05fc1a56e0 con 0x7f05fc1013c0 2024-06-06T07:01:33.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.750+0000 7f05f27fc700 1 -- 172.21.15.73:0/612457753 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f05f80049e0 con 0x7f05fc1013c0 2024-06-06T07:01:33.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.751+0000 7f05f27fc700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 0x7f05e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.751+0000 7f05f27fc700 1 -- 172.21.15.73:0/612457753 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f05f80cf650 con 0x7f05fc1013c0 2024-06-06T07:01:33.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.751+0000 7f0600832700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 0x7f05e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.751+0000 7f0600832700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 0x7f05e80a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00b5c0 tx=0x7f05ec005960 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 Processor -- start 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 -- start start 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 0x55f35a1583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a152820 0x55f35a152c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35a044fe0 con 0x55f35a158900 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f35a0a38d0 con 0x55f35a152820 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f060db1ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f35a00b4e0 con 0x55f35a157fe0 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f0600832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f0600832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43138/0 (socket says 172.21.15.73:43138) 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f0600832700 1 -- 172.21.15.73:0/2934290605 learned_addr learned my addr 172.21.15.73:0/2934290605 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.752+0000 7f0601033700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 0x55f35a1583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0601834700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a152820 0x55f35a152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 -- 172.21.15.73:0/2934290605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 msgr2=0x55f35a1583c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 0x55f35a1583c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 -- 172.21.15.73:0/2934290605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a152820 msgr2=0x55f35a152c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a152820 0x55f35a152c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 -- 172.21.15.73:0/2934290605 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f359ff2950 con 0x55f35a158900 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0601033700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 0x55f35a1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f0600832700 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a02bc97d34c5e69e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f05e77fe700 1 -- 172.21.15.73:0/2934290605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec034070 con 0x55f35a158900 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f05e77fe700 1 -- 172.21.15.73:0/2934290605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05ec00be40 con 0x55f35a158900 2024-06-06T07:01:33.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.753+0000 7f05e77fe700 1 -- 172.21.15.73:0/2934290605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec02f900 con 0x55f35a158900 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- 172.21.15.73:0/2934290605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 msgr2=0x55f35a1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f05ec009990 tx=0x7f05ec01f000 comp rx=0 tx=0).stop 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- 172.21.15.73:0/2934290605 shutdown_connections 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a157fe0 0x55f35a1583c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a152820 0x55f35a152c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2934290605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a158900 0x55f35a1522e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- 172.21.15.73:0/2934290605 >> 172.21.15.73:0/2934290605 conn(0x55f35a07f100 msgr2=0x55f35a14d2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- 172.21.15.73:0/2934290605 shutdown_connections 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- 172.21.15.73:0/2934290605 wait complete. 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 -- start start 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.754+0000 7f060db1ab80 1 Processor -- start 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 0x55f35a15be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f35a044040 con 0x55f35a157fe0 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f35a09f890 con 0x55f35a158900 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f35a15f280 con 0x55f35a152820 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f060db1ab80 10 client.0 _ll_register_callbacks cb 0x55f35a039630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0601033700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43154/0 (socket says 172.21.15.73:43154) 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 -- 172.21.15.73:0/1536839644 learned_addr learned my addr 172.21.15.73:0/1536839644 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0601834700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 0x55f35a15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 -- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 msgr2=0x55f35a15e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 -- 172.21.15.73:0/1536839644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 msgr2=0x55f35a15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 0x55f35a15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 -- 172.21.15.73:0/1536839644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35a0697d0 con 0x55f35a157fe0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0601033700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0601834700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 0x55f35a15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f0600832700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05ec005f50 tx=0x7f05ec02f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e56eabc7f56ec9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.755+0000 7f05e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05ec03f060 con 0x55f35a157fe0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f05ec024480 con 0x55f35a157fe0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f35a0b0dd0 con 0x55f35a157fe0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05ec046050 con 0x55f35a157fe0 2024-06-06T07:01:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 10 client.15453 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f35a1699f0 con 0x55f35a157fe0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55f35a0b0dd0 con 0x55f35a157fe0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f05ec00f790 con 0x55f35a157fe0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 10 client.15453 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 20 client.15453 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a16bec0 0x55f35a16e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 4 client.15453 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.756+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f35a0ab350 con 0x55f35a16bec0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.757+0000 7f0601033700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a16bec0 0x55f35a16e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.757+0000 7f0601033700 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a16bec0 0x55f35a16e2b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55ba9d4c4670fc26 server_cookie=a82a5a544879d1f7 in_seq=0 out_seq=0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.757+0000 7f05e4ff9700 10 client.15453 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f05e4ff9700 1 -- 172.21.15.73:0/1536839644 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f35a0ab350 con 0x55f35a16bec0 2024-06-06T07:01:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f05e4ff9700 10 client.15453 handle_command_reply: tid=0 2024-06-06T07:01:33.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 client.15453 shutdown 2024-06-06T07:01:33.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 msgr2=0x55f35a15b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f05ec005f50 tx=0x7f05ec02f7b0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 shutdown_connections 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a16bec0 0x55f35a16e2b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a158900 0x55f35a15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f05e4ff9700 0 client.15453 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 --2- 172.21.15.73:0/1536839644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a15b900 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 >> 172.21.15.73:0/1536839644 conn(0x55f35a07f100 msgr2=0x55f35a14c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 shutdown_connections 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 1 -- 172.21.15.73:0/1536839644 wait complete. 2024-06-06T07:01:33.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.758+0000 7f060db1ab80 20 client.15453 trim_cache size 0 max 16384 2024-06-06T07:01:33.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 Processor -- start 2024-06-06T07:01:33.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 -- start start 2024-06-06T07:01:33.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a157fe0 0x55f35a14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1ae460 con 0x55f35a07f100 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1affa0 con 0x55f35a157fe0 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f060db1ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1aeb30 con 0x55f35a152820 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43158/0 (socket says 172.21.15.73:43158) 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 -- 172.21.15.73:0/2191808366 learned_addr learned my addr 172.21.15.73:0/2191808366 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0600832700 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 -- 172.21.15.73:0/2191808366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 msgr2=0x55f35a15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 -- 172.21.15.73:0/2191808366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a157fe0 msgr2=0x55f35a14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a157fe0 0x55f35a14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 -- 172.21.15.73:0/2191808366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f35a159e70 con 0x55f35a07f100 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f0601033700 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000c00 tx=0x7f05f8097980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c5c28009b1ddb19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.779+0000 7f05e6ffd700 1 -- 172.21.15.73:0/2191808366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f809b650 con 0x55f35a07f100 2024-06-06T07:01:33.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f05e6ffd700 1 -- 172.21.15.73:0/2191808366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f809fa00 con 0x55f35a07f100 2024-06-06T07:01:33.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f05e6ffd700 1 -- 172.21.15.73:0/2191808366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f809bb10 con 0x55f35a07f100 2024-06-06T07:01:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 -- 172.21.15.73:0/2191808366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 msgr2=0x55f35a1522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000c00 tx=0x7f05f8097980 comp rx=0 tx=0).stop 2024-06-06T07:01:33.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 -- 172.21.15.73:0/2191808366 shutdown_connections 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a157fe0 0x55f35a14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a07f100 0x55f35a1522e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 --2- 172.21.15.73:0/2191808366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 -- 172.21.15.73:0/2191808366 >> 172.21.15.73:0/2191808366 conn(0x55f35a156220 msgr2=0x55f35a15d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 -- 172.21.15.73:0/2191808366 shutdown_connections 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.780+0000 7f060db1ab80 1 -- 172.21.15.73:0/2191808366 wait complete. 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 -- start start 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 Processor -- start 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a07f100 0x55f35a16f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a16f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1af8d0 con 0x55f35a157fe0 2024-06-06T07:01:33.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1af200 con 0x55f35a07f100 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f05fc1ac400 con 0x55f35a152820 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f060db1ab80 10 client.0 _ll_register_callbacks cb 0x55f35a039630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f0601834700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f0601834700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43160/0 (socket says 172.21.15.73:43160) 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f0601834700 1 -- 172.21.15.73:0/626983755 learned_addr learned my addr 172.21.15.73:0/626983755 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f0600832700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a16f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.781+0000 7f0601033700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a07f100 0x55f35a16f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 -- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 msgr2=0x55f35a16f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a16f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 -- 172.21.15.73:0/626983755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a07f100 msgr2=0x55f35a16f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a07f100 0x55f35a16f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 -- 172.21.15.73:0/626983755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f35a1d9ec0 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f0601834700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05f4004270 tx=0x7f05f4004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba568c5426dc803 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f05e4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05f401a020 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f05f4016870 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05f809fa00 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f05f4025050 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f060db1ab80 10 client.15459 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f35a1db240 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.782+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f05f809fa00 con 0x55f35a157fe0 2024-06-06T07:01:33.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f05f4007be0 con 0x55f35a157fe0 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 10 client.15459 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 20 client.15459 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a158900 0x55f35a1dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 4 client.15459 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f35a05efd0 con 0x55f35a158900 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f0601033700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a158900 0x55f35a1dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f05e4ff9700 10 client.15459 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.783+0000 7f0601033700 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a158900 0x55f35a1dda10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f620f59a0359afd server_cookie=1492c7fd8ec4de88 in_seq=0 out_seq=0 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f05e4ff9700 1 -- 172.21.15.73:0/626983755 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f35a05efd0 con 0x55f35a158900 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f05e4ff9700 10 client.15459 handle_command_reply: tid=0 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 client.15459 shutdown 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 msgr2=0x55f35a16fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f05f4004270 tx=0x7f05f4004580 comp rx=0 tx=0).stop 2024-06-06T07:01:33.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 shutdown_connections 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f35a158900 0x55f35a1dda10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f35a157fe0 0x55f35a16fdb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f35a07f100 0x55f35a16f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f05e4ff9700 0 client.15459 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 --2- 172.21.15.73:0/626983755 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f35a152820 0x55f35a16f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.784+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 >> 172.21.15.73:0/626983755 conn(0x55f35a156220 msgr2=0x55f35a156a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 shutdown_connections 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f060db1ab80 1 -- 172.21.15.73:0/626983755 wait complete. 2024-06-06T07:01:33.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f060db1ab80 20 client.15459 trim_cache size 0 max 16384 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stdout: "return_code": 0, 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_tag": "a1ba64f1-6d4c-448c-90ff-95721c06e9d7", 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": "asynchronous" 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:33.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 msgr2=0x7f05e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 0x7f05e80a52e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f05ec00b5c0 tx=0x7f05ec005960 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 msgr2=0x7f05fc1999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f05f800d980 tx=0x7f05f800dc90 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 shutdown_connections 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f05e80a2e20 0x7f05e80a52e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f05fc107430 0x7f05fc1a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc1013c0 0x7f05fc1999e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 --2- 172.21.15.73:0/612457753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f05fc101d80 0x7f05fc199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 >> 172.21.15.73:0/612457753 conn(0x7f05fc0783d0 msgr2=0x7f05fc100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 shutdown_connections 2024-06-06T07:01:33.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.785+0000 7f0603297700 1 -- 172.21.15.73:0/612457753 wait complete. 2024-06-06T07:01:33.830 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.830+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f9d1c0fe1e0 con 0x7f9d080a2ec0 2024-06-06T07:01:33.838 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.838+0000 7f9d12ffd640 1 -- 172.21.15.73:0/1666721943 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f9d1c0fe1e0 con 0x7f9d080a2ec0 2024-06-06T07:01:33.839 INFO:teuthology.orchestra.run.smithi073.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T07:01:33", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 msgr2=0x7f9d080a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 0x7f9d080a5380 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9d18009900 tx=0x7f9d18000bc0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 msgr2=0x7f9d1c19c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9d1400eaa0 tx=0x7f9d1400ef60 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 shutdown_connections 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d1c1079a0 0x7f9d1c19c920 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d1c102350 0x7f9d1c1985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d080a2ec0 0x7f9d080a5380 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 --2- 172.21.15.73:0/1666721943 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d1c101990 0x7f9d1c198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.841 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 >> 172.21.15.73:0/1666721943 conn(0x7f9d1c0fd460 msgr2=0x7f9d1c0fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 shutdown_connections 2024-06-06T07:01:33.842 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.840+0000 7f9d23ba2640 1 -- 172.21.15.73:0/1666721943 wait complete. 2024-06-06T07:01:33.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 -- 172.21.15.73:0/1740709158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b281079a0 msgr2=0x7f6b28107d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1740709158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b281079a0 0x7f6b28107d80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009930 tx=0x7f6b1802a420 comp rx=0 tx=0).stop 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 -- 172.21.15.73:0/1740709158 shutdown_connections 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1740709158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b2806ab40 0x7f6b2810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1740709158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806a180 0x7f6b2806a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1740709158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b281079a0 0x7f6b28107d80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.957 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 -- 172.21.15.73:0/1740709158 >> 172.21.15.73:0/1740709158 conn(0x7f6b28077680 msgr2=0x7f6b28077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 -- 172.21.15.73:0/1740709158 shutdown_connections 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.954+0000 7f6b2d279640 1 -- 172.21.15.73:0/1740709158 wait complete. 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 Processor -- start 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 -- start start 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806ab40 0x7f6b28198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b281079a0 0x7f6b2819c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111c30 con 0x7f6b2806a180 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111ab0 con 0x7f6b281079a0 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b2d279640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b28111db0 con 0x7f6b2806ab40 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43184/0 (socket says 172.21.15.73:43184) 2024-06-06T07:01:33.958 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 -- 172.21.15.73:0/1336554706 learned_addr learned my addr 172.21.15.73:0/1336554706 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b267fc640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806ab40 0x7f6b28198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b277fe640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b281079a0 0x7f6b2819c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 -- 172.21.15.73:0/1336554706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806ab40 msgr2=0x7f6b28198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806ab40 0x7f6b28198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 -- 172.21.15.73:0/1336554706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b281079a0 msgr2=0x7f6b2819c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b281079a0 0x7f6b2819c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.955+0000 7f6b26ffd640 1 -- 172.21.15.73:0/1336554706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b2819d120 con 0x7f6b2806a180 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b26ffd640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b18004770 tx=0x7f6b180042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1803f070 con 0x7f6b2806a180 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b2819d3b0 con 0x7f6b2806a180 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b1803b460 con 0x7f6b2806a180 2024-06-06T07:01:33.959 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b18035070 con 0x7f6b2806a180 2024-06-06T07:01:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.956+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b281a4ca0 con 0x7f6b2806a180 2024-06-06T07:01:33.960 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.957+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b281030d0 con 0x7f6b2806a180 2024-06-06T07:01:33.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.958+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f6b1803b610 con 0x7f6b2806a180 2024-06-06T07:01:33.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.959+0000 7f6b0ffff640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 0x7f6b100a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:33.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.959+0000 7f6b267fc640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 0x7f6b100a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:33.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.959+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f6b180effe0 con 0x7f6b2806a180 2024-06-06T07:01:33.961 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.959+0000 7f6b267fc640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 0x7f6b100a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b28100650 tx=0x7f6b1c00b410 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:33.963 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:33.962+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6b180bc6e0 con 0x7f6b2806a180 2024-06-06T07:01:34.174 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.174+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f6b28061de0 con 0x7f6b100a2e40 2024-06-06T07:01:34.181 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.180+0000 7f6b0ffff640 1 -- 172.21.15.73:0/1336554706 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f6b28061de0 con 0x7f6b100a2e40 2024-06-06T07:01:34.181 INFO:teuthology.orchestra.run.smithi073.stdout:/ 1m 6m3h 2024-06-06T07:01:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 msgr2=0x7f6b100a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 0x7f6b100a5300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6b28100650 tx=0x7f6b1c00b410 comp rx=0 tx=0).stop 2024-06-06T07:01:34.183 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 msgr2=0x7f6b28198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6b18004770 tx=0x7f6b180042f0 comp rx=0 tx=0).stop 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 shutdown_connections 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b281079a0 0x7f6b2819c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b100a2e40 0x7f6b100a5300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b2806ab40 0x7f6b28198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 --2- 172.21.15.73:0/1336554706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b2806a180 0x7f6b28198130 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 >> 172.21.15.73:0/1336554706 conn(0x7f6b28077680 msgr2=0x7f6b280fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 shutdown_connections 2024-06-06T07:01:34.184 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:34.182+0000 7f6b2d279640 1 -- 172.21.15.73:0/1336554706 wait complete. 2024-06-06T07:01:34.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-06-06T07:01:34.220 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-06-06T07:01:34.231 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-06T07:01:34.250 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:01:34.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-06T07:01:34.263 INFO:teuthology.run_tasks:Running task workunit... 2024-06-06T07:01:34.277 INFO:tasks.workunit:Pulling workunits from ref 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T07:01:34.277 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-06T07:01:34.277 INFO:tasks.workunit:timeout=3h 2024-06-06T07:01:34.277 INFO:tasks.workunit:cleanup=True 2024-06-06T07:01:34.278 DEBUG:teuthology.orchestra.run.smithi073:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:fusefs_t:s0 2024-06-06T07:01:34.294 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-06-06 06:58:27.181144001 +0000 2024-06-06T07:01:34.295 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-06-06 06:58:27.181144001 +0000 2024-06-06T07:01:34.295 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-06-06 07:01:19.298434459 +0000 2024-06-06T07:01:34.295 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-06-06T07:01:34.295 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-06T07:01:34.295 DEBUG:teuthology.orchestra.run.smithi073:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-06T07:01:34.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:34 smithi170 ceph-mon[36108]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:34 smithi170 ceph-mon[36108]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.356 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:34 smithi073 ceph-mon[32303]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.356 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:34 smithi073 ceph-mon[32303]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.366 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 3d746ca5346f34eef7e826a1eb71e59e02b682ae 2024-06-06T07:01:34.423 INFO:tasks.workunit.client.0.smithi073.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-06T07:01:34.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:34 smithi136 ceph-mon[35901]: from='client.15429 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:34 smithi136 ceph-mon[35901]: from='client.15435 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:34.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:01:34.899 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:01:35.272 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-06T07:01:35.272 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: scrub queued for path: / 2024-06-06T07:01:35.272 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.272 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: scrub summary: active paths [/] 2024-06-06T07:01:35.272 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.273 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:35.273 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:35 smithi073 ceph-mon[32303]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-06T07:01:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: scrub queued for path: / 2024-06-06T07:01:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: scrub summary: active paths [/] 2024-06-06T07:01:35.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:35.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:35 smithi170 ceph-mon[36108]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: pgmap v9: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 0 B/s wr, 20 op/s 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: scrub queued for path: / 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: scrub summary: active paths [/] 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:01:35.477 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: from='client.15441 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:35.478 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:35 smithi136 ceph-mon[35901]: from='client.15465 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:01:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 -- 172.21.15.73:0/274414023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c1030f0 msgr2=0x7fae3c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 --2- 172.21.15.73:0/274414023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c1030f0 0x7fae3c107110 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fae280099c0 tx=0x7fae28009cd0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 -- 172.21.15.73:0/274414023 shutdown_connections 2024-06-06T07:01:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 --2- 172.21.15.73:0/274414023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 0x7fae3c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 --2- 172.21.15.73:0/274414023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c102740 0x7fae3c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 --2- 172.21.15.73:0/274414023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c1030f0 0x7fae3c107110 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 -- 172.21.15.73:0/274414023 >> 172.21.15.73:0/274414023 conn(0x7fae3c0795f0 msgr2=0x7fae3c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 -- 172.21.15.73:0/274414023 shutdown_connections 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 -- 172.21.15.73:0/274414023 wait complete. 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.611+0000 7fae43040700 1 Processor -- start 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 -- start start 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c1030f0 0x7fae3c19ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 0x7fae3c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c112620 con 0x7fae3c102740 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1124a0 con 0x7fae3c1030f0 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae43040700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1127a0 con 0x7fae3c107840 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55938/0 (socket says 172.21.15.73:55938) 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 -- 172.21.15.73:0/547123688 learned_addr learned my addr 172.21.15.73:0/547123688 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae3bfff700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c1030f0 0x7fae3c19ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae415dd700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 0x7fae3c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 -- 172.21.15.73:0/547123688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 msgr2=0x7fae3c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 0x7fae3c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 -- 172.21.15.73:0/547123688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c1030f0 msgr2=0x7fae3c19ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c1030f0 0x7fae3c19ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 -- 172.21.15.73:0/547123688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3c199040 con 0x7fae3c102740 2024-06-06T07:01:36.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae40ddc700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae3000d950 tx=0x7fae3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.612+0000 7fae39ffb700 1 -- 172.21.15.73:0/547123688 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30016070 con 0x7fae3c102740 2024-06-06T07:01:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.613+0000 7fae39ffb700 1 -- 172.21.15.73:0/547123688 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae3000fd40 con 0x7fae3c102740 2024-06-06T07:01:36.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.613+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3c199330 con 0x7fae3c102740 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.613+0000 7fae39ffb700 1 -- 172.21.15.73:0/547123688 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3000e3b0 con 0x7fae3c102740 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.613+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3c1a55f0 con 0x7fae3c102740 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.615+0000 7fae39ffb700 1 -- 172.21.15.73:0/547123688 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fae30019070 con 0x7fae3c102740 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.615+0000 7fae39ffb700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 0x7fae2c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.616+0000 7fae39ffb700 1 -- 172.21.15.73:0/547123688 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fae300cf420 con 0x7fae3c102740 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.616+0000 7fae3bfff700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 0x7fae2c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.616+0000 7fae3bfff700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 0x7fae2c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae3c1020b0 tx=0x7fae28005fb0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 Processor -- start 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 -- start start 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 0x562de101c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10206f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562de0f69810 con 0x562de1020c30 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562de0f11720 con 0x562de101bfd0 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562de0f657d0 con 0x562de101c8f0 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55944/0 (socket says 172.21.15.73:55944) 2024-06-06T07:01:36.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 -- 172.21.15.73:0/2236590412 learned_addr learned my addr 172.21.15.73:0/2236590412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 -- 172.21.15.73:0/2236590412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 msgr2=0x562de10206f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10206f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 -- 172.21.15.73:0/2236590412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 msgr2=0x562de101c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae40ddc700 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 0x562de101c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 0x562de101c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae415dd700 1 -- 172.21.15.73:0/2236590412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de0f07f50 con 0x562de1020c30 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.617+0000 7fae40ddc700 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 0x562de101c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae415dd700 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae340098f0 tx=0x7fae34009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58daf1c539035555 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae26ffd700 1 -- 172.21.15.73:0/2236590412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34004bb0 con 0x562de1020c30 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae26ffd700 1 -- 172.21.15.73:0/2236590412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae34004d50 con 0x562de1020c30 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae26ffd700 1 -- 172.21.15.73:0/2236590412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae34007e30 con 0x562de1020c30 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/2236590412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 msgr2=0x562de1021090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fae340098f0 tx=0x7fae34009c00 comp rx=0 tx=0).stop 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/2236590412 shutdown_connections 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de1020c30 0x562de1021090 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101bfd0 0x562de101c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/2236590412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10206f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.618+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/2236590412 >> 172.21.15.73:0/2236590412 conn(0x562de0f42cc0 msgr2=0x562de1012850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/2236590412 shutdown_connections 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/2236590412 wait complete. 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- start start 2024-06-06T07:01:36.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 Processor -- start 2024-06-06T07:01:36.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de1025580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de1025ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562de1028dd0 con 0x562de101bfd0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562de0f09bf0 con 0x562de1020c30 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.619+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562de1028f50 con 0x562de101c8f0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae4d8c3b80 10 client.0 _ll_register_callbacks cb 0x562de0f03920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55956/0 (socket says 172.21.15.73:55956) 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 -- 172.21.15.73:0/3459093963 learned_addr learned my addr 172.21.15.73:0/3459093963 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae415dd700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de1025ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae3bfff700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de1025580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 -- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 msgr2=0x562de1025580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de1025580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 -- 172.21.15.73:0/3459093963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 msgr2=0x562de1025ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de1025ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 -- 172.21.15.73:0/3459093963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de0eb88d0 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae415dd700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de1025ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae40ddc700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fae30097880 tx=0x7fae3009ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afee77e6624c5dc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30020040 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fae30010710 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae34004d50 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae3009b420 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae4d8c3b80 10 client.15477 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.620+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562de1033750 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae34004d50 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae300a3c90 con 0x562de101bfd0 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 10 client.15477 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:36.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 10 client.15477 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 20 client.15477 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de1038180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 4 client.15477 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de0f76d10 con 0x562de1035d90 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.621+0000 7fae3bfff700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de1038180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.622+0000 7fae3bfff700 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de1038180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3615a229b43d18eb server_cookie=d4e7c9e827557ff0 in_seq=0 out_seq=0 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.622+0000 7fae1bfff700 10 client.15477 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.622+0000 7fae1bfff700 1 -- 172.21.15.73:0/3459093963 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562de0f76d10 con 0x562de1035d90 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.622+0000 7fae1bfff700 10 client.15477 handle_command_reply: tid=0 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.622+0000 7fae4d8c3b80 1 client.15477 shutdown 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 msgr2=0x562de1028280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fae30097880 tx=0x7fae3009ad80 comp rx=0 tx=0).stop 2024-06-06T07:01:36.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 shutdown_connections 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de1038180 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de1025ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae1bfff700 0 client.15477 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de1028280 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/3459093963 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de1025580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 >> 172.21.15.73:0/3459093963 conn(0x562de0f42cc0 msgr2=0x562de10169d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 shutdown_connections 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/3459093963 wait complete. 2024-06-06T07:01:36.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.623+0000 7fae4d8c3b80 20 client.15477 trim_cache size 0 max 16384 2024-06-06T07:01:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.643+0000 7fae4d8c3b80 1 Processor -- start 2024-06-06T07:01:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.643+0000 7fae4d8c3b80 1 -- start start 2024-06-06T07:01:36.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de10245a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1ac370 con 0x562de101bfd0 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1adeb0 con 0x562de1020c30 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1aca40 con 0x562de101c8f0 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55970/0 (socket says 172.21.15.73:55970) 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 -- 172.21.15.73:0/223017910 learned_addr learned my addr 172.21.15.73:0/223017910 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae3bfff700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae415dd700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de10245a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 -- 172.21.15.73:0/223017910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 msgr2=0x562de10230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 -- 172.21.15.73:0/223017910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 msgr2=0x562de10245a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de10245a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 -- 172.21.15.73:0/223017910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de1023b10 con 0x562de101bfd0 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae415dd700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de10245a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae40ddc700 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fae3009b0d0 tx=0x7fae30005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06ff64b2b694b08 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae267fc700 1 -- 172.21.15.73:0/223017910 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30020070 con 0x562de101bfd0 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae267fc700 1 -- 172.21.15.73:0/223017910 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae300a4090 con 0x562de101bfd0 2024-06-06T07:01:36.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.644+0000 7fae267fc700 1 -- 172.21.15.73:0/223017910 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae3009b680 con 0x562de101bfd0 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/223017910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 msgr2=0x562de10129f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fae3009b0d0 tx=0x7fae30005910 comp rx=0 tx=0).stop 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/223017910 shutdown_connections 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de1020c30 0x562de10245a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de10129f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/223017910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de101c8f0 0x562de10230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/223017910 >> 172.21.15.73:0/223017910 conn(0x562de0f42cc0 msgr2=0x562de1023e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/223017910 shutdown_connections 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/223017910 wait complete. 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.645+0000 7fae4d8c3b80 1 -- start start 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 Processor -- start 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 0x562de102e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de1020c30 0x562de102e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1ad7e0 con 0x562de101bfd0 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1ad110 con 0x562de101c8f0 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fae3c1aa310 con 0x562de1020c30 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae4d8c3b80 10 client.0 _ll_register_callbacks cb 0x562de0f03920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae40ddc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55976/0 (socket says 172.21.15.73:55976) 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae40ddc700 1 -- 172.21.15.73:0/348619100 learned_addr learned my addr 172.21.15.73:0/348619100 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae3bfff700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 0x562de102e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.646+0000 7fae415dd700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de1020c30 0x562de102e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 -- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de1020c30 msgr2=0x562de102e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de1020c30 0x562de102e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 -- 172.21.15.73:0/348619100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 msgr2=0x562de102e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 0x562de102e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 -- 172.21.15.73:0/348619100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de1018d40 con 0x562de101bfd0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae3bfff700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 0x562de102e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae40ddc700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae300996d0 tx=0x7fae300a3fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877937f8b0b0add6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae1bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30020050 con 0x562de101bfd0 2024-06-06T07:01:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fae3009b680 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae300a4090 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fae3009b8c0 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae4d8c3b80 10 client.15483 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562de1018d40 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.647+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae300a4090 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fae3007d070 con 0x562de101bfd0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 20 client.15483 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de10a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 4 client.15483 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562de0f71290 con 0x562de1035d90 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae3bfff700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de10a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:36.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae1bfff700 10 client.15483 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.648+0000 7fae3bfff700 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de10a7740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fda03d4451397e server_cookie=cd6fc03491abcfe4 in_seq=0 out_seq=0 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae1bfff700 1 -- 172.21.15.73:0/348619100 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562de0f71290 con 0x562de1035d90 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae1bfff700 10 client.15483 handle_command_reply: tid=0 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 client.15483 shutdown 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 msgr2=0x562de101b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fae300996d0 tx=0x7fae300a3fc0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 shutdown_connections 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562de1035d90 0x562de10a7740 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562de1020c30 0x562de102e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562de101bfd0 0x562de101b8a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 --2- 172.21.15.73:0/348619100 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562de101c8f0 0x562de102e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 >> 172.21.15.73:0/348619100 conn(0x562de0f42cc0 msgr2=0x562de1021700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.655 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae1bfff700 0 client.15483 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 shutdown_connections 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 1 -- 172.21.15.73:0/348619100 wait complete. 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.649+0000 7fae4d8c3b80 20 client.15483 trim_cache size 0 max 16384 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "scrubs": { 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "a1ba64f1-6d4c-448c-90ff-95721c06e9d7": { 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/", 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "tag": "a1ba64f1-6d4c-448c-90ff-95721c06e9d7", 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: "options": "recursive,force" 2024-06-06T07:01:36.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:01:36.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:01:36.657 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:01:36.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 msgr2=0x7fae2c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 0x7fae2c0a52e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fae3c1020b0 tx=0x7fae28005fb0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 msgr2=0x7fae3c19e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fae3000d950 tx=0x7fae3000dc60 comp rx=0 tx=0).stop 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 shutdown_connections 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fae2c0a2e20 0x7fae2c0a52e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fae3c107840 0x7fae3c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fae3c102740 0x7fae3c19e720 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 --2- 172.21.15.73:0/547123688 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fae3c1030f0 0x7fae3c19ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 >> 172.21.15.73:0/547123688 conn(0x7fae3c0795f0 msgr2=0x7fae3c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 shutdown_connections 2024-06-06T07:01:36.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:01:36.650+0000 7fae43040700 1 -- 172.21.15.73:0/547123688 wait complete. 2024-06-06T07:01:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:37 smithi170 ceph-mon[36108]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-06T07:01:37.469 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:37 smithi073 ceph-mon[32303]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-06T07:01:37.496 INFO:tasks.cephfs.filesystem:scrub status for tag:a1ba64f1-6d4c-448c-90ff-95721c06e9d7 - {'path': '/', 'tag': 'a1ba64f1-6d4c-448c-90ff-95721c06e9d7', 'options': 'recursive,force'} 2024-06-06T07:01:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:37 smithi136 ceph-mon[35901]: pgmap v10: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 0 B/s wr, 18 op/s 2024-06-06T07:01:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:39 smithi170 ceph-mon[36108]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 49 op/s 2024-06-06T07:01:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:39 smithi073 ceph-mon[32303]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 49 op/s 2024-06-06T07:01:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:39 smithi136 ceph-mon[35901]: pgmap v11: 129 pgs: 129 active+clean; 1.1 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 38 KiB/s wr, 49 op/s 2024-06-06T07:01:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:41 smithi170 ceph-mon[36108]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:01:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:41 smithi073 ceph-mon[32303]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:01:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:41 smithi136 ceph-mon[35901]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 77 KiB/s wr, 52 op/s 2024-06-06T07:01:43.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:43 smithi170 ceph-mon[36108]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:43 smithi073 ceph-mon[32303]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:43 smithi136 ceph-mon[35901]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:45.326 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:45 smithi073 ceph-mon[32303]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-06-06T07:01:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:45 smithi170 ceph-mon[36108]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-06-06T07:01:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:45 smithi136 ceph-mon[35901]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 77 KiB/s wr, 44 op/s 2024-06-06T07:01:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:47 smithi170 ceph-mon[36108]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 -- 172.21.15.73:0/1623609052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85400ffea0 msgr2=0x7f8540100320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 --2- 172.21.15.73:0/1623609052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85400ffea0 0x7f8540100320 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8534009930 tx=0x7f853402a420 comp rx=0 tx=0).stop 2024-06-06T07:01:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 -- 172.21.15.73:0/1623609052 shutdown_connections 2024-06-06T07:01:47.401 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 --2- 172.21.15.73:0/1623609052 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540100860 0x7f854010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 --2- 172.21.15.73:0/1623609052 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85400ffea0 0x7f8540100320 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 --2- 172.21.15.73:0/1623609052 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8540106980 0x7f8540106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 -- 172.21.15.73:0/1623609052 >> 172.21.15.73:0/1623609052 conn(0x7f8540076e20 msgr2=0x7f85400fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 -- 172.21.15.73:0/1623609052 shutdown_connections 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.397+0000 7f85469dc640 1 -- 172.21.15.73:0/1623609052 wait complete. 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 Processor -- start 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 -- start start 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85400ffea0 0x7f8540198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 0x7f854019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.402 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8540111d60 con 0x7f8540100860 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8540111be0 con 0x7f8540106980 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f85469dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8540111ee0 con 0x7f85400ffea0 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47324/0 (socket says 172.21.15.73:47324) 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f853f7fe640 1 -- 172.21.15.73:0/1119166740 learned_addr learned my addr 172.21.15.73:0/1119166740 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.398+0000 7f853ffff640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85400ffea0 0x7f8540198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 -- 172.21.15.73:0/1119166740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85400ffea0 msgr2=0x7f8540198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85400ffea0 0x7f8540198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 -- 172.21.15.73:0/1119166740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 msgr2=0x7f854019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f8544f79640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 0x7f854019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 0x7f854019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 -- 172.21.15.73:0/1119166740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019d220 con 0x7f8540100860 2024-06-06T07:01:47.403 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f8544f79640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 0x7f854019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8534004770 tx=0x7f85340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853d7fa640 1 -- 172.21.15.73:0/1119166740 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853403f070 con 0x7f8540100860 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853d7fa640 1 -- 172.21.15.73:0/1119166740 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f853403b460 con 0x7f8540100860 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019d4b0 con 0x7f8540100860 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f853d7fa640 1 -- 172.21.15.73:0/1119166740 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534035070 con 0x7f8540100860 2024-06-06T07:01:47.404 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.399+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a4da0 con 0x7f8540100860 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.401+0000 7f853d7fa640 1 -- 172.21.15.73:0/1119166740 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7f853403b610 con 0x7f8540100860 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 Processor -- start 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 -- start start 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853d7fa640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 0x7f85280a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324100740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5563240574f0 con 0x5563240f3a90 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556323ff5870 con 0x5563240fe350 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f85497ca740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5563240534b0 con 0x5563240f3170 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f8544f79640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 0x7f85280a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47328/0 (socket says 172.21.15.73:47328) 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 -- 172.21.15.73:0/3760782160 learned_addr learned my addr 172.21.15.73:0/3760782160 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853d7fa640 1 -- 172.21.15.73:0/1119166740 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f85340efd30 con 0x7f8540100860 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853ffff640 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 -- 172.21.15.73:0/3760782160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 msgr2=0x5563240f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 -- 172.21.15.73:0/3760782160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 msgr2=0x556324100740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324100740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 -- 172.21.15.73:0/3760782160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563240119b0 con 0x5563240f3a90 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853ffff640 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f8544f79640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 0x7f85280a5300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f8538007900 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.402+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85340ed6d0 tx=0x7f8534046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54d0479298d60571 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85267fc640 1 -- 172.21.15.73:0/3760782160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534048070 con 0x5563240f3a90 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85267fc640 1 -- 172.21.15.73:0/3760782160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85340bc430 con 0x5563240f3a90 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85267fc640 1 -- 172.21.15.73:0/3760782160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85340bbd30 con 0x5563240f3a90 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 -- 172.21.15.73:0/3760782160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 msgr2=0x5563240fde10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f85340ed6d0 tx=0x7f8534046000 comp rx=0 tx=0).stop 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 -- 172.21.15.73:0/3760782160 shutdown_connections 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.412 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x5563240fde10 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 --2- 172.21.15.73:0/3760782160 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 -- 172.21.15.73:0/3760782160 >> 172.21.15.73:0/3760782160 conn(0x5563240ef680 msgr2=0x5563240efa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 -- 172.21.15.73:0/3760782160 shutdown_connections 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.403+0000 7f85497ca740 1 -- 172.21.15.73:0/3760782160 wait complete. 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 -- start start 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 Processor -- start 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 0x5563240fb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x5563240fb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556324100d40 con 0x5563240fe350 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556323fe0ae0 con 0x5563240f3170 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556324100ec0 con 0x5563240f3a90 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f85497ca740 10 client.0 _ll_register_callbacks cb 0x5563240199b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f8544f79640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f8544f79640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47338/0 (socket says 172.21.15.73:47338) 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.404+0000 7f8544f79640 1 -- 172.21.15.73:0/1432995279 learned_addr learned my addr 172.21.15.73:0/1432995279 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f853ffff640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 0x5563240fb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x5563240fb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 -- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 msgr2=0x5563240fb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x5563240fb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 -- 172.21.15.73:0/1432995279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 msgr2=0x5563240fb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 0x5563240fb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 -- 172.21.15.73:0/1432995279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556323d6b210 con 0x5563240fe350 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f853ffff640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 0x5563240fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x5563240fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f8544f79640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8538011a90 tx=0x7f8538004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10112e616fddad3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f851f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538019050 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f853801e440 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85340bc430 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f853802c050 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f85497ca740 10 client.15495 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556323e37620 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.405+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85340bc430 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f85380275e0 con 0x5563240fe350 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 10 client.15495 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 10 client.15495 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 20 client.15495 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55632410e1a0 0x556324110590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 4 client.15495 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55632406af10 con 0x55632410e1a0 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f853ffff640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55632410e1a0 0x556324110590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f851f7fe640 10 client.15495 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.406+0000 7f853ffff640 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55632410e1a0 0x556324110590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8abef037501feec2 server_cookie=8d35ae14c99b2bd0 in_seq=0 out_seq=0 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.407+0000 7f851f7fe640 1 -- 172.21.15.73:0/1432995279 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55632406af10 con 0x55632410e1a0 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.407+0000 7f851f7fe640 10 client.15495 handle_command_reply: tid=0 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.407+0000 7f85497ca740 1 client.15495 shutdown 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 msgr2=0x5563240fbd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8538011a90 tx=0x7f8538004b50 comp rx=0 tx=0).stop 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 shutdown_connections 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55632410e1a0 0x556324110590 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240fe350 0x5563240fbd70 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x5563240fb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 --2- 172.21.15.73:0/1432995279 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240f3170 0x5563240fb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 >> 172.21.15.73:0/1432995279 conn(0x5563240ef680 msgr2=0x5563240f5400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f851f7fe640 0 client.15495 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 shutdown_connections 2024-06-06T07:01:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 1 -- 172.21.15.73:0/1432995279 wait complete. 2024-06-06T07:01:47.417 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.408+0000 7f85497ca740 20 client.15495 trim_cache size 0 max 16384 2024-06-06T07:01:47.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 Processor -- start 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 -- start start 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x5563241163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85401aedb0 con 0x5563240f3a90 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85401b08f0 con 0x5563240fe350 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f85497ca740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85401af480 con 0x5563240f3170 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47348/0 (socket says 172.21.15.73:47348) 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 -- 172.21.15.73:0/3910368536 learned_addr learned my addr 172.21.15.73:0/3910368536 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853ffff640 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 -- 172.21.15.73:0/3910368536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 msgr2=0x5563240f5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 -- 172.21.15.73:0/3910368536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 msgr2=0x5563241163e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.423+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x5563241163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f853f7fe640 1 -- 172.21.15.73:0/3910368536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563240f6070 con 0x5563240f3a90 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f853ffff640 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f853f7fe640 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f85340ed0f0 tx=0x7f85340bb790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db97b6681c37f45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f8525ffb640 1 -- 172.21.15.73:0/3910368536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8534048070 con 0x5563240f3a90 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f8525ffb640 1 -- 172.21.15.73:0/3910368536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85340c00e0 con 0x5563240f3a90 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f8525ffb640 1 -- 172.21.15.73:0/3910368536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853403b940 con 0x5563240f3a90 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 -- 172.21.15.73:0/3910368536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 msgr2=0x556324115ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f85340ed0f0 tx=0x7f85340bb790 comp rx=0 tx=0).stop 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 -- 172.21.15.73:0/3910368536 shutdown_connections 2024-06-06T07:01:47.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x5563241163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3a90 0x556324115ea0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 --2- 172.21.15.73:0/3910368536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3170 0x5563240f5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 -- 172.21.15.73:0/3910368536 >> 172.21.15.73:0/3910368536 conn(0x5563240ef680 msgr2=0x5563240f9bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.424+0000 7f85497ca740 1 -- 172.21.15.73:0/3910368536 shutdown_connections 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 -- 172.21.15.73:0/3910368536 wait complete. 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 -- start start 2024-06-06T07:01:47.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 Processor -- start 2024-06-06T07:01:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x556324101cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324170ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85401b0220 con 0x5563240f3170 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85401afb50 con 0x5563240fe350 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85401acd90 con 0x5563240f3a90 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f85497ca740 10 client.0 _ll_register_callbacks cb 0x7f8538007c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f853ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f853ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47360/0 (socket says 172.21.15.73:47360) 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.425+0000 7f853ffff640 1 -- 172.21.15.73:0/1266772420 learned_addr learned my addr 172.21.15.73:0/1266772420 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x556324101cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 -- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 msgr2=0x556324101cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x556324101cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 -- 172.21.15.73:0/1266772420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 msgr2=0x556324170ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:01:47.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324170ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 -- 172.21.15.73:0/1266772420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563241714e0 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x556324101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f853ffff640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f853000baa0 tx=0x7f853000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237db1591416766e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530023670 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f8530023810 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85340c00e0 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f853002c050 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 10 client.15501 fetch_fsmap learned FSMap version 30 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556324178900 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85340c00e0 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8530011a60 con 0x5563240f3170 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 10 client.15501 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:01:47.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 10 client.15501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 20 client.15501 populate_metadata read hostname 'smithi073' 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556324179090 0x55632417b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 4 client.15501 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.426+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556324064c20 con 0x556324179090 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.427+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556324179090 0x55632417b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.427+0000 7f853f7fe640 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556324179090 0x55632417b4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be399228f2720b2a server_cookie=a8f8da12af893b5c in_seq=0 out_seq=0 2024-06-06T07:01:47.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.427+0000 7f851f7fe640 10 client.15501 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:47.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f851f7fe640 1 -- 172.21.15.73:0/1266772420 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556324064c20 con 0x556324179090 2024-06-06T07:01:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f851f7fe640 10 client.15501 handle_command_reply: tid=0 2024-06-06T07:01:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 client.15501 shutdown 2024-06-06T07:01:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 msgr2=0x556324101790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f853000baa0 tx=0x7f853000bf60 comp rx=0 tx=0).stop 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 shutdown_connections 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556324179090 0x55632417b4b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5563240fe350 0x556324170ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5563240f3a90 0x556324101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f851f7fe640 0 client.15501 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:01:47.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 --2- 172.21.15.73:0/1266772420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5563240f3170 0x556324101790 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.439+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 >> 172.21.15.73:0/1266772420 conn(0x5563240ef680 msgr2=0x5563240fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 shutdown_connections 2024-06-06T07:01:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85497ca740 1 -- 172.21.15.73:0/1266772420 wait complete. 2024-06-06T07:01:47.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85497ca740 20 client.15501 trim_cache size 0 max 16384 2024-06-06T07:01:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:01:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:01:47.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:01:47.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 msgr2=0x7f85280a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 0x7f85280a5300 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8538007fa0 tx=0x7f8538007900 comp rx=0 tx=0).stop 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 msgr2=0x7f85401987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f8534004770 tx=0x7f85340042f0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 shutdown_connections 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8540106980 0x7f854019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8540100860 0x7f85401987c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85280a2e40 0x7f85280a5300 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 --2- 172.21.15.73:0/1119166740 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85400ffea0 0x7f8540198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:01:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 >> 172.21.15.73:0/1119166740 conn(0x7f8540076e20 msgr2=0x7f85401049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:01:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 shutdown_connections 2024-06-06T07:01:47.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:01:47.440+0000 7f85469dc640 1 -- 172.21.15.73:0/1119166740 wait complete. 2024-06-06T07:01:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:47 smithi073 ceph-mon[32303]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:47 smithi136 ceph-mon[35901]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 40 op/s 2024-06-06T07:01:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:49 smithi170 ceph-mon[36108]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 46 op/s 2024-06-06T07:01:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:49 smithi170 ceph-mon[36108]: scrub summary: idle 2024-06-06T07:01:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:49 smithi073 ceph-mon[32303]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 46 op/s 2024-06-06T07:01:49.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:49 smithi073 ceph-mon[32303]: scrub summary: idle 2024-06-06T07:01:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:49 smithi136 ceph-mon[35901]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 78 KiB/s wr, 46 op/s 2024-06-06T07:01:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:49 smithi136 ceph-mon[35901]: scrub summary: idle 2024-06-06T07:01:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:51 smithi170 ceph-mon[36108]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-06-06T07:01:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:51 smithi073 ceph-mon[32303]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-06-06T07:01:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:51 smithi136 ceph-mon[35901]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-06-06T07:01:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:53 smithi170 ceph-mon[36108]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-06T07:01:53.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:53 smithi073 ceph-mon[32303]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-06T07:01:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:53 smithi136 ceph-mon[35901]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-06T07:01:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:55 smithi170 ceph-mon[36108]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-06-06T07:01:55.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:55 smithi073 ceph-mon[32303]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-06-06T07:01:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:55 smithi136 ceph-mon[35901]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.1 KiB/s wr, 16 op/s 2024-06-06T07:01:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:57 smithi170 ceph-mon[36108]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:01:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:57 smithi073 ceph-mon[32303]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:01:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:57 smithi136 ceph-mon[35901]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-06-06T07:01:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:01:59 smithi170 ceph-mon[36108]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-06-06T07:01:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:01:59 smithi073 ceph-mon[32303]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-06-06T07:01:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:01:59 smithi136 ceph-mon[35901]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.0 KiB/s wr, 15 op/s 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/4188795264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c103550 msgr2=0x7fd09c10ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/4188795264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c103550 0x7fd09c10ba30 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd090009900 tx=0x7fd09002a510 comp rx=0 tx=0).stop 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/4188795264 shutdown_connections 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/4188795264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c10bf70 0x7fd09c10e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/4188795264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c103550 0x7fd09c10ba30 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/4188795264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c102c30 0x7fd09c103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/4188795264 >> 172.21.15.73:0/4188795264 conn(0x7fd09c077a20 msgr2=0x7fd09c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/4188795264 shutdown_connections 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.550+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/4188795264 wait complete. 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 Processor -- start 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 -- start start 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c102c30 0x7fd09c100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 0x7fd09c101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111c20 con 0x7fd09c10bf70 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111aa0 con 0x7fd09c102c30 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd0a0ed4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111da0 con 0x7fd09c103550 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd09ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 0x7fd09c101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.551+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43968/0 (socket says 172.21.15.73:43968) 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 -- 172.21.15.73:0/3762586180 learned_addr learned my addr 172.21.15.73:0/3762586180 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd09ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 0x7fd09c101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47886/0 (socket says 172.21.15.73:47886) 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd09a59c640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c102c30 0x7fd09c100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 -- 172.21.15.73:0/3762586180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c102c30 msgr2=0x7fd09c100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c102c30 0x7fd09c100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 -- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 msgr2=0x7fd09c101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 0x7fd09c101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 -- 172.21.15.73:0/3762586180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c1a6600 con 0x7fd09c103550 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd099d9b640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd090005940 tx=0x7fd0900042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09003f030 con 0x7fd09c103550 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c1a6890 con 0x7fd09c103550 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd090003cb0 con 0x7fd09c103550 2024-06-06T07:02:00.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090035070 con 0x7fd09c103550 2024-06-06T07:02:00.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.552+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c1a6dd0 con 0x7fd09c103550 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 Processor -- start 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 23) ==== 138009+0+0 (secure 0 0 0) 0x7fd090042030 con 0x7fd09c103550 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 -- start start 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7aa90 0x55563aa84e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa85350 0x55563aa87740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55563a9de4f0 con 0x55563aa85350 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55563a97c870 con 0x55563aa7aa90 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55563a9da4b0 con 0x55563aa7a170 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd09ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa85350 0x55563aa87740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.556+0000 7fd09ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43978/0 (socket says 172.21.15.73:43978) 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3532713620 learned_addr learned my addr 172.21.15.73:0/3532713620 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09a59c640 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7aa90 0x55563aa84e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd08b7fe640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 0x7fd0840a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3532713620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7aa90 msgr2=0x55563aa84e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7aa90 0x55563aa84e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3532713620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa85350 msgr2=0x55563aa87740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa85350 0x55563aa87740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3532713620 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563a9989b0 con 0x55563aa7a170 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fd0900f0110 con 0x7fd09c103550 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 0x7fd0840a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd094009f70 tx=0x7fd094010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78f72b7972b381f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd088ff9640 1 -- 172.21.15.73:0/3532713620 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094022070 con 0x55563aa7a170 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd088ff9640 1 -- 172.21.15.73:0/3532713620 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0940113e0 con 0x55563aa7a170 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.557+0000 7fd088ff9640 1 -- 172.21.15.73:0/3532713620 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094016070 con 0x55563aa7a170 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3532713620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 msgr2=0x55563aa7a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd094009f70 tx=0x7fd094010800 comp rx=0 tx=0).stop 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 0x7fd0840a5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd09401f000 tx=0x7fd094022000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3532713620 shutdown_connections 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa85350 0x55563aa87740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7aa90 0x55563aa84e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3532713620 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa7a550 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.558+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3532713620 >> 172.21.15.73:0/3532713620 conn(0x55563aa76680 msgr2=0x55563aa76a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3532713620 shutdown_connections 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3532713620 wait complete. 2024-06-06T07:02:00.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- start start 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 Processor -- start 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 0x55563aa82310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 0x55563aa82d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55563aa87f80 con 0x55563aa7aa90 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55563a967ae0 con 0x55563aa7a170 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55563aa88100 con 0x55563aa85350 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.559+0000 7fd0a3be7740 10 client.0 _ll_register_callbacks cb 0x55563a9a09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 0x55563aa82310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 0x55563aa82d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47898/0 (socket says 172.21.15.73:47898) 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd099d9b640 1 -- 172.21.15.73:0/3857022829 learned_addr learned my addr 172.21.15.73:0/3857022829 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:00.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3857022829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 msgr2=0x55563aa82310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 0x55563aa82310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3857022829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 msgr2=0x55563aa82850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 -- 172.21.15.73:0/3857022829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563a6f2210 con 0x55563aa85350 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd099d9b640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09a59c640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 0x55563aa82310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 0x55563aa82d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd094005f10 tx=0x7fd094005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab861cb635782bbd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.560+0000 7fd081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094034060 con 0x55563aa85350 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fd094011790 con 0x55563aa85350 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0940113e0 con 0x55563aa85350 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd094016050 con 0x55563aa85350 2024-06-06T07:02:00.569 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 10 client.25117 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55563a7be620 con 0x55563aa85350 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0940113e0 con 0x55563aa85350 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd09402e030 con 0x55563aa85350 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 10 client.25117 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 10 client.25117 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 20 client.25117 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aa95440 0x55563aa97830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 4 client.25117 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.561+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55563a9f1f10 con 0x55563aa95440 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.562+0000 7fd09a59c640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aa95440 0x55563aa97830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.562+0000 7fd09a59c640 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aa95440 0x55563aa97830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f05d15c51e73a server_cookie=5d9c86b3a82e72a7 in_seq=0 out_seq=0 2024-06-06T07:02:00.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.562+0000 7fd081ffb640 10 client.25117 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd081ffb640 1 -- 172.21.15.73:0/3857022829 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55563a9f1f10 con 0x55563aa95440 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd081ffb640 10 client.25117 handle_command_reply: tid=0 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 client.25117 shutdown 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 msgr2=0x55563aa82d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 0x55563aa82d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd094005f10 tx=0x7fd094005c50 comp rx=0 tx=0).stop 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 shutdown_connections 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aa95440 0x55563aa97830 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa85350 0x55563aa82d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa82850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd081ffb640 0 client.25117 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/3857022829 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa7a170 0x55563aa82310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 >> 172.21.15.73:0/3857022829 conn(0x55563aa76680 msgr2=0x55563aa7c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:00.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 shutdown_connections 2024-06-06T07:02:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 1 -- 172.21.15.73:0/3857022829 wait complete. 2024-06-06T07:02:00.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.563+0000 7fd0a3be7740 20 client.25117 trim_cache size 0 max 16384 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 Processor -- start 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 -- start start 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7aa90 0x55563aa76a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa9cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b0fa0 con 0x55563aa7a170 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b2ae0 con 0x55563aa85350 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b1670 con 0x55563aa7aa90 2024-06-06T07:02:00.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47908/0 (socket says 172.21.15.73:47908) 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09a59c640 1 -- 172.21.15.73:0/2380453420 learned_addr learned my addr 172.21.15.73:0/2380453420 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd099d9b640 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7aa90 0x55563aa76a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa9cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09a59c640 1 -- 172.21.15.73:0/2380453420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7aa90 msgr2=0x55563aa76a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.581+0000 7fd09a59c640 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7aa90 0x55563aa76a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd09a59c640 1 -- 172.21.15.73:0/2380453420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 msgr2=0x55563aa9cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd09a59c640 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa9cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd09a59c640 1 -- 172.21.15.73:0/2380453420 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55563aa911e0 con 0x55563aa7a170 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd09a59c640 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55563aa97f10 tx=0x7fd08c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447d73937e692ffd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd083fff640 1 -- 172.21.15.73:0/2380453420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c004030 con 0x55563aa7a170 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd083fff640 1 -- 172.21.15.73:0/2380453420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c00fd20 con 0x55563aa7a170 2024-06-06T07:02:00.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.582+0000 7fd083fff640 1 -- 172.21.15.73:0/2380453420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd08c003ab0 con 0x55563aa7a170 2024-06-06T07:02:00.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- 172.21.15.73:0/2380453420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 msgr2=0x55563aa71840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55563aa97f10 tx=0x7fd08c005f90 comp rx=0 tx=0).stop 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- 172.21.15.73:0/2380453420 shutdown_connections 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa9cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7aa90 0x55563aa76a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/2380453420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7a170 0x55563aa71840 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- 172.21.15.73:0/2380453420 >> 172.21.15.73:0/2380453420 conn(0x55563aa76680 msgr2=0x55563aa81ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- 172.21.15.73:0/2380453420 shutdown_connections 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- 172.21.15.73:0/2380453420 wait complete. 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- start start 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 Processor -- start 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa883f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa88eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b2410 con 0x55563aa7aa90 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b1d40 con 0x55563aa85350 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1aef80 con 0x55563aa7a170 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd0a3be7740 10 client.0 _ll_register_callbacks cb 0x7fd094020070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.583+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47912/0 (socket says 172.21.15.73:47912) 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 -- 172.21.15.73:0/636118369 learned_addr learned my addr 172.21.15.73:0/636118369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd09a59c640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa883f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd09ad9d640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa88eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 -- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 msgr2=0x55563aa883f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa883f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 -- 172.21.15.73:0/636118369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 msgr2=0x55563aa88eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa88eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 -- 172.21.15.73:0/636118369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55563aafe220 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd099d9b640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ed4d0 tx=0x7fd0900f3830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2b3fb6381adfde1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd081ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090049040 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fd090003800 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd08c00fd20 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd0900bbd00 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd0a3be7740 10 client.15525 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55563aaff530 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.584+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd08c00fd20 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fd0900c0140 con 0x55563aa7aa90 2024-06-06T07:02:00.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 10 client.15525 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 20 client.15525 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aaffcd0 0x55563ab020c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 4 client.15525 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55563a9ebc20 con 0x55563aaffcd0 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd09a59c640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aaffcd0 0x55563ab020c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd081ffb640 10 client.15525 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:00.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:00.585+0000 7fd09a59c640 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aaffcd0 0x55563ab020c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4efb7ff7876bbbc server_cookie=9706317bd31f284d in_seq=0 out_seq=0 2024-06-06T07:02:00.774 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:02:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:02:00.501+0000 7f984bb7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:02:00.775 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:02:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:02:00.501+0000 7f984bb7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:02:00.775 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:02:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:02:00.501+0000 7f984bb7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:02:00.775 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:02:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:02:00.501+0000 7f984bb7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:02:00.775 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:02:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:02:00.501+0000 7f984bb7d700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-06T07:02:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:01 smithi073 ceph-mon[32303]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:02:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:01 smithi136 ceph-mon[35901]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:02:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:01 smithi170 ceph-mon[36108]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-06T07:02:02.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:02.113+0000 7fd08b7fe640 1 -- 172.21.15.73:0/3762586180 <== mon.1 v2:172.21.15.170:3300/0 6 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fd090004750 con 0x7fd09c103550 2024-06-06T07:02:03.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:03 smithi170 ceph-mon[36108]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:03.400 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:03 smithi170 ceph-mon[36108]: mgrmap e24: x(active, since 40s), standbys: y, z 2024-06-06T07:02:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:03 smithi073 ceph-mon[32303]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:03 smithi073 ceph-mon[32303]: mgrmap e24: x(active, since 40s), standbys: y, z 2024-06-06T07:02:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:03 smithi136 ceph-mon[35901]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:03 smithi136 ceph-mon[35901]: mgrmap e24: x(active, since 40s), standbys: y, z 2024-06-06T07:02:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd081ffb640 1 -- 172.21.15.73:0/636118369 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55563a9ebc20 con 0x55563aaffcd0 2024-06-06T07:02:03.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd081ffb640 10 client.15525 handle_command_reply: tid=0 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 client.15525 shutdown 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 msgr2=0x55563aa88930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fd0900ed4d0 tx=0x7fd0900f3830 comp rx=0 tx=0).stop 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 shutdown_connections 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55563aaffcd0 0x55563ab020c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55563aa85350 0x55563aa88eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55563aa7aa90 0x55563aa88930 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 --2- 172.21.15.73:0/636118369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55563aa7a170 0x55563aa883f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd081ffb640 0 client.15525 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 >> 172.21.15.73:0/636118369 conn(0x55563aa76680 msgr2=0x55563aa76a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 shutdown_connections 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 1 -- 172.21.15.73:0/636118369 wait complete. 2024-06-06T07:02:03.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.743+0000 7fd0a3be7740 20 client.15525 trim_cache size 0 max 16384 2024-06-06T07:02:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:02:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:02:03.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 msgr2=0x7fd0840a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 0x7fd0840a5300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd09401f000 tx=0x7fd094022000 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 msgr2=0x7fd09c100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd090005940 tx=0x7fd0900042f0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 shutdown_connections 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd09c10bf70 0x7fd09c101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd0840a2e40 0x7fd0840a5300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd09c103550 0x7fd09c100dc0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 --2- 172.21.15.73:0/3762586180 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd09c102c30 0x7fd09c100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 >> 172.21.15.73:0/3762586180 conn(0x7fd09c077a20 msgr2=0x7fd09c0fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:03.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 shutdown_connections 2024-06-06T07:02:03.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:03.744+0000 7fd0a0ed4640 1 -- 172.21.15.73:0/3762586180 wait complete. 2024-06-06T07:02:05.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: *** Caught signal (Segmentation fault) ** 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: in thread 7f5c31d01700 thread_name:md_log_replay 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5c430c3d20] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2: /usr/bin/ceph-mds(+0x549a12) [0x556c175d3a12] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c175d71c3] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 4: (MDLog::_replay_thread()+0x617) [0x556c17631ff7] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c1763b861] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5c430b91ca] 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 7: clone() 2024-06-06T07:02:05.053 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:04.738+0000 7f5c31d01700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: in thread 7f5c31d01700 thread_name:md_log_replay 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5c430c3d20] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2: /usr/bin/ceph-mds(+0x549a12) [0x556c175d3a12] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c175d71c3] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 4: (MDLog::_replay_thread()+0x617) [0x556c17631ff7] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c1763b861] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5c430b91ca] 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 7: clone() 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.054 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 0> 2024-06-06T07:02:04.738+0000 7f5c31d01700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: in thread 7f5c31d01700 thread_name:md_log_replay 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5c430c3d20] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2: /usr/bin/ceph-mds(+0x549a12) [0x556c175d3a12] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c175d71c3] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 4: (MDLog::_replay_thread()+0x617) [0x556c17631ff7] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c1763b861] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5c430b91ca] 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 7: clone() 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.055 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 0> 2024-06-06T07:02:04.738+0000 7f5c31d01700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: in thread 7f5c31d01700 thread_name:md_log_replay 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f5c430c3d20] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2: /usr/bin/ceph-mds(+0x549a12) [0x556c175d3a12] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x556c175d71c3] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 4: (MDLog::_replay_thread()+0x617) [0x556c17631ff7] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 5: (MDLog::ReplayThread::entry()+0x11) [0x556c1763b861] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f5c430b91ca] 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 7: clone() 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[65348]: 2024-06-06T07:02:05.056 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:04 smithi136 podman[69639]: 2024-06-06 07:02:04.800900259 +0000 UTC m=+0.021392774 container died b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-06T07:02:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:05 smithi073 ceph-mon[32303]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:05 smithi136 ceph-mon[35901]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:05 smithi170 ceph-mon[36108]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:05.876 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:05 smithi136 podman[69639]: 2024-06-06 07:02:05.608605138 +0000 UTC m=+0.829097695 container remove b8cc2f31a968f2d35824d87abde541b8cfd6e3a7e7e05146f5743a4c0fc75375 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-06T07:02:05.876 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:05 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-06-06T07:02:06.301 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:05 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-06-06T07:02:06.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:05 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Consumed 1.039s CPU time. 2024-06-06T07:02:07.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:02:07.523 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:07 smithi073 ceph-mon[32303]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:02:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:07 smithi136 ceph-mon[35901]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:02:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:07 smithi170 ceph-mon[36108]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-06-06T07:02:07.756 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:09.172 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:09 smithi073 ceph-mon[32303]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:09 smithi136 ceph-mon[35901]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:09.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 -- 172.21.15.73:0/1163400149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4100b50 msgr2=0x7effa4100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 --2- 172.21.15.73:0/1163400149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4100b50 0x7effa4100fd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 -- 172.21.15.73:0/1163400149 shutdown_connections 2024-06-06T07:02:09.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 --2- 172.21.15.73:0/1163400149 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4106b90 0x7effa4106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 --2- 172.21.15.73:0/1163400149 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4101510 0x7effa410d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 --2- 172.21.15.73:0/1163400149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4100b50 0x7effa4100fd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 -- 172.21.15.73:0/1163400149 >> 172.21.15.73:0/1163400149 conn(0x7effa4077a00 msgr2=0x7effa40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 -- 172.21.15.73:0/1163400149 shutdown_connections 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.580+0000 7effa945a700 1 -- 172.21.15.73:0/1163400149 wait complete. 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 Processor -- start 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 -- start start 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4100b50 0x7effa40fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4101510 0x7effa40fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effa41115a0 con 0x7effa4106b90 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111420 con 0x7effa4100b50 2024-06-06T07:02:09.583 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa945a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7effa4111720 con 0x7effa4101510 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34468/0 (socket says 172.21.15.73:34468) 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa37fe700 1 -- 172.21.15.73:0/4065750757 learned_addr learned my addr 172.21.15.73:0/4065750757 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa2ffd700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4100b50 0x7effa40fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.581+0000 7effa27fc700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4101510 0x7effa40fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 -- 172.21.15.73:0/4065750757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4101510 msgr2=0x7effa40fd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4101510 0x7effa40fd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 -- 172.21.15.73:0/4065750757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4100b50 msgr2=0x7effa40fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4100b50 0x7effa40fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.584 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 -- 172.21.15.73:0/4065750757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa41a7070 con 0x7effa4106b90 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa37fe700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff9c007ee0 tx=0x7eff9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7eff9bfff700 1 -- 172.21.15.73:0/4065750757 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c004bb0 con 0x7effa4106b90 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa41a7360 con 0x7effa4106b90 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa41a7870 con 0x7effa4106b90 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7eff9bfff700 1 -- 172.21.15.73:0/4065750757 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff9c004d50 con 0x7effa4106b90 2024-06-06T07:02:09.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.582+0000 7eff9bfff700 1 -- 172.21.15.73:0/4065750757 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c0058e0 con 0x7effa4106b90 2024-06-06T07:02:09.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.584+0000 7eff9bfff700 1 -- 172.21.15.73:0/4065750757 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7eff9c01a020 con 0x7effa4106b90 2024-06-06T07:02:09.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.585+0000 7eff9bfff700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 0x7eff900a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.585+0000 7effa2ffd700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 0x7eff900a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.585+0000 7eff9bfff700 1 -- 172.21.15.73:0/4065750757 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7eff9c0cf1e0 con 0x7effa4106b90 2024-06-06T07:02:09.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.586+0000 7effa2ffd700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 0x7eff900a53f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400a300 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.586+0000 7effb3cddb80 1 Processor -- start 2024-06-06T07:02:09.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.586+0000 7effb3cddb80 1 -- start start 2024-06-06T07:02:09.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 0x558b87759a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87754340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b876a5810 con 0x558b87759f60 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b8764d720 con 0x558b87759640 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effb3cddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558b876a17d0 con 0x558b87753f20 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34470/0 (socket says 172.21.15.73:34470) 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 -- 172.21.15.73:0/3871097899 learned_addr learned my addr 172.21.15.73:0/3871097899 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 0x558b87759a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa37fe700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87754340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 -- 172.21.15.73:0/3871097899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 msgr2=0x558b87754340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87754340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.589 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 -- 172.21.15.73:0/3871097899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 msgr2=0x558b87759a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 0x558b87759a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 -- 172.21.15.73:0/3871097899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b87643f50 con 0x558b87759f60 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 0x558b87759a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa37fe700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87754340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.587+0000 7effa27fc700 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c79c0a280c50c4cd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.588+0000 7eff997fa700 1 -- 172.21.15.73:0/3871097899 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005390 con 0x558b87759f60 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.588+0000 7eff997fa700 1 -- 172.21.15.73:0/3871097899 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8c026070 con 0x558b87759f60 2024-06-06T07:02:09.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.588+0000 7eff997fa700 1 -- 172.21.15.73:0/3871097899 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00bb00 con 0x558b87759f60 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.588+0000 7effb3cddb80 1 -- 172.21.15.73:0/3871097899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 msgr2=0x558b877539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.588+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c009f80 comp rx=0 tx=0).stop 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- 172.21.15.73:0/3871097899 shutdown_connections 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87754340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759640 0x558b87759a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3871097899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b877539e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- 172.21.15.73:0/3871097899 >> 172.21.15.73:0/3871097899 conn(0x558b8767ecc0 msgr2=0x558b8774e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.594 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- 172.21.15.73:0/3871097899 shutdown_connections 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- 172.21.15.73:0/3871097899 wait complete. 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- start start 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 Processor -- start 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b8775fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759640 0x558b8775cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558b87760970 con 0x558b87759f60 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558b87645bf0 con 0x558b87753f20 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558b87760af0 con 0x558b87759640 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effa37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.589+0000 7effb3cddb80 10 client.0 _ll_register_callbacks cb 0x558b8763f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34480/0 (socket says 172.21.15.73:34480) 2024-06-06T07:02:09.595 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 -- 172.21.15.73:0/429475054 learned_addr learned my addr 172.21.15.73:0/429475054 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa2ffd700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b8775fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa27fc700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759640 0x558b8775cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 -- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759640 msgr2=0x558b8775cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759640 0x558b8775cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 -- 172.21.15.73:0/429475054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 msgr2=0x558b8775fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b8775fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 -- 172.21.15.73:0/429475054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b875f48d0 con 0x558b87759f60 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effa37fe700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff9c09b030 tx=0x7eff9c09ac60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875c6f3e6da779f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7eff8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9c09b7a0 con 0x558b87759f60 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7eff9c09f080 con 0x558b87759f60 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c026070 con 0x558b87759f60 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff9c09b0a0 con 0x558b87759f60 2024-06-06T07:02:09.596 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effb3cddb80 10 client.15537 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.590+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b8776b1f0 con 0x558b87759f60 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8c026070 con 0x558b87759f60 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7eff9c0b9ce0 con 0x558b87759f60 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 10 client.15537 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:09.597 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 20 client.15537 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b8776fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 4 client.15537 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b876b2d10 con 0x558b8776d830 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effa2ffd700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b8776fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7eff8a7fc700 10 client.15537 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.591+0000 7effa2ffd700 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b8776fc20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc9bb9c2b516509e server_cookie=11c37afeaa721cf0 in_seq=0 out_seq=0 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7eff8a7fc700 1 -- 172.21.15.73:0/429475054 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558b876b2d10 con 0x558b8776d830 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7eff8a7fc700 10 client.15537 handle_command_reply: tid=0 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 client.15537 shutdown 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 msgr2=0x558b8775d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff9c09b030 tx=0x7eff9c09ac60 comp rx=0 tx=0).stop 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 shutdown_connections 2024-06-06T07:02:09.598 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b8776fc20 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759f60 0x558b8775d430 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7eff8a7fc700 0 client.15537 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b8775fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 --2- 172.21.15.73:0/429475054 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759640 0x558b8775cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.592+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 >> 172.21.15.73:0/429475054 conn(0x558b8767ecc0 msgr2=0x558b877529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.593+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 shutdown_connections 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.593+0000 7effb3cddb80 1 -- 172.21.15.73:0/429475054 wait complete. 2024-06-06T07:02:09.599 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.593+0000 7effb3cddb80 20 client.15537 trim_cache size 0 max 16384 2024-06-06T07:02:09.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:09 smithi170 ceph-mon[36108]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-06-06T07:02:09.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 Processor -- start 2024-06-06T07:02:09.614 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 -- start start 2024-06-06T07:02:09.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b877539e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.615 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759f60 0x558b877754a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b0600 con 0x558b87759640 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b2140 con 0x558b87753f20 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.613+0000 7effb3cddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b0cd0 con 0x558b87759f60 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34490/0 (socket says 172.21.15.73:34490) 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 -- 172.21.15.73:0/613285539 learned_addr learned my addr 172.21.15.73:0/613285539 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa2ffd700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b877539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa37fe700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759f60 0x558b877754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 -- 172.21.15.73:0/613285539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759f60 msgr2=0x558b877754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759f60 0x558b877754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 -- 172.21.15.73:0/613285539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 msgr2=0x558b877539e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b877539e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 -- 172.21.15.73:0/613285539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8775f830 con 0x558b87759640 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa2ffd700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b877539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7effa27fc700 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0b13c40453a2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7eff98ff9700 1 -- 172.21.15.73:0/613285539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c005550 con 0x558b87759640 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7eff98ff9700 1 -- 172.21.15.73:0/613285539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8c026040 con 0x558b87759640 2024-06-06T07:02:09.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.614+0000 7eff98ff9700 1 -- 172.21.15.73:0/613285539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00f460 con 0x558b87759640 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 -- 172.21.15.73:0/613285539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 msgr2=0x558b87748510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009990 tx=0x7eff8c005770 comp rx=0 tx=0).stop 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 -- 172.21.15.73:0/613285539 shutdown_connections 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87759f60 0x558b877754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87753f20 0x558b877539e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 --2- 172.21.15.73:0/613285539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87748510 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 -- 172.21.15.73:0/613285539 >> 172.21.15.73:0/613285539 conn(0x558b8767ecc0 msgr2=0x558b87747e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 -- 172.21.15.73:0/613285539 shutdown_connections 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.615+0000 7effb3cddb80 1 -- 172.21.15.73:0/613285539 wait complete. 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 -- start start 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 Processor -- start 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87765de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759f60 0x558b87762db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b1a70 con 0x558b87759640 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7effa41b13a0 con 0x558b87759f60 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7effa41ae5a0 con 0x558b87753f20 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effb3cddb80 10 client.0 _ll_register_callbacks cb 0x558b8763f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:09.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34494/0 (socket says 172.21.15.73:34494) 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 -- 172.21.15.73:0/3683310501 learned_addr learned my addr 172.21.15.73:0/3683310501 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87765de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 -- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 msgr2=0x558b87765de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87765de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 -- 172.21.15.73:0/3683310501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759f60 msgr2=0x558b87762db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759f60 0x558b87762db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa27fc700 1 -- 172.21.15.73:0/3683310501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b87763370 con 0x558b87759640 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.616+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87765de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:09.623 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effa27fc700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009ce0 tx=0x7eff8c003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba581d93f49e151d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00ba80 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7eff8c00bc20 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8c026040 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7eff8c00be20 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 10 client.15543 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558b877d8d10 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8c026040 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7eff8c02a460 con 0x558b87759640 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 10 client.15543 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:09.624 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 20 client.15543 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.617+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b877daf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 4 client.15543 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558b876ad290 con 0x558b8776d830 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b877daf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effa2ffd700 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b877daf40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e0795170fb9467 server_cookie=3f0bafdb345eebeb in_seq=0 out_seq=0 2024-06-06T07:02:09.625 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7eff8a7fc700 10 client.15543 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:09.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7eff8a7fc700 1 -- 172.21.15.73:0/3683310501 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x558b876ad290 con 0x558b8776d830 2024-06-06T07:02:09.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7eff8a7fc700 10 client.15543 handle_command_reply: tid=0 2024-06-06T07:02:09.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 client.15543 shutdown 2024-06-06T07:02:09.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 msgr2=0x558b87766320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.626 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7eff8c009ce0 tx=0x7eff8c003b10 comp rx=0 tx=0).stop 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 shutdown_connections 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558b8776d830 0x558b877daf40 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558b87753f20 0x558b87765de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558b87759f60 0x558b87762db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 --2- 172.21.15.73:0/3683310501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558b87759640 0x558b87766320 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 >> 172.21.15.73:0/3683310501 conn(0x558b8767ecc0 msgr2=0x558b8775aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.618+0000 7eff8a7fc700 0 client.15543 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 shutdown_connections 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effb3cddb80 1 -- 172.21.15.73:0/3683310501 wait complete. 2024-06-06T07:02:09.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effb3cddb80 20 client.15543 trim_cache size 0 max 16384 2024-06-06T07:02:09.628 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:02:09.628 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "no active scrubs running", 2024-06-06T07:02:09.628 INFO:teuthology.orchestra.run.smithi073.stdout: "scrubs": {} 2024-06-06T07:02:09.628 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 msgr2=0x7eff900a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 0x7eff900a53f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7eff94005950 tx=0x7eff9400a300 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 msgr2=0x7effa40fd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7eff9c007ee0 tx=0x7eff9c00d3b0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 shutdown_connections 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7eff900a2f30 0x7eff900a53f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa4106b90 0x7effa40fd8f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7effa4100b50 0x7effa40fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 --2- 172.21.15.73:0/4065750757 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7effa4101510 0x7effa40fd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.619+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 >> 172.21.15.73:0/4065750757 conn(0x7effa4077a00 msgr2=0x7effa410e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:09.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.620+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 shutdown_connections 2024-06-06T07:02:09.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:09.620+0000 7effa945a700 1 -- 172.21.15.73:0/4065750757 wait complete. 2024-06-06T07:02:10.430 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:02:10.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:02:10.735 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:11 smithi073 ceph-mon[32303]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:11 smithi136 ceph-mon[35901]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:11 smithi170 ceph-mon[36108]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 -- 172.21.15.73:0/4026576352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 msgr2=0x7f502010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 --2- 172.21.15.73:0/4026576352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f502010e240 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f501800b0a0 tx=0x7f501800b3b0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 -- 172.21.15.73:0/4026576352 shutdown_connections 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 --2- 172.21.15.73:0/4026576352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f502010e240 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 --2- 172.21.15.73:0/4026576352 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020107400 0x7f50201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 --2- 172.21.15.73:0/4026576352 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5020101390 0x7f5020101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 -- 172.21.15.73:0/4026576352 >> 172.21.15.73:0/4026576352 conn(0x7f5020078340 msgr2=0x7f5020078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 -- 172.21.15.73:0/4026576352 shutdown_connections 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 -- 172.21.15.73:0/4026576352 wait complete. 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.481+0000 7f5026b65700 1 Processor -- start 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 -- start start 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020101390 0x7f5020103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5020107400 0x7f5020103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f50201115a0 con 0x7f5020101d50 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5020111420 con 0x7f5020101390 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5026b65700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5020111720 con 0x7f5020107400 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34522/0 (socket says 172.21.15.73:34522) 2024-06-06T07:02:12.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 -- 172.21.15.73:0/1516502206 learned_addr learned my addr 172.21.15.73:0/1516502206 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f5024901700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020101390 0x7f5020103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 -- 172.21.15.73:0/1516502206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5020107400 msgr2=0x7f5020103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5020107400 0x7f5020103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 -- 172.21.15.73:0/1516502206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020101390 msgr2=0x7f5020103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020101390 0x7f5020103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 -- 172.21.15.73:0/1516502206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50201a70f0 con 0x7f5020101d50 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.482+0000 7f501ffff700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f500c00d980 tx=0x7f500c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.483+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c017070 con 0x7f5020101d50 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.483+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f500c004830 con 0x7f5020101d50 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.483+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50201a73e0 con 0x7f5020101d50 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.483+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f500c005440 con 0x7f5020101d50 2024-06-06T07:02:12.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.483+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50201a7920 con 0x7f5020101d50 2024-06-06T07:02:12.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.484+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5020069180 con 0x7f5020101d50 2024-06-06T07:02:12.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.485+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f500c0055e0 con 0x7f5020101d50 2024-06-06T07:02:12.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.486+0000 7f501dffb700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 0x7f50100a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:12.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.486+0000 7f5024901700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 0x7f50100a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:12.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.486+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f500c0cfde0 con 0x7f5020101d50 2024-06-06T07:02:12.488 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.487+0000 7f5024901700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 0x7f50100a53f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5014005fd0 tx=0x7f5014005f60 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:12.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.490+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f500c020020 con 0x7f5020101d50 2024-06-06T07:02:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.745+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50201a7b80 con 0x7f5020101d50 2024-06-06T07:02:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.746+0000 7f501dffb700 1 -- 172.21.15.73:0/1516502206 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11932 (secure 0 0 0) 0x7f500c09c420 con 0x7f5020101d50 2024-06-06T07:02:12.747 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:02:12.747 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T07:01:19.680600+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24380":{"gid":24380,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/687158910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":687158910},{"type":"v1","addr":"172.21.15.136:6835","nonce":687158910}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24419,24433,24421,24404,24415]},"id":1}]} 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 msgr2=0x7f50100a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 0x7f50100a53f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5014005fd0 tx=0x7f5014005f60 comp rx=0 tx=0).stop 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 msgr2=0x7f50201035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f500c00d980 tx=0x7f500c00dd40 comp rx=0 tx=0).stop 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 shutdown_connections 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f50100a2f30 0x7f50100a53f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5020107400 0x7f5020103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5020101390 0x7f5020103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 --2- 172.21.15.73:0/1516502206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5020101d50 0x7f50201035b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 >> 172.21.15.73:0/1516502206 conn(0x7f5020078340 msgr2=0x7f5020100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 shutdown_connections 2024-06-06T07:02:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:12.749+0000 7f5026b65700 1 -- 172.21.15.73:0/1516502206 wait complete. 2024-06-06T07:02:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 30 2024-06-06T07:02:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:13 smithi073 ceph-mon[32303]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:13 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1516502206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:13 smithi136 ceph-mon[35901]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:13 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1516502206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:13.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-06T07:02:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:13 smithi170 ceph-mon[36108]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-06-06T07:02:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:13 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1516502206' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:13.832 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:15.371 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:15 smithi073 ceph-mon[32303]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-06-06T07:02:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:15 smithi136 ceph-mon[35901]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-06-06T07:02:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:15 smithi170 ceph-mon[36108]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 27 op/s 2024-06-06T07:02:15.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 -- 172.21.15.73:0/2265243689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 msgr2=0x7fce8410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/2265243689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce8410f3f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00b0d0 tx=0x7fce7c00b3e0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 -- 172.21.15.73:0/2265243689 shutdown_connections 2024-06-06T07:02:15.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/2265243689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce841027a0 0x7fce84102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.778 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/2265243689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce8410f3f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/2265243689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce84103150 0x7fce84107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.637+0000 7fce8b6eb700 1 -- 172.21.15.73:0/2265243689 >> 172.21.15.73:0/2265243689 conn(0x7fce84079520 msgr2=0x7fce84079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.638+0000 7fce8b6eb700 1 -- 172.21.15.73:0/2265243689 shutdown_connections 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.638+0000 7fce8b6eb700 1 -- 172.21.15.73:0/2265243689 wait complete. 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.638+0000 7fce8b6eb700 1 Processor -- start 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.638+0000 7fce8b6eb700 1 -- start start 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce841027a0 0x7fce8419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce84103150 0x7fce8419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce84112660 con 0x7fce84107870 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce841124e0 con 0x7fce841027a0 2024-06-06T07:02:15.779 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce8b6eb700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce841127e0 con 0x7fce84103150 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53912/0 (socket says 172.21.15.73:53912) 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 -- 172.21.15.73:0/4125439412 learned_addr learned my addr 172.21.15.73:0/4125439412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce88c86700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce84103150 0x7fce8419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89487700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce841027a0 0x7fce8419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 -- 172.21.15.73:0/4125439412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce84103150 msgr2=0x7fce8419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce84103150 0x7fce8419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 -- 172.21.15.73:0/4125439412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce841027a0 msgr2=0x7fce8419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce841027a0 0x7fce8419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.780 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 -- 172.21.15.73:0/4125439412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce84198fc0 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce89c88700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009850 tx=0x7fce7c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.639+0000 7fce7a7fc700 1 -- 172.21.15.73:0/4125439412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01f070 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.640+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce84199250 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.640+0000 7fce7a7fc700 1 -- 172.21.15.73:0/4125439412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce7c004c50 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.640+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce841a5570 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.640+0000 7fce7a7fc700 1 -- 172.21.15.73:0/4125439412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c01b670 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.643+0000 7fce7a7fc700 1 -- 172.21.15.73:0/4125439412 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fce7c022070 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce7a7fc700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 0x7fce700a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce89487700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 0x7fce700a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce7a7fc700 1 -- 172.21.15.73:0/4125439412 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fce7c00f020 con 0x7fce84107870 2024-06-06T07:02:15.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce89487700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 0x7fce700a5370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce841999a0 tx=0x7fce80009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce95f6eb80 1 Processor -- start 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.644+0000 7fce95f6eb80 1 -- start start 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 0x55a400f649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f65340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a400eb6810 con 0x55a400f6a640 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a400e5e720 con 0x55a400f6af60 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a400eb27d0 con 0x55a400f64f20 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53920/0 (socket says 172.21.15.73:53920) 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 -- 172.21.15.73:0/3241235490 learned_addr learned my addr 172.21.15.73:0/3241235490 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:15.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce88c86700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 0x55a400f649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89c88700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f65340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 -- 172.21.15.73:0/3241235490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 msgr2=0x55a400f65340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f65340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 -- 172.21.15.73:0/3241235490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 msgr2=0x55a400f649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 0x55a400f649e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 -- 172.21.15.73:0/3241235490 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a400e54f50 con 0x55a400f6a640 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89c88700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce88c86700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 0x55a400f649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce89487700 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce80011dd0 tx=0x7fce80014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e9bc3a5ee35ff2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.645+0000 7fce6f7fe700 1 -- 172.21.15.73:0/3241235490 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80003880 con 0x55a400f6a640 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.646+0000 7fce6f7fe700 1 -- 172.21.15.73:0/3241235490 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce8001e420 con 0x55a400f6a640 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.646+0000 7fce6f7fe700 1 -- 172.21.15.73:0/3241235490 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80026660 con 0x55a400f6a640 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.646+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3241235490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 msgr2=0x55a400f6aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.646+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fce80011dd0 tx=0x7fce80014000 comp rx=0 tx=0).stop 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3241235490 shutdown_connections 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f65340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6a640 0x55a400f6aa20 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3241235490 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6af60 0x55a400f649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3241235490 >> 172.21.15.73:0/3241235490 conn(0x55a400e8fcc0 msgr2=0x55a400f5f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3241235490 shutdown_connections 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3241235490 wait complete. 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- start start 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 Processor -- start 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f70ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a400f719d0 con 0x55a400f6af60 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a400e56bf0 con 0x55a400f64f20 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.647+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a400f71b50 con 0x55a400f6a640 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce95f6eb80 10 client.0 _ll_register_callbacks cb 0x55a400e50920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53936/0 (socket says 172.21.15.73:53936) 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 -- 172.21.15.73:0/3463837900 learned_addr learned my addr 172.21.15.73:0/3463837900 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89487700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f70ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce88c86700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 -- 172.21.15.73:0/3463837900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 msgr2=0x55a400f6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 -- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 msgr2=0x55a400f70ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f70ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 -- 172.21.15.73:0/3463837900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a400e058d0 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce89c88700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce7c09c7b0 tx=0x7fce7c09c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7963e6b7362ddac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c09d1b0 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fce7c09cd10 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8001e420 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce7c09cab0 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.648+0000 7fce95f6eb80 10 client.15561 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a400f7c2e0 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce8001e420 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce7c011070 con 0x55a400f6af60 2024-06-06T07:02:15.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 10 client.15561 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 10 client.15561 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 20 client.15561 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f7e7b0 0x55a400f80bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 4 client.15561 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.649+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a400ec3d10 con 0x55a400f7e7b0 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.650+0000 7fce89487700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f7e7b0 0x55a400f80bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.650+0000 7fce6cff9700 10 client.15561 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.650+0000 7fce89487700 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f7e7b0 0x55a400f80bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1544113bfa7f06cf server_cookie=d768d1036744a2f5 in_seq=0 out_seq=0 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce6cff9700 1 -- 172.21.15.73:0/3463837900 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a400ec3d10 con 0x55a400f7e7b0 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce6cff9700 10 client.15561 handle_command_reply: tid=0 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 client.15561 shutdown 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 msgr2=0x55a400f6e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fce7c09c7b0 tx=0x7fce7c09c500 comp rx=0 tx=0).stop 2024-06-06T07:02:15.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 shutdown_connections 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400f6af60 0x55a400f6e520 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f70ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f7e7b0 0x55a400f80bd0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/3463837900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce6cff9700 0 client.15561 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 >> 172.21.15.73:0/3463837900 conn(0x55a400e8fcc0 msgr2=0x55a400f639d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 shutdown_connections 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 1 -- 172.21.15.73:0/3463837900 wait complete. 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.651+0000 7fce95f6eb80 20 client.15561 trim_cache size 0 max 16384 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 Processor -- start 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 -- start start 2024-06-06T07:02:15.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f864a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ae370 con 0x55a400e8fcc0 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce841afeb0 con 0x55a400f64f20 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce841aea40 con 0x55a400f6a640 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53952/0 (socket says 172.21.15.73:53952) 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89487700 1 -- 172.21.15.73:0/320423336 learned_addr learned my addr 172.21.15.73:0/320423336 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce88c86700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89487700 1 -- 172.21.15.73:0/320423336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 msgr2=0x55a400f864a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89c88700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f864a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.682+0000 7fce89487700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f864a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce89487700 1 -- 172.21.15.73:0/320423336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 msgr2=0x55a400f59510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.789 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce89487700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f59510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce89487700 1 -- 172.21.15.73:0/320423336 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a400f811b0 con 0x55a400e8fcc0 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce88c86700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce89c88700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce89487700 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce80025000 tx=0x7fce8001e520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dee038e72a1603b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce6effd700 1 -- 172.21.15.73:0/320423336 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce8001ea30 con 0x55a400e8fcc0 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce6effd700 1 -- 172.21.15.73:0/320423336 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce8001ebd0 con 0x55a400e8fcc0 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.683+0000 7fce6effd700 1 -- 172.21.15.73:0/320423336 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80003d90 con 0x55a400e8fcc0 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- 172.21.15.73:0/320423336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 msgr2=0x55a400f649e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fce80025000 tx=0x7fce8001e520 comp rx=0 tx=0).stop 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- 172.21.15.73:0/320423336 shutdown_connections 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f6a640 0x55a400f864a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f649e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/320423336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f64f20 0x55a400f59510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- 172.21.15.73:0/320423336 >> 172.21.15.73:0/320423336 conn(0x55a400f6dba0 msgr2=0x55a400f58e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- 172.21.15.73:0/320423336 shutdown_connections 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- 172.21.15.73:0/320423336 wait complete. 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 -- start start 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.684+0000 7fce95f6eb80 1 Processor -- start 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6a640 0x55a400f85c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af7e0 con 0x55a400e8fcc0 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fce841af110 con 0x55a400f6a640 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fce841ac310 con 0x55a400f64f20 2024-06-06T07:02:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce95f6eb80 10 client.0 _ll_register_callbacks cb 0x55a400e50920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53964/0 (socket says 172.21.15.73:53964) 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 -- 172.21.15.73:0/1704544975 learned_addr learned my addr 172.21.15.73:0/1704544975 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce88c86700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89c88700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6a640 0x55a400f85c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 -- 172.21.15.73:0/1704544975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 msgr2=0x55a400f776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 -- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6a640 msgr2=0x55a400f85c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6a640 0x55a400f85c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.685+0000 7fce89487700 1 -- 172.21.15.73:0/1704544975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a400f86480 con 0x55a400e8fcc0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce89487700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fce80012e90 tx=0x7fce800119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fda3ec14357613 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:15.792 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce80041070 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7fce80003c10 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce8001ebd0 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fce8004d050 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 10 client.15567 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a400fe9040 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fce8001ebd0 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fce80004720 con 0x55a400e8fcc0 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 10 client.15567 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 10 client.15567 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 20 client.15567 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f6af60 0x55a400feb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 4 client.15567 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:15.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.686+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a400ebe290 con 0x55a400f6af60 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce88c86700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f6af60 0x55a400feb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce6cff9700 10 client.15567 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce88c86700 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f6af60 0x55a400feb3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=acaa26bdfb052b39 server_cookie=c4221db2a71246c5 in_seq=0 out_seq=0 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce6cff9700 1 -- 172.21.15.73:0/1704544975 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a400ebe290 con 0x55a400f6af60 2024-06-06T07:02:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce6cff9700 10 client.15567 handle_command_reply: tid=0 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.687+0000 7fce95f6eb80 1 client.15567 shutdown 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 msgr2=0x55a400f771b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fce80012e90 tx=0x7fce800119e0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 shutdown_connections 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a400f6a640 0x55a400f85c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a400e8fcc0 0x55a400f771b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55a400f6af60 0x55a400feb3f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce6cff9700 0 client.15567 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 --2- 172.21.15.73:0/1704544975 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a400f64f20 0x55a400f776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 >> 172.21.15.73:0/1704544975 conn(0x55a400f6dba0 msgr2=0x55a400f6c950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 shutdown_connections 2024-06-06T07:02:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 1 -- 172.21.15.73:0/1704544975 wait complete. 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce95f6eb80 20 client.15567 trim_cache size 0 max 16384 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 msgr2=0x7fce700a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 0x7fce700a5370 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fce841999a0 tx=0x7fce80009260 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 msgr2=0x7fce84198860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fce7c009850 tx=0x7fce7c007a20 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 shutdown_connections 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fce700a2eb0 0x7fce700a5370 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce84107870 0x7fce84198860 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fce841027a0 0x7fce8419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.688+0000 7fce8b6eb700 1 --2- 172.21.15.73:0/4125439412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fce84103150 0x7fce8419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.689+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 >> 172.21.15.73:0/4125439412 conn(0x7fce84079520 msgr2=0x7fce841089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.689+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 shutdown_connections 2024-06-06T07:02:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:15.689+0000 7fce8b6eb700 1 -- 172.21.15.73:0/4125439412 wait complete. 2024-06-06T07:02:16.412 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 1. 2024-06-06T07:02:16.413 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 systemd[1]: Stopped Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T07:02:16.413 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Consumed 1.039s CPU time. 2024-06-06T07:02:16.413 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 systemd[1]: Starting Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:02:16.413 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 podman[69730]: 2024-06-06 07:02:16.194568047 +0000 UTC m=+0.018095395 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:02:16.413 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 podman[69730]: 2024-06-06 07:02:16.300133727 +0000 UTC m=+0.123661102 container create 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T07:02:16.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-06T07:02:16.672 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 podman[69730]: 2024-06-06 07:02:16.527810553 +0000 UTC m=+0.351337883 container init 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T07:02:16.672 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 podman[69730]: 2024-06-06 07:02:16.531179583 +0000 UTC m=+0.354706918 container start 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-06T07:02:16.673 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 ceph-mds[69750]: -- 172.21.15.136:0/4159335884 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f16138ba40 con 0x55f16135ec00 2024-06-06T07:02:16.673 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: starting mds.e at 2024-06-06T07:02:16.673 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 bash[69730]: 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 2024-06-06T07:02:16.673 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:02:16 smithi136 systemd[1]: Started Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T07:02:16.856 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 -- 172.21.15.73:0/132628237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d54100260 msgr2=0x7f9d54100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:16.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 --2- 172.21.15.73:0/132628237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d54100260 0x7f9d54100640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d44009900 tx=0x7f9d4402a410 comp rx=0 tx=0).stop 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 -- 172.21.15.73:0/132628237 shutdown_connections 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 --2- 172.21.15.73:0/132628237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54105330 0x7f9d5410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 --2- 172.21.15.73:0/132628237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54100c10 0x7f9d54104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 --2- 172.21.15.73:0/132628237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d54100260 0x7f9d54100640 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.856+0000 7f9d5934f640 1 -- 172.21.15.73:0/132628237 >> 172.21.15.73:0/132628237 conn(0x7f9d54076e30 msgr2=0x7f9d540fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.857+0000 7f9d5934f640 1 -- 172.21.15.73:0/132628237 shutdown_connections 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.857+0000 7f9d5934f640 1 -- 172.21.15.73:0/132628237 wait complete. 2024-06-06T07:02:16.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.857+0000 7f9d5934f640 1 Processor -- start 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.857+0000 7f9d5934f640 1 -- start start 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 0x7f9d54198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54105330 0x7f9d54198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54110b70 con 0x7f9d5419ca10 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541109f0 con 0x7f9d54100260 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d54110cf0 con 0x7f9d54105330 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53992/0 (socket says 172.21.15.73:53992) 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 -- 172.21.15.73:0/2088465692 learned_addr learned my addr 172.21.15.73:0/2088465692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 0x7f9d54198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54105330 0x7f9d54198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 -- 172.21.15.73:0/2088465692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54105330 msgr2=0x7f9d54198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54105330 0x7f9d54198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 -- 172.21.15.73:0/2088465692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 msgr2=0x7f9d54198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 0x7f9d54198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 -- 172.21.15.73:0/2088465692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5419d500 con 0x7f9d5419ca10 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 0x7f9d54198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00eaa0 tx=0x7f9d4c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d50ff9640 1 -- 172.21.15.73:0/2088465692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004d20 con 0x7f9d5419ca10 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5419d7f0 con 0x7f9d5419ca10 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d50ff9640 1 -- 172.21.15.73:0/2088465692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c005020 con 0x7f9d5419ca10 2024-06-06T07:02:16.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.858+0000 7f9d50ff9640 1 -- 172.21.15.73:0/2088465692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0116b0 con 0x7f9d5419ca10 2024-06-06T07:02:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.859+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d541a4fe0 con 0x7f9d5419ca10 2024-06-06T07:02:16.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.861+0000 7f9d50ff9640 1 -- 172.21.15.73:0/2088465692 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f9d4c0051d0 con 0x7f9d5419ca10 2024-06-06T07:02:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 Processor -- start 2024-06-06T07:02:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 -- start start 2024-06-06T07:02:16.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d50ff9640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 0x7f9d400a5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d50ff9640 1 -- 172.21.15.73:0/2088465692 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f9d4c0cfc50 con 0x7f9d5419ca10 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a8987550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 0x7f9d400a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a8991e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5601a88eb4f0 con 0x5601a8992350 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601a8889870 con 0x5601a8987170 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d5c056740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5601a88e74b0 con 0x5601a8987a90 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54004/0 (socket says 172.21.15.73:54004) 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d52ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a8987550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.862+0000 7f9d537fe640 1 -- 172.21.15.73:0/2289938735 learned_addr learned my addr 172.21.15.73:0/2289938735 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a8991e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 -- 172.21.15.73:0/2289938735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 msgr2=0x5601a8991e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:16.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a8991e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 -- 172.21.15.73:0/2289938735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 msgr2=0x5601a8987550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a8987550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 -- 172.21.15.73:0/2289938735 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a88a59b0 con 0x5601a8992350 2024-06-06T07:02:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a8987550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:16.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 0x7f9d400a5380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009330 tx=0x7f9d3c0156d0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:16.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d537fe640 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009270 tx=0x7f9d4c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=153dc611c036cea5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d497fa640 1 -- 172.21.15.73:0/2289938735 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c01f070 con 0x5601a8992350 2024-06-06T07:02:16.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d497fa640 1 -- 172.21.15.73:0/2289938735 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c09ff20 con 0x5601a8992350 2024-06-06T07:02:16.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.863+0000 7f9d497fa640 1 -- 172.21.15.73:0/2289938735 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c00caf0 con 0x5601a8992350 2024-06-06T07:02:16.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- 172.21.15.73:0/2289938735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 msgr2=0x5601a8994740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009270 tx=0x7f9d4c005620 comp rx=0 tx=0).stop 2024-06-06T07:02:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- 172.21.15.73:0/2289938735 shutdown_connections 2024-06-06T07:02:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a8994740 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a8991e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 --2- 172.21.15.73:0/2289938735 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a8987550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- 172.21.15.73:0/2289938735 >> 172.21.15.73:0/2289938735 conn(0x5601a8983680 msgr2=0x5601a8983a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- 172.21.15.73:0/2289938735 shutdown_connections 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- 172.21.15.73:0/2289938735 wait complete. 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 -- start start 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.864+0000 7f9d5c056740 1 Processor -- start 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5601a8999330 con 0x5601a8992350 2024-06-06T07:02:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601a8874ae0 con 0x5601a8987170 2024-06-06T07:02:16.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5601a89994b0 con 0x5601a8987a90 2024-06-06T07:02:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d5c056740 10 client.0 _ll_register_callbacks cb 0x5601a88ad9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:16.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54020/0 (socket says 172.21.15.73:54020) 2024-06-06T07:02:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 -- 172.21.15.73:0/762148978 learned_addr learned my addr 172.21.15.73:0/762148978 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:16.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 -- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 msgr2=0x5601a898f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 -- 172.21.15.73:0/762148978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 msgr2=0x5601a898f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 -- 172.21.15.73:0/762148978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a85ff210 con 0x5601a8992350 2024-06-06T07:02:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d537fe640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c097fd0 tx=0x7f9d4c09bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe1838b1583eb45 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:16.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.865+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0b8030 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f9d4c099e70 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4c09ff20 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d4c01f040 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 10 client.15585 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601a86cb620 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c09ff20 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d4c00f040 con 0x5601a8992350 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 10 client.15585 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 20 client.15585 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a89a66d0 0x5601a89a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 4 client.15585 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601a88fef10 con 0x5601a89a66d0 2024-06-06T07:02:16.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.866+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a89a66d0 0x5601a89a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.867+0000 7f9d3a7fc640 10 client.15585 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.867+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a89a66d0 0x5601a89a8ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=936de14bd80fa87 server_cookie=6152f3a314a88231 in_seq=0 out_seq=0 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/762148978 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5601a88fef10 con 0x5601a89a66d0 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d3a7fc640 10 client.15585 handle_command_reply: tid=0 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 client.15585 shutdown 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 msgr2=0x5601a898fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c097fd0 tx=0x7f9d4c09bc30 comp rx=0 tx=0).stop 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 shutdown_connections 2024-06-06T07:02:16.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a89a66d0 0x5601a89a8ac0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898fcf0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d3a7fc640 0 client.15585 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 --2- 172.21.15.73:0/762148978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 >> 172.21.15.73:0/762148978 conn(0x5601a8983680 msgr2=0x5601a89893b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 shutdown_connections 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 1 -- 172.21.15.73:0/762148978 wait complete. 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.868+0000 7f9d5c056740 20 client.15585 trim_cache size 0 max 16384 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.899+0000 7f9d5c056740 1 Processor -- start 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.899+0000 7f9d5c056740 1 -- start start 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a89ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:16.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8992350 0x5601a8981f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541acaf0 con 0x5601a8987a90 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541ae630 con 0x5601a8987170 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d5c056740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541ad1c0 con 0x5601a8992350 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54026/0 (socket says 172.21.15.73:54026) 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 -- 172.21.15.73:0/3754285594 learned_addr learned my addr 172.21.15.73:0/3754285594 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a89ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 -- 172.21.15.73:0/3754285594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8992350 msgr2=0x5601a8981f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d537fe640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8992350 0x5601a8981f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8992350 0x5601a8981f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 -- 172.21.15.73:0/3754285594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 msgr2=0x5601a89ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a89ae0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 -- 172.21.15.73:0/3754285594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601a89848b0 con 0x5601a8987a90 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a89ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0094f0 tx=0x7f9d3c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa1ea4badb411f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.900+0000 7f9d48ff9640 1 -- 172.21.15.73:0/3754285594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c026460 con 0x5601a8987a90 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d48ff9640 1 -- 172.21.15.73:0/3754285594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c02b070 con 0x5601a8987a90 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d48ff9640 1 -- 172.21.15.73:0/3754285594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d3c0177c0 con 0x5601a8987a90 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 -- 172.21.15.73:0/3754285594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 msgr2=0x5601a89ae630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c0094f0 tx=0x7f9d3c01c000 comp rx=0 tx=0).stop 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 -- 172.21.15.73:0/3754285594 shutdown_connections 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8992350 0x5601a8981f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8987a90 0x5601a89ae630 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 --2- 172.21.15.73:0/3754285594 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a89ae0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 -- 172.21.15.73:0/3754285594 >> 172.21.15.73:0/3754285594 conn(0x5601a8983680 msgr2=0x5601a8983a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:17.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 -- 172.21.15.73:0/3754285594 shutdown_connections 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.901+0000 7f9d5c056740 1 -- 172.21.15.73:0/3754285594 wait complete. 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 -- start start 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 Processor -- start 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541adf60 con 0x5601a8992350 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541ad890 con 0x5601a8987170 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541aaad0 con 0x5601a8987a90 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d5c056740 10 client.0 _ll_register_callbacks cb 0x7f9d44032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:17.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54038/0 (socket says 172.21.15.73:54038) 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 -- 172.21.15.73:0/4248748940 learned_addr learned my addr 172.21.15.73:0/4248748940 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 -- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 msgr2=0x5601a898c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 -- 172.21.15.73:0/4248748940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 msgr2=0x5601a898c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d537fe640 1 -- 172.21.15.73:0/4248748940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601a8a10170 con 0x5601a8992350 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.902+0000 7f9d4a7fc640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d537fe640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009270 tx=0x7f9d4c097b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2940cc699555c09c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:17.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c0b8070 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9466+0+0 (secure 0 0 0) 0x7f9d4c0ba2c0 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d3c02b070 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f9d4c0ad050 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 10 client.15591 fetch_fsmap learned FSMap version 30 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5601a8a11460 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d3c02b070 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9d4c0a10b0 con 0x5601a8992350 2024-06-06T07:02:17.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 10 client.15591 fetch_fsmap finished waiting for FSMap version 30 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 20 client.15591 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a8a119f0 0x5601a8a13e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 4 client.15591 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.903+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601a88f8c20 con 0x5601a8a119f0 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.904+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a8a119f0 0x5601a8a13e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.904+0000 7f9d52ffd640 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a8a119f0 0x5601a8a13e10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e7de14d5aae7be4 server_cookie=4d267438fa9aa9c in_seq=0 out_seq=0 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.904+0000 7f9d3a7fc640 10 client.15591 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d3a7fc640 1 -- 172.21.15.73:0/4248748940 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5601a88f8c20 con 0x5601a8a119f0 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d3a7fc640 10 client.15591 handle_command_reply: tid=0 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 client.15591 shutdown 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 msgr2=0x5601a898ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:17.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009270 tx=0x7f9d4c097b70 comp rx=0 tx=0).stop 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 shutdown_connections 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601a8a119f0 0x5601a8a13e10 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601a8992350 0x5601a898ce20 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601a8987a90 0x5601a898c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 --2- 172.21.15.73:0/4248748940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601a8987170 0x5601a898c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 >> 172.21.15.73:0/4248748940 conn(0x5601a8983680 msgr2=0x5601a8981f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d3a7fc640 0 client.15591 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 shutdown_connections 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 1 -- 172.21.15.73:0/4248748940 wait complete. 2024-06-06T07:02:17.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.907+0000 7f9d5c056740 20 client.15591 trim_cache size 0 max 16384 2024-06-06T07:02:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:02:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:02:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:02:17.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 msgr2=0x7f9d400a5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 0x7f9d400a5380 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009330 tx=0x7f9d3c0156d0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 msgr2=0x7f9d5419cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c00eaa0 tx=0x7f9d4c00eea0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 shutdown_connections 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d5419ca10 0x7f9d5419cec0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d400a2ec0 0x7f9d400a5380 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9d54105330 0x7f9d54198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 --2- 172.21.15.73:0/2088465692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9d54100260 0x7f9d54198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 >> 172.21.15.73:0/2088465692 conn(0x7f9d54076e30 msgr2=0x7f9d5410dad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 shutdown_connections 2024-06-06T07:02:17.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:16.908+0000 7f9d5934f640 1 -- 172.21.15.73:0/2088465692 wait complete. 2024-06-06T07:02:17.263 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:17 smithi073 ceph-mon[32303]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T07:02:17.264 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:17 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:02:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:17 smithi136 ceph-mon[35901]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T07:02:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:17 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:02:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:17 smithi170 ceph-mon[36108]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 341 B/s wr, 7 op/s 2024-06-06T07:02:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:17 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:02:18.450 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:18 smithi073 ceph-mon[32303]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:02:18.450 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:18 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/887133051,v1:172.21.15.136:6835/887133051] up:boot 2024-06-06T07:02:18.450 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:18 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:02:18.450 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:18 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:02:18.450 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:18 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:02:18.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:18 smithi136 ceph-mon[35901]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:02:18.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:18 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/887133051,v1:172.21.15.136:6835/887133051] up:boot 2024-06-06T07:02:18.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:18 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:02:18.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:18 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:02:18.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:18 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:02:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:18 smithi170 ceph-mon[36108]: osdmap e102: 12 total, 12 up, 12 in 2024-06-06T07:02:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:18 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/887133051,v1:172.21.15.136:6835/887133051] up:boot 2024-06-06T07:02:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:18 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:02:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:18 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:02:18.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:18 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:02:18.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 -- 172.21.15.73:0/2503484324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b401073f0 msgr2=0x7f1b401077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 --2- 172.21.15.73:0/2503484324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b401073f0 0x7f1b401077d0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009990 tx=0x7f1b30009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 -- 172.21.15.73:0/2503484324 shutdown_connections 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 --2- 172.21.15.73:0/2503484324 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 0x7f1b4010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 --2- 172.21.15.73:0/2503484324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b401073f0 0x7f1b401077d0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 --2- 172.21.15.73:0/2503484324 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b40101380 0x7f1b40101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 -- 172.21.15.73:0/2503484324 >> 172.21.15.73:0/2503484324 conn(0x7f1b40078300 msgr2=0x7f1b40078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 -- 172.21.15.73:0/2503484324 shutdown_connections 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.697+0000 7f1b4578c700 1 -- 172.21.15.73:0/2503484324 wait complete. 2024-06-06T07:02:18.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 Processor -- start 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 -- start start 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 0x7f1b401035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b401073f0 0x7f1b40103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40111610 con 0x7f1b40101380 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40111490 con 0x7f1b40101d40 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.698+0000 7f1b4578c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b40111790 con 0x7f1b401073f0 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54054/0 (socket says 172.21.15.73:54054) 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 -- 172.21.15.73:0/3725936345 learned_addr learned my addr 172.21.15.73:0/3725936345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:18.701 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3f7fe700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b401073f0 0x7f1b40103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 0x7f1b401035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 -- 172.21.15.73:0/3725936345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b401073f0 msgr2=0x7f1b40103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b401073f0 0x7f1b40103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 -- 172.21.15.73:0/3725936345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 msgr2=0x7f1b401035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 0x7f1b401035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 -- 172.21.15.73:0/3725936345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b401a7120 con 0x7f1b40101380 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1b30000c00 tx=0x7f1b30003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b27fff700 1 -- 172.21.15.73:0/3725936345 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b30027070 con 0x7f1b40101380 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b27fff700 1 -- 172.21.15.73:0/3725936345 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b3000bdc0 con 0x7f1b40101380 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b401a73b0 con 0x7f1b40101380 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.699+0000 7f1b27fff700 1 -- 172.21.15.73:0/3725936345 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3000f460 con 0x7f1b40101380 2024-06-06T07:02:18.702 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.700+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b401a7860 con 0x7f1b40101380 2024-06-06T07:02:18.703 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.702+0000 7f1b27fff700 1 -- 172.21.15.73:0/3725936345 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f1b3002a070 con 0x7f1b40101380 2024-06-06T07:02:18.704 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.703+0000 7f1b27fff700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 0x7f1b280a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.703+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 0x7f1b280a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b27fff700 1 -- 172.21.15.73:0/3725936345 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1b300d0b30 con 0x7f1b40101380 2024-06-06T07:02:18.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 0x7f1b280a53f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b40078870 tx=0x7f1b34009190 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 Processor -- start 2024-06-06T07:02:18.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 -- start start 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 0x557883ee8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883ee2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557883dd4f60 con 0x557883ee8880 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557883e33850 con 0x557883ee27a0 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557883d9b460 con 0x557883ee7f60 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54056/0 (socket says 172.21.15.73:54056) 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.704+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3030840555 learned_addr learned my addr 172.21.15.73:0/3030840555 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:18.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3f7fe700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883ee2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 0x557883ee8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3030840555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 msgr2=0x557883ee8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 0x557883ee8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3030840555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 msgr2=0x557883ee2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883ee2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3030840555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557883d828d0 con 0x557883ee8880 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 0x557883ee8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=205f67822d8c1615 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b257fa700 1 -- 172.21.15.73:0/3030840555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3401b040 con 0x557883ee8880 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b257fa700 1 -- 172.21.15.73:0/3030840555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b34024070 con 0x557883ee8880 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.705+0000 7f1b257fa700 1 -- 172.21.15.73:0/3030840555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b340136b0 con 0x557883ee8880 2024-06-06T07:02:18.708 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3030840555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 msgr2=0x557883ee2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34011000 comp rx=0 tx=0).stop 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3030840555 shutdown_connections 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee7f60 0x557883ee8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883ee2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3030840555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee8880 0x557883ee2260 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3030840555 >> 172.21.15.73:0/3030840555 conn(0x557883e0f080 msgr2=0x557883edd220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3030840555 shutdown_connections 2024-06-06T07:02:18.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3030840555 wait complete. 2024-06-06T07:02:18.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.706+0000 7f1b5000fb80 1 -- start start 2024-06-06T07:02:18.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 Processor -- start 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee7f60 0x557883eeb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883eebe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557883dd3fc0 con 0x557883ee27a0 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557883e2f810 con 0x557883ee7f60 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557883eef260 con 0x557883ee8880 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54058/0 (socket says 172.21.15.73:54058) 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 -- 172.21.15.73:0/1655473207 learned_addr learned my addr 172.21.15.73:0/1655473207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b5000fb80 10 client.0 _ll_register_callbacks cb 0x557883dc95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee7f60 0x557883eeb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 -- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 msgr2=0x557883eebe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883eebe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 -- 172.21.15.73:0/1655473207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee7f60 msgr2=0x557883eeb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee7f60 0x557883eeb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.707+0000 7f1b3effd700 1 -- 172.21.15.73:0/1655473207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557883df9750 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b3effd700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1b300098e0 tx=0x7f1b300228c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7304ea8c43737057 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3009ca80 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b34024070 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1b300cdfd0 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1b300a0e30 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 10 client.15603 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557883ef9a00 con 0x557883ee27a0 2024-06-06T07:02:18.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b34024070 con 0x557883ee27a0 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1b300987d0 con 0x557883ee27a0 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 10 client.15603 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 10 client.15603 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 20 client.15603 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883efe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 4 client.15603 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.708+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557883e40d50 con 0x557883efbf70 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.709+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883efe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.709+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883efe410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4185c3c863db399e server_cookie=3adc93b8e35b1e27 in_seq=0 out_seq=0 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.709+0000 7f1b22ff5700 10 client.15603 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b22ff5700 1 -- 172.21.15.73:0/1655473207 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557883e40d50 con 0x557883efbf70 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b22ff5700 10 client.15603 handle_command_reply: tid=0 2024-06-06T07:02:18.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 client.15603 shutdown 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 msgr2=0x557883eee590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1b300098e0 tx=0x7f1b300228c0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 shutdown_connections 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883efe410 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883eebe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee27a0 0x557883eee590 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b22ff5700 0 client.15603 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/1655473207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee7f60 0x557883eeb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.710+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 >> 172.21.15.73:0/1655473207 conn(0x557883e0f080 msgr2=0x557883edc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.711+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 shutdown_connections 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.711+0000 7f1b5000fb80 1 -- 172.21.15.73:0/1655473207 wait complete. 2024-06-06T07:02:18.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.711+0000 7f1b5000fb80 20 client.15603 trim_cache size 0 max 16384 2024-06-06T07:02:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 Processor -- start 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 -- start start 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee27a0 0x557883ee2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee8880 0x557883eee140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b06a0 con 0x557883ee7f60 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b21e0 con 0x557883ee8880 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b0d70 con 0x557883ee27a0 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54072/0 (socket says 172.21.15.73:54072) 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3813032861 learned_addr learned my addr 172.21.15.73:0/3813032861 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3f7fe700 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee8880 0x557883eee140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3effd700 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee27a0 0x557883ee2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3813032861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee27a0 msgr2=0x557883ee2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee27a0 0x557883ee2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3813032861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee8880 msgr2=0x557883eee140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee8880 0x557883eee140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.738+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/3813032861 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557883eec870 con 0x557883ee7f60 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34006d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64fbee0c1bdbc5c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b24ff9700 1 -- 172.21.15.73:0/3813032861 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400ee60 con 0x557883ee7f60 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b24ff9700 1 -- 172.21.15.73:0/3813032861 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b34034070 con 0x557883ee7f60 2024-06-06T07:02:18.741 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b24ff9700 1 -- 172.21.15.73:0/3813032861 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3402fba0 con 0x557883ee7f60 2024-06-06T07:02:18.744 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3813032861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 msgr2=0x557883ee93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.739+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009fd0 tx=0x7f1b34006d90 comp rx=0 tx=0).stop 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3813032861 shutdown_connections 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee27a0 0x557883ee2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee8880 0x557883eee140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/3813032861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883ee93b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3813032861 >> 172.21.15.73:0/3813032861 conn(0x557883e0f080 msgr2=0x557883ee8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3813032861 shutdown_connections 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- 172.21.15.73:0/3813032861 wait complete. 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- start start 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 Processor -- start 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883f038b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883f04330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b1b10 con 0x557883ee7f60 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401b1440 con 0x557883ee27a0 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.740+0000 7f1b5000fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1b401ae640 con 0x557883ee8880 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b5000fb80 10 client.0 _ll_register_callbacks cb 0x557883dc95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54082/0 (socket says 172.21.15.73:54082) 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/2500735699 learned_addr learned my addr 172.21.15.73:0/2500735699 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3effd700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883f038b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3f7fe700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883f04330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 msgr2=0x557883f04330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883f04330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/2500735699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 msgr2=0x557883f038b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883f038b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 -- 172.21.15.73:0/2500735699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557883f65cb0 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b3e7fc700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1b3402f650 tx=0x7f1b3402f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72deb7752c7b4aa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b22ff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3403c020 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1b3402f820 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.741+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b34034070 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1b3402fa20 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 10 client.15609 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557883f66f60 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b34024040 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f1b3401e030 con 0x557883ee7f60 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 10 client.15609 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:18.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 10 client.15609 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 20 client.15609 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883f692e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 4 client.15609 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f1b34034070 con 0x557883efbf70 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.742+0000 7f1b3effd700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883f692e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b22ff5700 10 client.15609 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b3effd700 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883f692e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce9d91605af3cbd2 server_cookie=1eaa164c6693f8b8 in_seq=0 out_seq=0 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b22ff5700 1 -- 172.21.15.73:0/2500735699 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f1b34034070 con 0x557883efbf70 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b22ff5700 10 client.15609 handle_command_reply: tid=0 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 client.15609 shutdown 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 msgr2=0x557883f03df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1b3402f650 tx=0x7f1b3402f680 comp rx=0 tx=0).stop 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 shutdown_connections 2024-06-06T07:02:18.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x557883efbf70 0x557883f692e0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557883ee8880 0x557883f04330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557883ee27a0 0x557883f038b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 --2- 172.21.15.73:0/2500735699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557883ee7f60 0x557883f03df0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 >> 172.21.15.73:0/2500735699 conn(0x557883e0f080 msgr2=0x557883ee6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b22ff5700 0 client.15609 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 shutdown_connections 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 1 -- 172.21.15.73:0/2500735699 wait complete. 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.743+0000 7f1b5000fb80 20 client.15609 trim_cache size 0 max 16384 2024-06-06T07:02:18.749 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 msgr2=0x7f1b280a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 0x7f1b280a53f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1b40078870 tx=0x7f1b34009190 comp rx=0 tx=0).stop 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 msgr2=0x7f1b40103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1b30000c00 tx=0x7f1b30003a80 comp rx=0 tx=0).stop 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 shutdown_connections 2024-06-06T07:02:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1b280a2f30 0x7f1b280a53f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1b401073f0 0x7f1b40103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b40101380 0x7f1b40103060 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 --2- 172.21.15.73:0/3725936345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1b40101d40 0x7f1b401035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 >> 172.21.15.73:0/3725936345 conn(0x7f1b40078300 msgr2=0x7f1b401006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 shutdown_connections 2024-06-06T07:02:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:18.744+0000 7f1b4578c700 1 -- 172.21.15.73:0/3725936345 wait complete. 2024-06-06T07:02:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:19 smithi073 ceph-mon[32303]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:19.554 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:19 smithi136 ceph-mon[35901]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:19 smithi170 ceph-mon[36108]: pgmap v32: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:19.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:02:19.903 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:21 smithi073 ceph-mon[32303]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:21 smithi136 ceph-mon[35901]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:21 smithi170 ceph-mon[36108]: pgmap v33: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:23.397 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:23 smithi073 ceph-mon[32303]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:23 smithi136 ceph-mon[35901]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:23 smithi170 ceph-mon[36108]: pgmap v34: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 307 B/s wr, 9 op/s 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 -- 172.21.15.73:0/1275749954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 msgr2=0x7f8414101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1275749954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414101800 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8404009990 tx=0x7f8404009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 -- 172.21.15.73:0/1275749954 shutdown_connections 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1275749954 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 0x7f84141077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1275749954 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 0x7f841410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1275749954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414101800 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 -- 172.21.15.73:0/1275749954 >> 172.21.15.73:0/1275749954 conn(0x7f8414078330 msgr2=0x7f8414078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 -- 172.21.15.73:0/1275749954 shutdown_connections 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.542+0000 7f841b8fc700 1 -- 172.21.15.73:0/1275749954 wait complete. 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.543+0000 7f841b8fc700 1 Processor -- start 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.543+0000 7f841b8fc700 1 -- start start 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.543+0000 7f841b8fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 0x7f8414111ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 0x7f84141123c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f84141114e0 con 0x7f8414101380 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111360 con 0x7f8414101d40 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8414111660 con 0x7f84141073c0 2024-06-06T07:02:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54098/0 (socket says 172.21.15.73:54098) 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 -- 172.21.15.73:0/1596077642 learned_addr learned my addr 172.21.15.73:0/1596077642 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8418e97700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 0x7f8414111ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419e99700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 0x7f84141123c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 -- 172.21.15.73:0/1596077642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 msgr2=0x7f84141123c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 0x7f84141123c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 -- 172.21.15.73:0/1596077642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 msgr2=0x7f8414111ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 0x7f8414111ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 -- 172.21.15.73:0/1596077642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84141a7040 con 0x7f8414101380 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f8419698700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f840a7fc700 1 -- 172.21.15.73:0/1596077642 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410016070 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84141a7330 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f840a7fc700 1 -- 172.21.15.73:0/1596077642 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f841000fd40 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.544+0000 7f840a7fc700 1 -- 172.21.15.73:0/1596077642 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f841000e3b0 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.545+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84141a7840 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.547+0000 7f840a7fc700 1 -- 172.21.15.73:0/1596077642 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f8410019070 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.548+0000 7f840a7fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 0x7f84000a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.548+0000 7f840a7fc700 1 -- 172.21.15.73:0/1596077642 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f84100cefd0 con 0x7f8414101380 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.548+0000 7f8418e97700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 0x7f84000a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.549+0000 7f8418e97700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 0x7f84000a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f840400b2e0 tx=0x7f8404005960 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.550+0000 7f842617fb80 1 Processor -- start 2024-06-06T07:02:24.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.550+0000 7f842617fb80 1 -- start start 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.550+0000 7f842617fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b40a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.550+0000 7f842617fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f842617fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f842617fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556cb7a8c810 con 0x556cb7b40f60 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54100/0 (socket says 172.21.15.73:54100) 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556cb7a34720 con 0x556cb7b3af20 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 -- 172.21.15.73:0/3512222587 learned_addr learned my addr 172.21.15.73:0/3512222587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556cb7a887d0 con 0x556cb7b40640 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8419698700 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b40a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8419e99700 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 -- 172.21.15.73:0/3512222587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 msgr2=0x556cb7b40a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b40a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 -- 172.21.15.73:0/3512222587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 msgr2=0x556cb7b3b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 -- 172.21.15.73:0/3512222587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb7a2af50 con 0x556cb7b40f60 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.551+0000 7f8418e97700 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8404015000 tx=0x7f840401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404c6633ae632df0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.552+0000 7f83ff7fe700 1 -- 172.21.15.73:0/3512222587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840400be70 con 0x556cb7b40f60 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.552+0000 7f83ff7fe700 1 -- 172.21.15.73:0/3512222587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8404003b30 con 0x556cb7b40f60 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.552+0000 7f83ff7fe700 1 -- 172.21.15.73:0/3512222587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8404030b30 con 0x556cb7b40f60 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 msgr2=0x556cb7b3a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f8404015000 tx=0x7f840401f000 comp rx=0 tx=0).stop 2024-06-06T07:02:24.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 shutdown_connections 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b40a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 --2- 172.21.15.73:0/3512222587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b3a9e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 >> 172.21.15.73:0/3512222587 conn(0x556cb7a65cc0 msgr2=0x556cb7b35850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 shutdown_connections 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- 172.21.15.73:0/3512222587 wait complete. 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.553+0000 7f842617fb80 1 -- start start 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.554+0000 7f842617fb80 1 Processor -- start 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.554+0000 7f842617fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b3af20 0x556cb7b46d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.554+0000 7f842617fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b44010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.554+0000 7f842617fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f842617fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556cb7b47970 con 0x556cb7b40f60 2024-06-06T07:02:24.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54110/0 (socket says 172.21.15.73:54110) 2024-06-06T07:02:24.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 -- 172.21.15.73:0/2412142766 learned_addr learned my addr 172.21.15.73:0/2412142766 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:24.376 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8418e97700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b44010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419698700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b3af20 0x556cb7b46d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556cb7a2cbf0 con 0x556cb7b40640 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556cb7b47af0 con 0x556cb7b3af20 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f842617fb80 10 client.0 _ll_register_callbacks cb 0x556cb7a26920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 -- 172.21.15.73:0/2412142766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b3af20 msgr2=0x556cb7b46d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b3af20 0x556cb7b46d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 -- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 msgr2=0x556cb7b44010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b44010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8419e99700 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb79db8d0 con 0x556cb7b40f60 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8404003b30 con 0x556cb7b40f60 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.555+0000 7f8418e97700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b44010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f8419e99700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f840c011f60 tx=0x7f840c014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82da3fc981bcde83 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f83fcff9700 10 client.15621 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f83fcff9700 1 -- 172.21.15.73:0/2412142766 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c014ba0 con 0x556cb7b40f60 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f83fcff9700 1 -- 172.21.15.73:0/2412142766 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f840c014d40 con 0x556cb7b40f60 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f83fcff9700 1 -- 172.21.15.73:0/2412142766 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f840c003a00 con 0x556cb7b40f60 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f842617fb80 10 client.15621 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:24.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=32}) -- 0x556cb7b522d0 con 0x556cb7b40f60 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.556+0000 7f83fcff9700 1 -- 172.21.15.73:0/2412142766 <== mon.0 v2:172.21.15.73:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f840c003ba0 con 0x556cb7b40f60 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 10 client.15621 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 10 client.15621 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 20 client.15621 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7b54800 0x556cb7b56bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 4 client.15621 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cb7a99d10 con 0x556cb7b54800 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.557+0000 7f8419698700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7b54800 0x556cb7b56bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.558+0000 7f83fcff9700 10 client.15621 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.558+0000 7f8419698700 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7b54800 0x556cb7b56bf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9bb1586ac1bbdda server_cookie=fef38209cdf28c8b in_seq=0 out_seq=0 2024-06-06T07:02:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.558+0000 7f83fcff9700 1 -- 172.21.15.73:0/2412142766 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556cb7a99d10 con 0x556cb7b54800 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.558+0000 7f83fcff9700 10 client.15621 handle_command_reply: tid=0 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 client.15621 shutdown 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 msgr2=0x556cb7b44550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f840c011f60 tx=0x7f840c014600 comp rx=0 tx=0).stop 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 shutdown_connections 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7b54800 0x556cb7b56bf0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b3af20 0x556cb7b46d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b44550 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 --2- 172.21.15.73:0/2412142766 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b44010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 >> 172.21.15.73:0/2412142766 conn(0x556cb7a65cc0 msgr2=0x556cb7b355f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.380 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f83fcff9700 0 client.15621 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 shutdown_connections 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 1 -- 172.21.15.73:0/2412142766 wait complete. 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.559+0000 7f842617fb80 20 client.15621 trim_cache size 0 max 16384 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.587+0000 7f842617fb80 1 Processor -- start 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.587+0000 7f842617fb80 1 -- start start 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b41c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b0770 con 0x556cb7b40f60 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b22b0 con 0x556cb7b3af20 2024-06-06T07:02:24.381 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f842617fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b0e40 con 0x556cb7b40640 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54126/0 (socket says 172.21.15.73:54126) 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 -- 172.21.15.73:0/856311936 learned_addr learned my addr 172.21.15.73:0/856311936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419698700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8418e97700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b41c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 -- 172.21.15.73:0/856311936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 msgr2=0x556cb7b41c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b41c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 -- 172.21.15.73:0/856311936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 msgr2=0x556cb7b3a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.382 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 -- 172.21.15.73:0/856311936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cb7b425a0 con 0x556cb7b40f60 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419698700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.588+0000 7f8419e99700 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f840c014640 tx=0x7f840c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4b29fb24384110 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.589+0000 7f83feffd700 1 -- 172.21.15.73:0/856311936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c003ba0 con 0x556cb7b40f60 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.589+0000 7f83feffd700 1 -- 172.21.15.73:0/856311936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8404003b30 con 0x556cb7b40f60 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.589+0000 7f83feffd700 1 -- 172.21.15.73:0/856311936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840c0041c0 con 0x556cb7b40f60 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- 172.21.15.73:0/856311936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 msgr2=0x556cb7b445a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f840c014640 tx=0x7f840c018000 comp rx=0 tx=0).stop 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- 172.21.15.73:0/856311936 shutdown_connections 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b40f60 0x556cb7b445a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.383 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b3af20 0x556cb7b3a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 --2- 172.21.15.73:0/856311936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40640 0x556cb7b41c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- 172.21.15.73:0/856311936 >> 172.21.15.73:0/856311936 conn(0x556cb7a65cc0 msgr2=0x556cb7b2ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- 172.21.15.73:0/856311936 shutdown_connections 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- 172.21.15.73:0/856311936 wait complete. 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.590+0000 7f842617fb80 1 -- start start 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 Processor -- start 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b5bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40f60 0x556cb7b5c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b1be0 con 0x556cb7b3af20 2024-06-06T07:02:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f84141b1510 con 0x556cb7b40640 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f84141ae710 con 0x556cb7b40f60 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f842617fb80 10 client.0 _ll_register_callbacks cb 0x556cb7a26920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f8419698700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f8419698700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54136/0 (socket says 172.21.15.73:54136) 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.591+0000 7f8419698700 1 -- 172.21.15.73:0/1647336303 learned_addr learned my addr 172.21.15.73:0/1647336303 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419e99700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40f60 0x556cb7b5c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8418e97700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b5bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 -- 172.21.15.73:0/1647336303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40f60 msgr2=0x556cb7b5c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40f60 0x556cb7b5c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 -- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 msgr2=0x556cb7b5bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b5bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 -- 172.21.15.73:0/1647336303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cb7b5c730 con 0x556cb7b3af20 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8418e97700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f8419698700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84100972f0 tx=0x7f8410010680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba9cdfddb7ea1cc8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f83fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100a3b00 con 0x556cb7b3af20 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f8410010710 con 0x556cb7b3af20 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8404003b30 con 0x556cb7b3af20 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f84100a3670 con 0x556cb7b3af20 2024-06-06T07:02:24.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f842617fb80 10 client.15627 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556cb7bbf210 con 0x556cb7b3af20 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.592+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8404003b30 con 0x556cb7b3af20 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f841009a2e0 con 0x556cb7b3af20 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 10 client.15627 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 20 client.15627 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7bbf8f0 0x556cb7bc1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 4 client.15627 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cb7a94290 con 0x556cb7bbf8f0 2024-06-06T07:02:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.593+0000 7f8418e97700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7bbf8f0 0x556cb7bc1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f83fcff9700 10 client.15627 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f8418e97700 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7bbf8f0 0x556cb7bc1d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44388ff22a03be87 server_cookie=184a31bcfce7ba8 in_seq=0 out_seq=0 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f83fcff9700 1 -- 172.21.15.73:0/1647336303 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cb7a94290 con 0x556cb7bbf8f0 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f83fcff9700 10 client.15627 handle_command_reply: tid=0 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 client.15627 shutdown 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 msgr2=0x556cb7b4d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f84100972f0 tx=0x7f8410010680 comp rx=0 tx=0).stop 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 shutdown_connections 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x556cb7bbf8f0 0x556cb7bc1d10 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f83fcff9700 0 client.15627 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cb7b40f60 0x556cb7b5c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cb7b3af20 0x556cb7b4d3a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 --2- 172.21.15.73:0/1647336303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cb7b40640 0x556cb7b5bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.594+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 >> 172.21.15.73:0/1647336303 conn(0x556cb7a65cc0 msgr2=0x556cb7b2f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.595+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 shutdown_connections 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.595+0000 7f842617fb80 1 -- 172.21.15.73:0/1647336303 wait complete. 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.595+0000 7f842617fb80 20 client.15627 trim_cache size 0 max 16384 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 msgr2=0x7f84000a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 0x7f84000a5370 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f840400b2e0 tx=0x7f8404005960 comp rx=0 tx=0).stop 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 msgr2=0x7f8414113960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000dc60 comp rx=0 tx=0).stop 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 shutdown_connections 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f84000a2eb0 0x7f84000a5370 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f84141073c0 0x7f84141123c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8414101380 0x7f8414113960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 --2- 172.21.15.73:0/1596077642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8414101d40 0x7f8414111ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 >> 172.21.15.73:0/1596077642 conn(0x7f8414078330 msgr2=0x7f84140ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 shutdown_connections 2024-06-06T07:02:24.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:23.596+0000 7f841b8fc700 1 -- 172.21.15.73:0/1596077642 wait complete. 2024-06-06T07:02:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:25 smithi073 ceph-mon[32303]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:25 smithi136 ceph-mon[35901]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:25 smithi170 ceph-mon[36108]: pgmap v35: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:26.820 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:02:27.080 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:27.519 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:27 smithi073 ceph-mon[32303]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:27 smithi136 ceph-mon[35901]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:27 smithi170 ceph-mon[36108]: pgmap v36: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 307 B/s wr, 10 op/s 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.992+0000 7f71c4e9c700 1 -- 172.21.15.73:0/35832649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0106b90 msgr2=0x7f71c0106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.992+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/35832649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0106b90 0x7f71c0106f70 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f71b000aaa0 tx=0x7f71b000adb0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.993+0000 7f71c4e9c700 1 -- 172.21.15.73:0/35832649 shutdown_connections 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.993+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/35832649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0101510 0x7f71c010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.993+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/35832649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0106b90 0x7f71c0106f70 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.993+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/35832649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0100b50 0x7f71c0100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.993+0000 7f71c4e9c700 1 -- 172.21.15.73:0/35832649 >> 172.21.15.73:0/35832649 conn(0x7f71c0077a00 msgr2=0x7f71c00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.994+0000 7f71c4e9c700 1 -- 172.21.15.73:0/35832649 shutdown_connections 2024-06-06T07:02:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.994+0000 7f71c4e9c700 1 -- 172.21.15.73:0/35832649 wait complete. 2024-06-06T07:02:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 Processor -- start 2024-06-06T07:02:28.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 -- start start 2024-06-06T07:02:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 0x7f71c019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:28.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0106b90 0x7f71c01989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01104b0 con 0x7f71c0101510 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0110330 con 0x7f71c0106b90 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71c4e9c700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71c0110630 con 0x7f71c0100b50 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35490/0 (socket says 172.21.15.73:35490) 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71bdd9b700 1 -- 172.21.15.73:0/1105352631 learned_addr learned my addr 172.21.15.73:0/1105352631 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.995+0000 7f71be59c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 0x7f71c019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:28.998 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71bed9d700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0106b90 0x7f71c01989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 -- 172.21.15.73:0/1105352631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0106b90 msgr2=0x7f71c01989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0106b90 0x7f71c01989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 -- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 msgr2=0x7f71c019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 -- 172.21.15.73:0/1105352631 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c0199120 con 0x7f71c0100b50 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71be59c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 0x7f71c019e7b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71b0000c00 tx=0x7f71b0003930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:28.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71af7fe700 1 -- 172.21.15.73:0/1105352631 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0023020 con 0x7f71c0100b50 2024-06-06T07:02:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c01993b0 con 0x7f71c0100b50 2024-06-06T07:02:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c01966c0 con 0x7f71c0100b50 2024-06-06T07:02:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.996+0000 7f71af7fe700 1 -- 172.21.15.73:0/1105352631 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71b000e450 con 0x7f71c0100b50 2024-06-06T07:02:29.000 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:28.997+0000 7f71af7fe700 1 -- 172.21.15.73:0/1105352631 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0016680 con 0x7f71c0100b50 2024-06-06T07:02:29.001 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.000+0000 7f71af7fe700 1 -- 172.21.15.73:0/1105352631 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f71b0026020 con 0x7f71c0100b50 2024-06-06T07:02:29.002 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.000+0000 7f71af7fe700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 0x7f71a80a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.000+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 0x7f71a80a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71af7fe700 1 -- 172.21.15.73:0/1105352631 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f71b00d0b90 con 0x7f71c0100b50 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71cf71fb80 1 Processor -- start 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 0x7f71a80a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005d90 tx=0x7f71b4005d20 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71cf71fb80 1 -- start start 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc81e640 0x55b0fc81ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.003 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.001+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc8189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc76a810 con 0x55b0fc818f20 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc712720 con 0x55b0fc81e640 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc7667d0 con 0x55b0fc81ef60 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35496/0 (socket says 172.21.15.73:35496) 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 -- 172.21.15.73:0/1245026682 learned_addr learned my addr 172.21.15.73:0/1245026682 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc8189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71be59c700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc81e640 0x55b0fc81ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 -- 172.21.15.73:0/1245026682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 msgr2=0x55b0fc8189e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc8189e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.005 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 -- 172.21.15.73:0/1245026682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc81e640 msgr2=0x55b0fc81ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc81e640 0x55b0fc81ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bed9d700 1 -- 172.21.15.73:0/1245026682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fc708f50 con 0x55b0fc818f20 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.002+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc8189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.003+0000 7f71bed9d700 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003fa0 tx=0x7f71b8009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b1cfef8ed24b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.003+0000 7f71acff9700 1 -- 172.21.15.73:0/1245026682 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8004bb0 con 0x55b0fc818f20 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.003+0000 7f71acff9700 1 -- 172.21.15.73:0/1245026682 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71b8004d50 con 0x55b0fc818f20 2024-06-06T07:02:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.003+0000 7f71acff9700 1 -- 172.21.15.73:0/1245026682 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8007d20 con 0x55b0fc818f20 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- 172.21.15.73:0/1245026682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 msgr2=0x55b0fc819340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f71b8003fa0 tx=0x7f71b8009ba0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- 172.21.15.73:0/1245026682 shutdown_connections 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc818f20 0x55b0fc819340 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc81e640 0x55b0fc81ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/1245026682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc8189e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- 172.21.15.73:0/1245026682 >> 172.21.15.73:0/1245026682 conn(0x55b0fc743cc0 msgr2=0x55b0fc813850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:29.009 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- 172.21.15.73:0/1245026682 shutdown_connections 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- 172.21.15.73:0/1245026682 wait complete. 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 -- start start 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.004+0000 7f71cf71fb80 1 Processor -- start 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc822010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc822550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc8259d0 con 0x55b0fc81ef60 2024-06-06T07:02:29.010 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc70abf0 con 0x55b0fc818f20 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b0fc825b50 con 0x55b0fc81e640 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56976/0 (socket says 172.21.15.73:56976) 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 -- 172.21.15.73:0/3517513697 learned_addr learned my addr 172.21.15.73:0/3517513697 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc822010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.013 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71bed9d700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc822550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 -- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 msgr2=0x55b0fc822010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc822010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 -- 172.21.15.73:0/3517513697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 msgr2=0x55b0fc822550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc822550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.005+0000 7f71be59c700 1 -- 172.21.15.73:0/3517513697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fc6b98d0 con 0x55b0fc818f20 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71be59c700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71b00995e0 tx=0x7f71b009d150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975da9962edd6fd0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 10 client.0 _ll_register_callbacks cb 0x55b0fc704920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b8004d50 con 0x55b0fc818f20 2024-06-06T07:02:29.014 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3517513697 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005c00 con 0x55b0fc818f20 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3517513697 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f71b009c060 con 0x55b0fc818f20 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3517513697 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71b009c260 con 0x55b0fc818f20 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 10 client.25133 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55b0fc82feb0 con 0x55b0fc818f20 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3517513697 <== mon.2 v2:172.21.15.136:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f71b000b040 con 0x55b0fc818f20 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 10 client.25133 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 10 client.25133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 20 client.25133 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.006+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc832730 0x55b0fc834b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.007+0000 7f71cf71fb80 4 client.25133 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:29.015 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.007+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f71b8004d50 con 0x55b0fc832730 2024-06-06T07:02:29.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.007+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc832730 0x55b0fc834b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.007+0000 7f71a5ffb700 10 client.25133 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:29.016 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.007+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc832730 0x55b0fc834b90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dbbefc69c062a689 server_cookie=7d2fd91aabef5b76 in_seq=0 out_seq=0 2024-06-06T07:02:29.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3517513697 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f71b8004d50 con 0x55b0fc832730 2024-06-06T07:02:29.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71a5ffb700 10 client.25133 handle_command_reply: tid=0 2024-06-06T07:02:29.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 client.25133 shutdown 2024-06-06T07:02:29.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 msgr2=0x55b0fc824d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.017 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71b00995e0 tx=0x7f71b009d150 comp rx=0 tx=0).stop 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 shutdown_connections 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc832730 0x55b0fc834b90 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc822550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc824d10 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3517513697 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc822010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 >> 172.21.15.73:0/3517513697 conn(0x55b0fc743cc0 msgr2=0x55b0fc8179d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.008+0000 7f71a5ffb700 0 client.25133 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.009+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 shutdown_connections 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.009+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3517513697 wait complete. 2024-06-06T07:02:29.018 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.009+0000 7f71cf71fb80 20 client.25133 trim_cache size 0 max 16384 2024-06-06T07:02:29.025 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 Processor -- start 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 -- start start 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8189e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc80ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b03b0 con 0x55b0fc81e640 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b1ef0 con 0x55b0fc818f20 2024-06-06T07:02:29.026 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b0a80 con 0x55b0fc81ef60 2024-06-06T07:02:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.024+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35510/0 (socket says 172.21.15.73:35510) 2024-06-06T07:02:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71bdd9b700 1 -- 172.21.15.73:0/124318320 learned_addr learned my addr 172.21.15.73:0/124318320 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:29.027 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8189e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.028 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71bed9d700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc80ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 -- 172.21.15.73:0/124318320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 msgr2=0x55b0fc80ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc80ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 -- 172.21.15.73:0/124318320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 msgr2=0x55b0fc81fb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 -- 172.21.15.73:0/124318320 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0fc824950 con 0x55b0fc818f20 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71be59c700 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8189e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71b009c910 tx=0x7f71b0029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=365a3c84630e06da server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71a7fff700 1 -- 172.21.15.73:0/124318320 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b0005c00 con 0x55b0fc818f20 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.025+0000 7f71a7fff700 1 -- 172.21.15.73:0/124318320 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71b009cc10 con 0x55b0fc818f20 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71a7fff700 1 -- 172.21.15.73:0/124318320 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b00a3420 con 0x55b0fc818f20 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 -- 172.21.15.73:0/124318320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 msgr2=0x55b0fc8189e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8189e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f71b009c910 tx=0x7f71b0029000 comp rx=0 tx=0).stop 2024-06-06T07:02:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 -- 172.21.15.73:0/124318320 shutdown_connections 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81ef60 0x55b0fc80ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8189e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/124318320 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81e640 0x55b0fc81fb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.026+0000 7f71cf71fb80 1 -- 172.21.15.73:0/124318320 >> 172.21.15.73:0/124318320 conn(0x55b0fc743cc0 msgr2=0x55b0fc81f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- 172.21.15.73:0/124318320 shutdown_connections 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- 172.21.15.73:0/124318320 wait complete. 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- start start 2024-06-06T07:02:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 Processor -- start 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8263e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc89b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b1820 con 0x55b0fc81ef60 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01b1150 con 0x55b0fc818f20 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71c01ae350 con 0x55b0fc81e640 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.027+0000 7f71cf71fb80 10 client.0 _ll_register_callbacks cb 0x55b0fc704920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71be59c700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8263e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:52294/0 (socket says 172.21.15.73:52294) 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 -- 172.21.15.73:0/3136010392 learned_addr learned my addr 172.21.15.73:0/3136010392 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bed9d700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc89b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 -- 172.21.15.73:0/3136010392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 msgr2=0x55b0fc8263e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8263e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 -- 172.21.15.73:0/3136010392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 msgr2=0x55b0fc89b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc89b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 -- 172.21.15.73:0/3136010392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0fc89b9a0 con 0x55b0fc81e640 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71bdd9b700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71b4007ac0 tx=0x7f71b4004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e417b14f0e9869f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.028+0000 7f71a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b4004020 con 0x55b0fc81e640 2024-06-06T07:02:29.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f71b40041c0 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71b009cc10 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f71b40043c0 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71cf71fb80 10 client.25213 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.029+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b0fc89cc60 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71b009cc10 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f71b402b500 con 0x55b0fc81e640 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 10 client.25213 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 10 client.25213 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 20 client.25213 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc89d390 0x55b0fc89f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 4 client.25213 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0fc777d10 con 0x55b0fc89d390 2024-06-06T07:02:29.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.030+0000 7f71be59c700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc89d390 0x55b0fc89f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71a5ffb700 10 client.25213 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71be59c700 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc89d390 0x55b0fc89f7b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f109da9fb7bf2a8 server_cookie=661ce24a4c26f860 in_seq=0 out_seq=0 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71a5ffb700 1 -- 172.21.15.73:0/3136010392 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0fc777d10 con 0x55b0fc89d390 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71a5ffb700 10 client.25213 handle_command_reply: tid=0 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 client.25213 shutdown 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 msgr2=0x55b0fc826920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f71b4007ac0 tx=0x7f71b4004750 comp rx=0 tx=0).stop 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 shutdown_connections 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55b0fc89d390 0x55b0fc89f7b0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b0fc81ef60 0x55b0fc89b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71a5ffb700 0 client.25213 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b0fc818f20 0x55b0fc8263e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 --2- 172.21.15.73:0/3136010392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b0fc81e640 0x55b0fc826920 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 >> 172.21.15.73:0/3136010392 conn(0x55b0fc743cc0 msgr2=0x55b0fc81f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.031+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 shutdown_connections 2024-06-06T07:02:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.032+0000 7f71cf71fb80 1 -- 172.21.15.73:0/3136010392 wait complete. 2024-06-06T07:02:29.037 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.032+0000 7f71cf71fb80 20 client.25213 trim_cache size 0 max 16384 2024-06-06T07:02:29.037 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.032+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 msgr2=0x7f71a80a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 0x7f71a80a52d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f71b4005d90 tx=0x7f71b4005d20 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 msgr2=0x7f71c019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 0x7f71c019e7b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f71b0000c00 tx=0x7f71b0003930 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 shutdown_connections 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71a80a2e10 0x7f71a80a52d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71c0100b50 0x7f71c019e7b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71c0106b90 0x7f71c01989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 --2- 172.21.15.73:0/1105352631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71c0101510 0x7f71c019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 >> 172.21.15.73:0/1105352631 conn(0x7f71c0077a00 msgr2=0x7f71c0103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:29.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 shutdown_connections 2024-06-06T07:02:29.039 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:29.033+0000 7f71c4e9c700 1 -- 172.21.15.73:0/1105352631 wait complete. 2024-06-06T07:02:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:29 smithi073 ceph-mon[32303]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 289 B/s wr, 14 op/s 2024-06-06T07:02:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:29 smithi136 ceph-mon[35901]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 289 B/s wr, 14 op/s 2024-06-06T07:02:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:29 smithi170 ceph-mon[36108]: pgmap v37: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 289 B/s wr, 14 op/s 2024-06-06T07:02:29.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:02:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.039+0000 7f1544aae640 1 -- 172.21.15.73:0/3063565330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 msgr2=0x7f1540100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.039+0000 7f1544aae640 1 --2- 172.21.15.73:0/3063565330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f1540100d50 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f153402a420 comp rx=0 tx=0).stop 2024-06-06T07:02:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 -- 172.21.15.73:0/3063565330 shutdown_connections 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 --2- 172.21.15.73:0/3063565330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540101290 0x7f154010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 --2- 172.21.15.73:0/3063565330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f1540100d50 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 --2- 172.21.15.73:0/3063565330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540106890 0x7f1540106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 -- 172.21.15.73:0/3063565330 >> 172.21.15.73:0/3063565330 conn(0x7f1540077a20 msgr2=0x7f1540077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 -- 172.21.15.73:0/3063565330 shutdown_connections 2024-06-06T07:02:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.040+0000 7f1544aae640 1 -- 172.21.15.73:0/3063565330 wait complete. 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.041+0000 7f1544aae640 1 Processor -- start 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.041+0000 7f1544aae640 1 -- start start 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.041+0000 7f1544aae640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.041+0000 7f1544aae640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540101290 0x7f154019be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.041+0000 7f1544aae640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540106890 0x7f1540195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35532/0 (socket says 172.21.15.73:35532) 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 -- 172.21.15.73:0/1945842368 learned_addr learned my addr 172.21.15.73:0/1945842368 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153dd9b640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540101290 0x7f154019be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153ed9d640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540106890 0x7f1540195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f154010fa90 con 0x7f15401008d0 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f154010f910 con 0x7f1540101290 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f154010fc10 con 0x7f1540106890 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 -- 172.21.15.73:0/1945842368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540106890 msgr2=0x7f1540195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540106890 0x7f1540195ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 -- 172.21.15.73:0/1945842368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540101290 msgr2=0x7f154019be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540101290 0x7f154019be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.042+0000 7f153e59c640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1540196250 con 0x7f15401008d0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f153e59c640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f153000d950 tx=0x7f153000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f152f7fe640 1 -- 172.21.15.73:0/1945842368 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530017070 con 0x7f15401008d0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f152f7fe640 1 -- 172.21.15.73:0/1945842368 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1530009c30 con 0x7f15401008d0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f152f7fe640 1 -- 172.21.15.73:0/1945842368 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153000f620 con 0x7f15401008d0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1540196540 con 0x7f15401008d0 2024-06-06T07:02:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.043+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15401a27e0 con 0x7f15401008d0 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.046+0000 7f152f7fe640 1 -- 172.21.15.73:0/1945842368 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f15300109a0 con 0x7f15401008d0 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 Processor -- start 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 -- start start 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 0x5648d4c66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5648d4bc04f0 con 0x5648d4c67350 2024-06-06T07:02:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648d4b5e870 con 0x5648d4c5ca90 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f15477b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5648d4bbc4b0 con 0x5648d4c5c170 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f152f7fe640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 0x7f15280a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.047+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35540/0 (socket says 172.21.15.73:35540) 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 -- 172.21.15.73:0/3985524330 learned_addr learned my addr 172.21.15.73:0/3985524330 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f152f7fe640 1 -- 172.21.15.73:0/1945842368 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f15300cf5a0 con 0x7f15401008d0 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153e59c640 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 0x5648d4c66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153ed9d640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 0x7f15280a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 -- 172.21.15.73:0/3985524330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 msgr2=0x5648d4c5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 -- 172.21.15.73:0/3985524330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 msgr2=0x5648d4c66e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 0x5648d4c66e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 -- 172.21.15.73:0/3985524330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d4b7a9b0 con 0x5648d4c67350 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153e59c640 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 0x5648d4c66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153dd9b640 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f1534000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61de461f8824103d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.048+0000 7f153ed9d640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 0x7f15280a5400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1538005fd0 tx=0x7f153800b710 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.049+0000 7f152cff9640 1 -- 172.21.15.73:0/3985524330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534035040 con 0x5648d4c67350 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.049+0000 7f152cff9640 1 -- 172.21.15.73:0/3985524330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15340047a0 con 0x5648d4c67350 2024-06-06T07:02:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.049+0000 7f152cff9640 1 -- 172.21.15.73:0/3985524330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534003850 con 0x5648d4c67350 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- 172.21.15.73:0/3985524330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 msgr2=0x5648d4c69740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1534009930 tx=0x7f1534000bc0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- 172.21.15.73:0/3985524330 shutdown_connections 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c67350 0x5648d4c69740 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5ca90 0x5648d4c66e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 --2- 172.21.15.73:0/3985524330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- 172.21.15.73:0/3985524330 >> 172.21.15.73:0/3985524330 conn(0x5648d4c58680 msgr2=0x5648d4c58a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- 172.21.15.73:0/3985524330 shutdown_connections 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- 172.21.15.73:0/3985524330 wait complete. 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 -- start start 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.050+0000 7f15477b6740 1 Processor -- start 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5c170 0x5648d4c642f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 0x5648d4c64d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5648d4c69f20 con 0x5648d4c5ca90 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5648d4b49ae0 con 0x5648d4c5c170 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5648d4c6a0a0 con 0x5648d4c67350 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f15477b6740 10 client.0 _ll_register_callbacks cb 0x5648d4b829b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35554/0 (socket says 172.21.15.73:35554) 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 -- 172.21.15.73:0/325803553 learned_addr learned my addr 172.21.15.73:0/325803553 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153e59c640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5c170 0x5648d4c642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153ed9d640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 0x5648d4c64d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 -- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 msgr2=0x5648d4c64d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 0x5648d4c64d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 -- 172.21.15.73:0/325803553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5c170 msgr2=0x5648d4c642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5c170 0x5648d4c642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 -- 172.21.15.73:0/325803553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d48d4210 con 0x5648d4c5ca90 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153ed9d640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 0x5648d4c64d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f153dd9b640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1534004770 tx=0x7f1534005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55db3e26022ec7a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.051+0000 7f1525ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534035040 con 0x5648d4c5ca90 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1534005230 con 0x5648d4c5ca90 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15340047a0 con 0x5648d4c5ca90 2024-06-06T07:02:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1534009520 con 0x5648d4c5ca90 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 10 client.15657 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5648d49a0620 con 0x5648d4c5ca90 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15340047a0 con 0x5648d4c5ca90 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f15340314a0 con 0x5648d4c5ca90 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 10 client.15657 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 10 client.15657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 20 client.15657 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4c7fb30 0x5648d4c81f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 4 client.15657 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.052+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648d4bd3f10 con 0x5648d4c7fb30 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.053+0000 7f153e59c640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4c7fb30 0x5648d4c81f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.053+0000 7f1525ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.053+0000 7f153e59c640 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4c7fb30 0x5648d4c81f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2779b860b2bef1c server_cookie=8fbf169055cce38 in_seq=0 out_seq=0 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f1525ffb640 1 -- 172.21.15.73:0/325803553 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5648d4bd3f10 con 0x5648d4c7fb30 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f1525ffb640 10 client.15657 handle_command_reply: tid=0 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 client.15657 shutdown 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 msgr2=0x5648d4c64830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1534004770 tx=0x7f1534005fd0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 shutdown_connections 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4c7fb30 0x5648d4c81f20 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c67350 0x5648d4c64d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c64830 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 --2- 172.21.15.73:0/325803553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c5c170 0x5648d4c642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 >> 172.21.15.73:0/325803553 conn(0x5648d4c58680 msgr2=0x5648d4c5e3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f1525ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 shutdown_connections 2024-06-06T07:02:30.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.054+0000 7f15477b6740 1 -- 172.21.15.73:0/325803553 wait complete. 2024-06-06T07:02:30.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.055+0000 7f15477b6740 20 client.15657 trim_cache size 0 max 16384 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.069+0000 7f15477b6740 1 Processor -- start 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 -- start start 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c57f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c649c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15401ac900 con 0x5648d4c5ca90 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15401ae440 con 0x5648d4c67350 2024-06-06T07:02:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f15477b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15401acfd0 con 0x5648d4c5c170 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35558/0 (socket says 172.21.15.73:35558) 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 -- 172.21.15.73:0/2369146167 learned_addr learned my addr 172.21.15.73:0/2369146167 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153e59c640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c57f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153ed9d640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c649c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 -- 172.21.15.73:0/2369146167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 msgr2=0x5648d4c57f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c57f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 -- 172.21.15.73:0/2369146167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 msgr2=0x5648d4c649c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c649c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153dd9b640 1 -- 172.21.15.73:0/2369146167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648d4c69ab0 con 0x5648d4c5ca90 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.070+0000 7f153e59c640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c57f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.071+0000 7f153dd9b640 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1534031a80 tx=0x7f1534005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df32dde38c48e9a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.071+0000 7f1527fff640 1 -- 172.21.15.73:0/2369146167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534035040 con 0x5648d4c5ca90 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.071+0000 7f1527fff640 1 -- 172.21.15.73:0/2369146167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15340314a0 con 0x5648d4c5ca90 2024-06-06T07:02:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.071+0000 7f1527fff640 1 -- 172.21.15.73:0/2369146167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1534005490 con 0x5648d4c5ca90 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- 172.21.15.73:0/2369146167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 msgr2=0x5648d4c5eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1534031a80 tx=0x7f1534005900 comp rx=0 tx=0).stop 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- 172.21.15.73:0/2369146167 shutdown_connections 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c649c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5ca90 0x5648d4c5eb30 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- 172.21.15.73:0/2369146167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5c170 0x5648d4c57f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- 172.21.15.73:0/2369146167 >> 172.21.15.73:0/2369146167 conn(0x5648d4c58680 msgr2=0x5648d4c642b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- 172.21.15.73:0/2369146167 shutdown_connections 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- 172.21.15.73:0/2369146167 wait complete. 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- start start 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 Processor -- start 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5ca90 0x5648d4c733f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c73930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15401add70 con 0x5648d4c5c170 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f15401ad6a0 con 0x5648d4c67350 2024-06-06T07:02:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f15401aa8e0 con 0x5648d4c5ca90 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.072+0000 7f15477b6740 10 client.0 _ll_register_callbacks cb 0x7f153000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35574/0 (socket says 172.21.15.73:35574) 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 -- 172.21.15.73:0/1189546484 learned_addr learned my addr 172.21.15.73:0/1189546484 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153ed9d640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c73930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153dd9b640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5ca90 0x5648d4c733f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 -- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5ca90 msgr2=0x5648d4c733f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5ca90 0x5648d4c733f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 -- 172.21.15.73:0/1189546484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 msgr2=0x5648d4c73930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c73930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 -- 172.21.15.73:0/1189546484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648d4c73ef0 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f153e59c640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f15300ccf40 tx=0x7f15300cf990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c573a3a343e0de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f1525ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1530020040 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f15300a3ca0 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.073+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15340314a0 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f153009b570 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 10 client.15663 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5648d4cea470 con 0x5648d4c5c170 2024-06-06T07:02:30.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15300a3ea0 con 0x5648d4c5c170 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f15300cc050 con 0x5648d4c5c170 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 10 client.15663 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 20 client.15663 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4ceab70 0x5648d4cecf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 4 client.15663 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f15340314a0 con 0x5648d4ceab70 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.074+0000 7f153dd9b640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4ceab70 0x5648d4cecf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.075+0000 7f153dd9b640 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4ceab70 0x5648d4cecf90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e315081fe74032ab server_cookie=ccf6ddcb12de6069 in_seq=0 out_seq=0 2024-06-06T07:02:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.075+0000 7f1525ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.078+0000 7f1525ffb640 1 -- 172.21.15.73:0/1189546484 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f15340314a0 con 0x5648d4ceab70 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f1525ffb640 10 client.15663 handle_command_reply: tid=0 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 client.15663 shutdown 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 msgr2=0x5648d4c78080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f15300ccf40 tx=0x7f15300cf990 comp rx=0 tx=0).stop 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 shutdown_connections 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5648d4ceab70 0x5648d4cecf90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f1525ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5648d4c67350 0x5648d4c73930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5648d4c5ca90 0x5648d4c733f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 --2- 172.21.15.73:0/1189546484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5648d4c5c170 0x5648d4c78080 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 >> 172.21.15.73:0/1189546484 conn(0x5648d4c58680 msgr2=0x5648d4c7b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 shutdown_connections 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.079+0000 7f15477b6740 1 -- 172.21.15.73:0/1189546484 wait complete. 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f15477b6740 20 client.15663 trim_cache size 0 max 16384 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:02:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:02:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 msgr2=0x7f15280a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 0x7f15280a5400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1538005fd0 tx=0x7f153800b710 comp rx=0 tx=0).stop 2024-06-06T07:02:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 msgr2=0x7f154019b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f153000d950 tx=0x7f153000de10 comp rx=0 tx=0).stop 2024-06-06T07:02:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 shutdown_connections 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f15280a2f40 0x7f15280a5400 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1540106890 0x7f1540195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1540101290 0x7f154019be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 --2- 172.21.15.73:0/1945842368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15401008d0 0x7f154019b920 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.080+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 >> 172.21.15.73:0/1945842368 conn(0x7f1540077a20 msgr2=0x7f1540103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.081+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 shutdown_connections 2024-06-06T07:02:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:30.081+0000 7f1544aae640 1 -- 172.21.15.73:0/1945842368 wait complete. 2024-06-06T07:02:30.198 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:31.461 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:31 smithi073 ceph-mon[32303]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:31 smithi136 ceph-mon[35901]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:31 smithi170 ceph-mon[36108]: pgmap v38: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 -- 172.21.15.73:0/1917796560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8108510 msgr2=0x7fc7b81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 --2- 172.21.15.73:0/1917796560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8108510 0x7fc7b81088f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac009990 tx=0x7fc7ac009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 -- 172.21.15.73:0/1917796560 shutdown_connections 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 --2- 172.21.15.73:0/1917796560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8101440 0x7fc7b810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 --2- 172.21.15.73:0/1917796560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8108510 0x7fc7b81088f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 --2- 172.21.15.73:0/1917796560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8100a80 0x7fc7b8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 -- 172.21.15.73:0/1917796560 >> 172.21.15.73:0/1917796560 conn(0x7fc7b8077a00 msgr2=0x7fc7b80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 -- 172.21.15.73:0/1917796560 shutdown_connections 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.064+0000 7fc7beb79700 1 -- 172.21.15.73:0/1917796560 wait complete. 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 Processor -- start 2024-06-06T07:02:32.067 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 -- start start 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 0x7fc7b819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8108510 0x7fc7b8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8112620 con 0x7fc7b8100a80 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81124a0 con 0x7fc7b8101440 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.065+0000 7fc7beb79700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81127a0 con 0x7fc7b8108510 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35600/0 (socket says 172.21.15.73:35600) 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 -- 172.21.15.73:0/2388250349 learned_addr learned my addr 172.21.15.73:0/2388250349 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 0x7fc7b819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8108510 0x7fc7b8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.070 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 -- 172.21.15.73:0/2388250349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8108510 msgr2=0x7fc7b8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8108510 0x7fc7b8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 -- 172.21.15.73:0/2388250349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 msgr2=0x7fc7b819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 0x7fc7b819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 -- 172.21.15.73:0/2388250349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b81990c0 con 0x7fc7b8100a80 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 0x7fc7b819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.066+0000 7fc7bc915700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000c00 tx=0x7fc7ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.067+0000 7fc7b5ffb700 1 -- 172.21.15.73:0/2388250349 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac027070 con 0x7fc7b8100a80 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.067+0000 7fc7b5ffb700 1 -- 172.21.15.73:0/2388250349 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7ac022470 con 0x7fc7b8100a80 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.067+0000 7fc7b5ffb700 1 -- 172.21.15.73:0/2388250349 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac017650 con 0x7fc7b8100a80 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.067+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b8199350 con 0x7fc7b8100a80 2024-06-06T07:02:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.067+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a5670 con 0x7fc7b8100a80 2024-06-06T07:02:32.073 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.071+0000 7fc7b5ffb700 1 -- 172.21.15.73:0/2388250349 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fc7ac02a070 con 0x7fc7b8100a80 2024-06-06T07:02:32.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.072+0000 7fc7c93fcb80 1 Processor -- start 2024-06-06T07:02:32.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.072+0000 7fc7c93fcb80 1 -- start start 2024-06-06T07:02:32.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.072+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 0x555bb17f7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.074 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.072+0000 7fc7b5ffb700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 0x7fc7a80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb17f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.075 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb180c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555bb16e9f60 con 0x555bb1809ec0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555bb1748850 con 0x555bb17f77a0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bb16b0460 con 0x555bb17fb000 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bd116700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 0x555bb17f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bd116700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 0x555bb17f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:52356/0 (socket says 172.21.15.73:52356) 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bd116700 1 -- 172.21.15.73:0/589235001 learned_addr learned my addr 172.21.15.73:0/589235001 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bd116700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb180c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb17f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bc915700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 0x7fc7a80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.084 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 -- 172.21.15.73:0/589235001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 msgr2=0x555bb17f7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 0x555bb17f7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 -- 172.21.15.73:0/589235001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 msgr2=0x555bb180c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb180c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7b7fff700 1 -- 172.21.15.73:0/589235001 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb16978d0 con 0x555bb17f77a0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.073+0000 7fc7bd116700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb180c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb17f7c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009ca0 tx=0x7fc7a4010370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77dd81377f2883bd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7b5ffb700 1 -- 172.21.15.73:0/2388250349 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc7ac0d05a0 con 0x7fc7b8100a80 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7a2ffd700 1 -- 172.21.15.73:0/589235001 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4010bf0 con 0x555bb17f77a0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7bc915700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 0x7fc7a80a53f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0cd820 tx=0x7fc7ac098190 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7a2ffd700 1 -- 172.21.15.73:0/589235001 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7a40044d0 con 0x555bb17f77a0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.074+0000 7fc7a2ffd700 1 -- 172.21.15.73:0/589235001 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4012450 con 0x555bb17f77a0 2024-06-06T07:02:32.085 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/589235001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 msgr2=0x555bb17f7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb17f7c20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4009ca0 tx=0x7fc7a4010370 comp rx=0 tx=0).stop 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/589235001 shutdown_connections 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17fb000 0x555bb17f7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb180c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/589235001 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb17f7c20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/589235001 >> 172.21.15.73:0/589235001 conn(0x555bb1724080 msgr2=0x555bb17f2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/589235001 shutdown_connections 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/589235001 wait complete. 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 -- start start 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.075+0000 7fc7c93fcb80 1 Processor -- start 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb1809650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.086 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35610/0 (socket says 172.21.15.73:35610) 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7b7fff700 1 -- 172.21.15.73:0/2626678533 learned_addr learned my addr 172.21.15.73:0/2626678533 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 0x555bb1806e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555bb16e8fc0 con 0x555bb17fb000 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555bb1744810 con 0x555bb1809ec0 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.076+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555bb180cb40 con 0x555bb17f77a0 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb1809650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 0x555bb1806e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 -- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 msgr2=0x555bb1809650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb1809650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 -- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 msgr2=0x555bb1806e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 0x555bb1806e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.087 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb170e750 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 0x555bb1806e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7c93fcb80 10 client.0 _ll_register_callbacks cb 0x555bb16de5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400a010 tx=0x7fc7a4005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd928ffd0c0fcb8c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc797fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.077+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a40044d0 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc797fff700 1 -- 172.21.15.73:0/2626678533 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a402a060 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc797fff700 1 -- 172.21.15.73:0/2626678533 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc7a4003fc0 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc797fff700 1 -- 172.21.15.73:0/2626678533 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7a4004240 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc7c93fcb80 10 client.15675 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=32}) -- 0x555bb1816f80 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.078+0000 7fc797fff700 1 -- 172.21.15.73:0/2626678533 <== mon.0 v2:172.21.15.73:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc7a400f040 con 0x555bb17fb000 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 10 client.15675 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:32.088 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 20 client.15675 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1819450 0x555bb181b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 4 client.15675 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc7a40044d0 con 0x555bb1819450 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1819450 0x555bb181b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1819450 0x555bb181b840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc13041b189d1727 server_cookie=c4d9095ef68bacde in_seq=0 out_seq=0 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.079+0000 7fc797fff700 10 client.15675 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.080+0000 7fc797fff700 1 -- 172.21.15.73:0/2626678533 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc7a40044d0 con 0x555bb1819450 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.080+0000 7fc797fff700 10 client.15675 handle_command_reply: tid=0 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.080+0000 7fc7c93fcb80 1 client.15675 shutdown 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.080+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 msgr2=0x555bb1806950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.080+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400a010 tx=0x7fc7a4005dc0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.089 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 shutdown_connections 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1819450 0x555bb181b840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb1809650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb1809ec0 0x555bb1806e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2626678533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1806950 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 >> 172.21.15.73:0/2626678533 conn(0x555bb1724080 msgr2=0x555bb17f1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc797fff700 0 client.15675 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 shutdown_connections 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2626678533 wait complete. 2024-06-06T07:02:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.081+0000 7fc7c93fcb80 20 client.15675 trim_cache size 0 max 16384 2024-06-06T07:02:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 Processor -- start 2024-06-06T07:02:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 -- start start 2024-06-06T07:02:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb181b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.110 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 0x555bb17f1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ac330 con 0x555bb1809ec0 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ade70 con 0x555bb17fb000 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7c93fcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81aca00 con 0x555bb17f77a0 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7bd116700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7bd116700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35624/0 (socket says 172.21.15.73:35624) 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7bd116700 1 -- 172.21.15.73:0/2545626356 learned_addr learned my addr 172.21.15.73:0/2545626356 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb181b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.111 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.108+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 0x555bb17f1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 -- 172.21.15.73:0/2545626356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 msgr2=0x555bb181b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb181b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 -- 172.21.15.73:0/2545626356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 msgr2=0x555bb17f1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.113 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 0x555bb17f1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 -- 172.21.15.73:0/2545626356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bb1804f00 con 0x555bb1809ec0 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 0x555bb17f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7bd116700 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00088d0 tx=0x7fc7b000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc56fc6794514e82 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7a27fc700 1 -- 172.21.15.73:0/2545626356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001c780 con 0x555bb1809ec0 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7a27fc700 1 -- 172.21.15.73:0/2545626356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7b001cdd0 con 0x555bb1809ec0 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.109+0000 7fc7a27fc700 1 -- 172.21.15.73:0/2545626356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001ad90 con 0x555bb1809ec0 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.110+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2545626356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 msgr2=0x555bb17f23c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.110+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00088d0 tx=0x7fc7b000b040 comp rx=0 tx=0).stop 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2545626356 shutdown_connections 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb17f77a0 0x555bb181b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb1809ec0 0x555bb17f23c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/2545626356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17fb000 0x555bb17f1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.114 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2545626356 >> 172.21.15.73:0/2545626356 conn(0x555bb1724080 msgr2=0x555bb1805280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2545626356 shutdown_connections 2024-06-06T07:02:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/2545626356 wait complete. 2024-06-06T07:02:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 -- start start 2024-06-06T07:02:32.115 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.111+0000 7fc7c93fcb80 1 Processor -- start 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb1811e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35632/0 (socket says 172.21.15.73:35632) 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb1809ec0 0x555bb18201b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7b7fff700 1 -- 172.21.15.73:0/1244399183 learned_addr learned my addr 172.21.15.73:0/1244399183 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ad7a0 con 0x555bb17fb000 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81ad0d0 con 0x555bb17f77a0 2024-06-06T07:02:32.116 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81aa2d0 con 0x555bb1809ec0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.112+0000 7fc7bd116700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb1811e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7c93fcb80 10 client.0 _ll_register_callbacks cb 0x555bb17f8130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb1809ec0 0x555bb18201b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 -- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb1809ec0 msgr2=0x555bb18201b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb1809ec0 0x555bb18201b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 -- 172.21.15.73:0/1244399183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 msgr2=0x555bb1812360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bb18206f0 con 0x555bb17f77a0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7bd116700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb1811e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009140 tx=0x7fc7b00091b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7f4d94d37fce6e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc797fff700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:02:32.117 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b001ae20 con 0x555bb17f77a0 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc7b001c580 con 0x555bb17f77a0 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b001cdd0 con 0x555bb17f77a0 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7b001a750 con 0x555bb17f77a0 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7c93fcb80 10 client.25172 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.113+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555bb1883830 con 0x555bb17f77a0 2024-06-06T07:02:32.118 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.114+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7b001cdd0 con 0x555bb17f77a0 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fc7b002c070 con 0x555bb17f77a0 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 10 client.25172 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 10 client.25172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 20 client.25172 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1883d90 0x555bb1886180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 4 client.25172 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555bb1755d50 con 0x555bb1883d90 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1883d90 0x555bb1886180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc797fff700 10 client.25172 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.115+0000 7fc7b7fff700 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1883d90 0x555bb1886180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c43735ecf8913744 server_cookie=f98164e5560c6ce8 in_seq=0 out_seq=0 2024-06-06T07:02:32.121 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc797fff700 1 -- 172.21.15.73:0/1244399183 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555bb1755d50 con 0x555bb1883d90 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc797fff700 10 client.25172 handle_command_reply: tid=0 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 client.25172 shutdown 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 msgr2=0x555bb1811e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb1811e20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009140 tx=0x7fc7b00091b0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 shutdown_connections 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555bb1883d90 0x555bb1886180 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555bb1809ec0 0x555bb18201b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555bb17f77a0 0x555bb1811e20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 --2- 172.21.15.73:0/1244399183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555bb17fb000 0x555bb1812360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 >> 172.21.15.73:0/1244399183 conn(0x555bb1724080 msgr2=0x555bb1803e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc797fff700 0 client.25172 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:32.122 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 shutdown_connections 2024-06-06T07:02:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 1 -- 172.21.15.73:0/1244399183 wait complete. 2024-06-06T07:02:32.123 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.116+0000 7fc7c93fcb80 20 client.25172 trim_cache size 0 max 16384 2024-06-06T07:02:32.123 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.117+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 msgr2=0x7fc7a80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.117+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 0x7fc7a80a53f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac0cd820 tx=0x7fc7ac098190 comp rx=0 tx=0).stop 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.117+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 msgr2=0x7fc7b819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.117+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac000c00 tx=0x7fc7ac0047b0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 shutdown_connections 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc7a80a2f30 0x7fc7a80a53f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc7b8108510 0x7fc7b8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.124 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7b8100a80 0x7fc7b819e7a0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 --2- 172.21.15.73:0/2388250349 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc7b8101440 0x7fc7b819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:32.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 >> 172.21.15.73:0/2388250349 conn(0x7fc7b8077a00 msgr2=0x7fc7b80ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:32.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 shutdown_connections 2024-06-06T07:02:32.125 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:32.118+0000 7fc7beb79700 1 -- 172.21.15.73:0/2388250349 wait complete. 2024-06-06T07:02:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:33 smithi073 ceph-mon[32303]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:33 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:02:33.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:33 smithi170 ceph-mon[36108]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:33.412 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:33 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:02:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:33 smithi136 ceph-mon[35901]: pgmap v39: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-06T07:02:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:33 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:02:33.996 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:02:33.996 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:02:34.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:34 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:02:34.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:34 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:02:34.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:34 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:02:34.280 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:34.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:34 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:02:34.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:34 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:02:34.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:34 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:02:34.591 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:34 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:02:34.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:34 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:02:34.592 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:34 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:02:35.428 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:35 smithi073 ceph-mon[32303]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-06T07:02:35.452 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:35 smithi136 ceph-mon[35901]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-06T07:02:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:35 smithi170 ceph-mon[36108]: pgmap v40: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-06T07:02:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 -- 172.21.15.73:0/477657384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec101df0 msgr2=0x7f6dec1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/477657384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec101df0 0x7f6dec1021d0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6de80099c0 tx=0x7f6de8009cd0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 -- 172.21.15.73:0/477657384 shutdown_connections 2024-06-06T07:02:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/477657384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 0x7f6dec10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/477657384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec101df0 0x7f6dec1021d0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/477657384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec1027a0 0x7f6dec106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 -- 172.21.15.73:0/477657384 >> 172.21.15.73:0/477657384 conn(0x7f6dec077a00 msgr2=0x7f6dec0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.186+0000 7f6df3b3b700 1 -- 172.21.15.73:0/477657384 shutdown_connections 2024-06-06T07:02:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.187+0000 7f6df3b3b700 1 -- 172.21.15.73:0/477657384 wait complete. 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df3b3b700 1 Processor -- start 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df3b3b700 1 -- start start 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df3b3b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec101df0 0x7f6dec10ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df3b3b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df3b3b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 0x7f6dec10f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df10d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df10d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48272/0 (socket says 172.21.15.73:48272) 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.188+0000 7f6df10d6700 1 -- 172.21.15.73:0/4252239862 learned_addr learned my addr 172.21.15.73:0/4252239862 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec112620 con 0x7f6dec1027a0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1124a0 con 0x7f6dec106750 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df20d8700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 0x7f6dec10f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dec1127a0 con 0x7f6dec101df0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df18d7700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec101df0 0x7f6dec10ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 -- 172.21.15.73:0/4252239862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec101df0 msgr2=0x7f6dec10ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec101df0 0x7f6dec10ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 -- 172.21.15.73:0/4252239862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 msgr2=0x7f6dec10f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 0x7f6dec10f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dec1ab3e0 con 0x7f6dec1027a0 2024-06-06T07:02:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.189+0000 7f6df10d6700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c9b0 tx=0x7f6ddc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.190+0000 7f6de2ffd700 1 -- 172.21.15.73:0/4252239862 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc007b00 con 0x7f6dec1027a0 2024-06-06T07:02:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.190+0000 7f6de2ffd700 1 -- 172.21.15.73:0/4252239862 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ddc00ce50 con 0x7f6dec1027a0 2024-06-06T07:02:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.190+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dec1ab670 con 0x7f6dec1027a0 2024-06-06T07:02:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.190+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dec1abc00 con 0x7f6dec1027a0 2024-06-06T07:02:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.190+0000 7f6de2ffd700 1 -- 172.21.15.73:0/4252239862 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc018660 con 0x7f6dec1027a0 2024-06-06T07:02:36.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.194+0000 7f6de2ffd700 1 -- 172.21.15.73:0/4252239862 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f6ddc007ca0 con 0x7f6dec1027a0 2024-06-06T07:02:36.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.194+0000 7f6de2ffd700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 0x7f6dd80a53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.194+0000 7f6de2ffd700 1 -- 172.21.15.73:0/4252239862 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6ddc0cf980 con 0x7f6dec1027a0 2024-06-06T07:02:36.195 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.194+0000 7f6df18d7700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 0x7f6dd80a53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.195+0000 7f6df18d7700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 0x7f6dd80a53f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009fd0 tx=0x7f6de8005fb0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.195+0000 7f6dfe3beb80 1 Processor -- start 2024-06-06T07:02:36.196 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.195+0000 7f6dfe3beb80 1 -- start start 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x559398999a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x55939899a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593988eb890 con 0x55939899ffe0 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5593988937a0 con 0x559398999fa0 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5593988e7850 con 0x55939899f6c0 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:42404/0 (socket says 172.21.15.73:42404) 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 -- 172.21.15.73:0/1740837786 learned_addr learned my addr 172.21.15.73:0/1740837786 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df20d8700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x55939899a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 -- 172.21.15.73:0/1740837786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 msgr2=0x55939899a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x55939899a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df18d7700 1 -- 172.21.15.73:0/1740837786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 msgr2=0x559398999a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.196+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x559398999a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x559398999a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6df18d7700 1 -- 172.21.15.73:0/1740837786 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559398889fd0 con 0x55939899f6c0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x559398999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6de8015000 tx=0x7f6de8020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1ec9010bc1ec66 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6dd7fff700 1 -- 172.21.15.73:0/1740837786 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de800bca0 con 0x55939899f6c0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6dd7fff700 1 -- 172.21.15.73:0/1740837786 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6de800be40 con 0x55939899f6c0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.197+0000 7f6dd7fff700 1 -- 172.21.15.73:0/1740837786 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6de8023bb0 con 0x55939899f6c0 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1740837786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 msgr2=0x55939899faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6de8015000 tx=0x7f6de8020000 comp rx=0 tx=0).stop 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1740837786 shutdown_connections 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x55939899faa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x55939899a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1740837786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x559398999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1740837786 >> 172.21.15.73:0/1740837786 conn(0x5593988c4d40 msgr2=0x5593989948d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1740837786 shutdown_connections 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.198+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1740837786 wait complete. 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6dfe3beb80 1 -- start start 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6dfe3beb80 1 Processor -- start 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.201 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x5593989a3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6df18d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.199+0000 7f6df18d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48284/0 (socket says 172.21.15.73:48284) 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df18d7700 1 -- 172.21.15.73:0/1912508243 learned_addr learned my addr 172.21.15.73:0/1912508243 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899ffe0 0x5593989a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593989a6990 con 0x559398999fa0 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x5593989a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df20d8700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899ffe0 0x5593989a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55939888bc70 con 0x55939899ffe0 2024-06-06T07:02:36.202 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5593989a6b10 con 0x55939899f6c0 2024-06-06T07:02:36.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 -- 172.21.15.73:0/1912508243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899ffe0 msgr2=0x5593989a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.203 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899ffe0 0x5593989a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 -- 172.21.15.73:0/1912508243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 msgr2=0x5593989a5d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6df10d6700 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55939883a950 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 10 client.0 _ll_register_callbacks cb 0x5593988859a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.200+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6de800be40 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6df10d6700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x5593989a3020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0ccb40 tx=0x7f6ddc09c080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9116ca949a69e76 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dd57fa700 10 client.25252 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dd57fa700 1 -- 172.21.15.73:0/1912508243 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0a4020 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dd57fa700 1 -- 172.21.15.73:0/1912508243 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6ddc0a41c0 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dd57fa700 1 -- 172.21.15.73:0/1912508243 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ddc0cce20 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dfe3beb80 10 client.25252 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.201+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=32}) -- 0x5593989b10e0 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dd57fa700 1 -- 172.21.15.73:0/1912508243 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6ddc09cb70 con 0x55939899f6c0 2024-06-06T07:02:36.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 10 client.25252 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 10 client.25252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 20 client.25252 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x5593989b5a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 4 client.25252 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593988f8d90 con 0x5593989b3610 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x5593989b5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6df18d7700 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x5593989b5a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1dc261e912763f2 server_cookie=af0082ca59162ebc in_seq=0 out_seq=0 2024-06-06T07:02:36.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.202+0000 7f6dd57fa700 10 client.25252 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.203+0000 7f6dd57fa700 1 -- 172.21.15.73:0/1912508243 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593988f8d90 con 0x5593989b3610 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.203+0000 7f6dd57fa700 10 client.25252 handle_command_reply: tid=0 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.203+0000 7f6dfe3beb80 1 client.25252 shutdown 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.203+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 msgr2=0x5593989a3020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.203+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x5593989a3020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc0ccb40 tx=0x7f6ddc09c080 comp rx=0 tx=0).stop 2024-06-06T07:02:36.206 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 shutdown_connections 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x5593989b5a40 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899ffe0 0x5593989a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559398999fa0 0x5593989a5d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dd57fa700 0 client.25252 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/1912508243 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899f6c0 0x5593989a3020 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 >> 172.21.15.73:0/1912508243 conn(0x5593988c4d40 msgr2=0x559398994670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 shutdown_connections 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/1912508243 wait complete. 2024-06-06T07:02:36.207 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.204+0000 7f6dfe3beb80 20 client.25252 trim_cache size 0 max 16384 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.244+0000 7f6dfe3beb80 1 Processor -- start 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.244+0000 7f6dfe3beb80 1 -- start start 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.244+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x559398999a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.244+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899ffe0 0x5593989a2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48288/0 (socket says 172.21.15.73:48288) 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 -- 172.21.15.73:0/3476861862 learned_addr learned my addr 172.21.15.73:0/3476861862 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd000ed10 con 0x55939899f6c0 2024-06-06T07:02:36.246 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0010850 con 0x559398999fa0 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd000f3e0 con 0x55939899ffe0 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df18d7700 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x559398999a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 -- 172.21.15.73:0/3476861862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899ffe0 msgr2=0x5593989a2920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899ffe0 0x5593989a2920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 -- 172.21.15.73:0/3476861862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 msgr2=0x559398999a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x559398999a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 -- 172.21.15.73:0/3476861862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55939899f130 con 0x55939899f6c0 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df18d7700 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x559398999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:36.247 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc098210 tx=0x7f6ddc0ce000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93b16b267b54600 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dd77fe700 1 -- 172.21.15.73:0/3476861862 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc09cb70 con 0x55939899f6c0 2024-06-06T07:02:36.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dd77fe700 1 -- 172.21.15.73:0/3476861862 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ddc07f070 con 0x55939899f6c0 2024-06-06T07:02:36.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.245+0000 7f6dd77fe700 1 -- 172.21.15.73:0/3476861862 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0977f0 con 0x55939899f6c0 2024-06-06T07:02:36.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 msgr2=0x5593989a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.249 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc098210 tx=0x7f6ddc0ce000 comp rx=0 tx=0).stop 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 shutdown_connections 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55939899ffe0 0x5593989a2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559398999fa0 0x559398999a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3476861862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899f6c0 0x5593989a0c50 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.246+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 >> 172.21.15.73:0/3476861862 conn(0x5593988c4d40 msgr2=0x55939898de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.247+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 shutdown_connections 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.247+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3476861862 wait complete. 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.247+0000 7f6dfe3beb80 1 -- start start 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.247+0000 7f6dfe3beb80 1 Processor -- start 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559398999fa0 0x5593989abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 0x5593989ac370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.250 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0010180 con 0x55939899ffe0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd000fab0 con 0x55939899f6c0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd000ccb0 con 0x559398999fa0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6dfe3beb80 10 client.0 _ll_register_callbacks cb 0x5593988859a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df18d7700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559398999fa0 0x5593989abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df20d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 0x5593989ac370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df20d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48294/0 (socket says 172.21.15.73:48294) 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df20d8700 1 -- 172.21.15.73:0/3047903304 learned_addr learned my addr 172.21.15.73:0/3047903304 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 -- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559398999fa0 msgr2=0x5593989abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559398999fa0 0x5593989abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 -- 172.21.15.73:0/3047903304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 msgr2=0x5593989a8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.251 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df10d6700 1 -- 172.21.15.73:0/3047903304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593989aca40 con 0x55939899f6c0 2024-06-06T07:02:36.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.248+0000 7f6df20d8700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:36.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6df10d6700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 0x5593989ac370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x559398995910 tx=0x7f6ddc097490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cddeb605f9891365 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:36.252 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dd57fa700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:02:36.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc0a8030 con 0x55939899f6c0 2024-06-06T07:02:36.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ddc07f070 con 0x55939899f6c0 2024-06-06T07:02:36.253 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6ddc0a4020 con 0x55939899f6c0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6ddc0a4220 con 0x55939899f6c0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dfe3beb80 10 client.25184 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5593989aca40 con 0x55939899f6c0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.249+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ddc07f070 con 0x55939899f6c0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f6ddc0a0f20 con 0x55939899f6c0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 10 client.25184 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 10 client.25184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 20 client.25184 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x559398a20be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 4 client.25184 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5593988f3310 con 0x5593989b3610 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6df18d7700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x559398a20be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:36.254 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6df18d7700 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x559398a20be0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e37da9c91044b81 server_cookie=7c86c8ee77c9e86b in_seq=0 out_seq=0 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.250+0000 7f6dd57fa700 10 client.25184 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.251+0000 7f6dd57fa700 1 -- 172.21.15.73:0/3047903304 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5593988f3310 con 0x5593989b3610 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.251+0000 7f6dd57fa700 10 client.25184 handle_command_reply: tid=0 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 client.25184 shutdown 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 msgr2=0x5593989ac370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 0x5593989ac370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x559398995910 tx=0x7f6ddc097490 comp rx=0 tx=0).stop 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 shutdown_connections 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5593989b3610 0x559398a20be0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559398999fa0 0x5593989abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55939899ffe0 0x5593989a8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 --2- 172.21.15.73:0/3047903304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55939899f6c0 0x5593989ac370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 >> 172.21.15.73:0/3047903304 conn(0x5593988c4d40 msgr2=0x55939899e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dd57fa700 0 client.25184 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 shutdown_connections 2024-06-06T07:02:36.255 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 1 -- 172.21.15.73:0/3047903304 wait complete. 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.252+0000 7f6dfe3beb80 20 client.25184 trim_cache size 0 max 16384 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stdout: "return_code": 0, 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_tag": "a15130bc-f1ed-4ce9-ad1e-f35bc876136e", 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": "asynchronous" 2024-06-06T07:02:36.256 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 msgr2=0x7f6dd80a53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 0x7f6dd80a53f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6de8009fd0 tx=0x7f6de8005fb0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 msgr2=0x7f6dec10f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00c9b0 tx=0x7f6ddc00ccc0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 shutdown_connections 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6dd80a2f30 0x7f6dd80a53f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6dec101df0 0x7f6dec10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6dec106750 0x7f6dec10f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 --2- 172.21.15.73:0/4252239862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dec1027a0 0x7f6dec10f170 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.253+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 >> 172.21.15.73:0/4252239862 conn(0x7f6dec077a00 msgr2=0x7f6dec104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.254+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 shutdown_connections 2024-06-06T07:02:36.257 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:36.254+0000 7f6df3b3b700 1 -- 172.21.15.73:0/4252239862 wait complete. 2024-06-06T07:02:37.079 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:02:37.348 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:37.478 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:37 smithi073 ceph-mon[32303]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:37.478 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:37 smithi073 ceph-mon[32303]: scrub queued for path: / 2024-06-06T07:02:37.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:37 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:37.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:37 smithi073 ceph-mon[32303]: scrub summary: active paths [/] 2024-06-06T07:02:37.479 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:37 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:37 smithi136 ceph-mon[35901]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:37.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:37 smithi136 ceph-mon[35901]: scrub queued for path: / 2024-06-06T07:02:37.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:37 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:37.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:37 smithi136 ceph-mon[35901]: scrub summary: active paths [/] 2024-06-06T07:02:37.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:37 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:37 smithi170 ceph-mon[36108]: pgmap v41: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:37 smithi170 ceph-mon[36108]: scrub queued for path: / 2024-06-06T07:02:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:37 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:37 smithi170 ceph-mon[36108]: scrub summary: active paths [/] 2024-06-06T07:02:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:37 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:02:39.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.275+0000 7f595b302700 1 -- 172.21.15.73:0/3826134505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954101d30 msgr2=0x7f595410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.275+0000 7f595b302700 1 --2- 172.21.15.73:0/3826134505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954101d30 0x7f595410e210 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f594c00b0a0 tx=0x7f594c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.279 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 -- 172.21.15.73:0/3826134505 shutdown_connections 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 --2- 172.21.15.73:0/3826134505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f59541073d0 0x7f59541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 --2- 172.21.15.73:0/3826134505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5954101d30 0x7f595410e210 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 --2- 172.21.15.73:0/3826134505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f59541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 -- 172.21.15.73:0/3826134505 >> 172.21.15.73:0/3826134505 conn(0x7f59540782c0 msgr2=0x7f59540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 -- 172.21.15.73:0/3826134505 shutdown_connections 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.276+0000 7f595b302700 1 -- 172.21.15.73:0/3826134505 wait complete. 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 Processor -- start 2024-06-06T07:02:39.280 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 -- start start 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954101d30 0x7f5954197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59541073d0 0x7f59541981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5954111500 con 0x7f59541073d0 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5954111380 con 0x7f5954101390 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595b302700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5954111680 con 0x7f5954101d30 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595909e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595909e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54556/0 (socket says 172.21.15.73:54556) 2024-06-06T07:02:39.281 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595909e700 1 -- 172.21.15.73:0/2839776147 learned_addr learned my addr 172.21.15.73:0/2839776147 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.277+0000 7f595889d700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954101d30 0x7f5954197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 -- 172.21.15.73:0/2839776147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954101d30 msgr2=0x7f5954197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954101d30 0x7f5954197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 -- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59541073d0 msgr2=0x7f59541981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59541073d0 0x7f59541981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 -- 172.21.15.73:0/2839776147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59541a6fc0 con 0x7f5954101390 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595909e700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f595000e9c0 tx=0x7f595000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f594a7fc700 1 -- 172.21.15.73:0/2839776147 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595000cbd0 con 0x7f5954101390 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59541a72b0 con 0x7f5954101390 2024-06-06T07:02:39.282 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f594a7fc700 1 -- 172.21.15.73:0/2839776147 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595000cd70 con 0x7f5954101390 2024-06-06T07:02:39.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f594a7fc700 1 -- 172.21.15.73:0/2839776147 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500056c0 con 0x7f5954101390 2024-06-06T07:02:39.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.278+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59541a77f0 con 0x7f5954101390 2024-06-06T07:02:39.283 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.282+0000 7f594a7fc700 1 -- 172.21.15.73:0/2839776147 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f5950004750 con 0x7f5954101390 2024-06-06T07:02:39.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 Processor -- start 2024-06-06T07:02:39.284 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 -- start start 2024-06-06T07:02:39.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f594a7fc700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 0x7f59400a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.285 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563870b56810 con 0x563870c04f20 2024-06-06T07:02:39.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563870afe720 con 0x563870c0af60 2024-06-06T07:02:39.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.284+0000 7f5965b85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563870b527d0 con 0x563870c0a640 2024-06-06T07:02:39.286 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.285+0000 7f595889d700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 0x7f59400a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.285+0000 7f595989f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.285+0000 7f595989f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48318/0 (socket says 172.21.15.73:48318) 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.285+0000 7f595989f700 1 -- 172.21.15.73:0/124590513 learned_addr learned my addr 172.21.15.73:0/124590513 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.285+0000 7f595909e700 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f594a7fc700 1 -- 172.21.15.73:0/2839776147 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f59500cfcd0 con 0x7f5954101390 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 -- 172.21.15.73:0/124590513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 msgr2=0x563870c0aa20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0aa20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 -- 172.21.15.73:0/124590513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 msgr2=0x563870c05340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595889d700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 0x7f59400a5470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59541998b0 tx=0x7f5944009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 -- 172.21.15.73:0/124590513 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563870af4f50 con 0x563870c0af60 2024-06-06T07:02:39.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595989f700 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:39.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.286+0000 7f595909e700 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c049e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59500cce90 tx=0x7f595000efd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d956751590eb4823 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.288 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.287+0000 7f593f7fe700 1 -- 172.21.15.73:0/124590513 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595009bb90 con 0x563870c0af60 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.287+0000 7f593f7fe700 1 -- 172.21.15.73:0/124590513 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595009ff40 con 0x563870c0af60 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.287+0000 7f593f7fe700 1 -- 172.21.15.73:0/124590513 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595009bff0 con 0x563870c0af60 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- 172.21.15.73:0/124590513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 msgr2=0x563870c049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c049e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f59500cce90 tx=0x7f595000efd0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- 172.21.15.73:0/124590513 shutdown_connections 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 --2- 172.21.15.73:0/124590513 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c049e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- 172.21.15.73:0/124590513 >> 172.21.15.73:0/124590513 conn(0x563870b2fcc0 msgr2=0x563870bff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- 172.21.15.73:0/124590513 shutdown_connections 2024-06-06T07:02:39.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- 172.21.15.73:0/124590513 wait complete. 2024-06-06T07:02:39.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.288+0000 7f5965b85b80 1 -- start start 2024-06-06T07:02:39.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 Processor -- start 2024-06-06T07:02:39.290 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c0e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f595909e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563870c1a120 con 0x563870c04f20 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f595909e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48330/0 (socket says 172.21.15.73:48330) 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f595909e700 1 -- 172.21.15.73:0/824053135 learned_addr learned my addr 172.21.15.73:0/824053135 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.289+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563870af6bf0 con 0x563870c0af60 2024-06-06T07:02:39.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563870c1a2a0 con 0x563870c0a640 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c0e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595889d700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f5965b85b80 10 client.0 _ll_register_callbacks cb 0x563870af0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 -- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 msgr2=0x563870c0e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 -- 172.21.15.73:0/824053135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 msgr2=0x563870c10e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595989f700 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563870aa58d0 con 0x563870c0af60 2024-06-06T07:02:39.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.290+0000 7f595909e700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f595989f700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c0e660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594c009fd0 tx=0x7f594c00bdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d660ecef11463679 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f593cff9700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c003e10 con 0x563870c0af60 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595009ff40 con 0x563870c0af60 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f594c0074e0 con 0x563870c0af60 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f594c02e050 con 0x563870c0af60 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f5965b85b80 10 client.25196 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.291+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563870c248e0 con 0x563870c0af60 2024-06-06T07:02:39.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.292+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595009ff40 con 0x563870c0af60 2024-06-06T07:02:39.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f594c011070 con 0x563870c0af60 2024-06-06T07:02:39.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 10 client.25196 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 10 client.25196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 20 client.25196 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c27210 0x563870c29630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 4 client.25196 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563870b63d10 con 0x563870c27210 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.293+0000 7f595909e700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c27210 0x563870c29630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.294+0000 7f593cff9700 10 client.25196 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:39.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.294+0000 7f595909e700 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c27210 0x563870c29630 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f799ad693649227 server_cookie=bfa107d44b90d2ae in_seq=0 out_seq=0 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.294+0000 7f593cff9700 1 -- 172.21.15.73:0/824053135 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563870b63d10 con 0x563870c27210 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.294+0000 7f593cff9700 10 client.25196 handle_command_reply: tid=0 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 client.25196 shutdown 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 msgr2=0x563870c0e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c0e660 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594c009fd0 tx=0x7f594c00bdb0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 shutdown_connections 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c27210 0x563870c29630 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c0e660 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c10e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 --2- 172.21.15.73:0/824053135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0a640 0x563870c0e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 >> 172.21.15.73:0/824053135 conn(0x563870b2fcc0 msgr2=0x563870c039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f593cff9700 0 client.25196 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 shutdown_connections 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 1 -- 172.21.15.73:0/824053135 wait complete. 2024-06-06T07:02:39.297 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.295+0000 7f5965b85b80 20 client.25196 trim_cache size 0 max 16384 2024-06-06T07:02:39.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.335+0000 7f5965b85b80 1 Processor -- start 2024-06-06T07:02:39.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.335+0000 7f5965b85b80 1 -- start start 2024-06-06T07:02:39.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0a640 0x563870c087d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0af60 0x563870c2ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59541ac140 con 0x563870c04f20 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59541adc80 con 0x563870c0a640 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f5965b85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f59541ac810 con 0x563870c0af60 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f595909e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f595909e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48346/0 (socket says 172.21.15.73:48346) 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f595909e700 1 -- 172.21.15.73:0/3146707256 learned_addr learned my addr 172.21.15.73:0/3146707256 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f595889d700 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0a640 0x563870c087d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.336+0000 7f595989f700 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0af60 0x563870c2ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 -- 172.21.15.73:0/3146707256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0af60 msgr2=0x563870c2ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0af60 0x563870c2ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 -- 172.21.15.73:0/3146707256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0a640 msgr2=0x563870c087d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0a640 0x563870c087d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 -- 172.21.15.73:0/3146707256 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563870bffd90 con 0x563870c04f20 2024-06-06T07:02:39.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f595909e700 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5950099e40 tx=0x7f59500cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be09e5e1ff99812 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f593effd700 1 -- 172.21.15.73:0/3146707256 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59500a4800 con 0x563870c04f20 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f593effd700 1 -- 172.21.15.73:0/3146707256 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595007f070 con 0x563870c04f20 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.337+0000 7f593effd700 1 -- 172.21.15.73:0/3146707256 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595009bdf0 con 0x563870c04f20 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 -- 172.21.15.73:0/3146707256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 msgr2=0x563870c049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5950099e40 tx=0x7f59500cf040 comp rx=0 tx=0).stop 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 -- 172.21.15.73:0/3146707256 shutdown_connections 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c0af60 0x563870c2ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c04f20 0x563870c049e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3146707256 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0a640 0x563870c087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 -- 172.21.15.73:0/3146707256 >> 172.21.15.73:0/3146707256 conn(0x563870b2fcc0 msgr2=0x563870c080c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.338+0000 7f5965b85b80 1 -- 172.21.15.73:0/3146707256 shutdown_connections 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.339+0000 7f5965b85b80 1 -- 172.21.15.73:0/3146707256 wait complete. 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.339+0000 7f5965b85b80 1 -- start start 2024-06-06T07:02:39.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.339+0000 7f5965b85b80 1 Processor -- start 2024-06-06T07:02:39.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 0x563870c1f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c1fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f59541ad5b0 con 0x563870c0a640 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f59541acee0 con 0x563870c0af60 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f59541aa1b0 con 0x563870c2e2f0 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f5965b85b80 10 client.0 _ll_register_callbacks cb 0x563870af0920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f595989f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f595889d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c1fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f595989f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:42500/0 (socket says 172.21.15.73:42500) 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f595989f700 1 -- 172.21.15.73:0/3723933460 learned_addr learned my addr 172.21.15.73:0/3723933460 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.340+0000 7f595909e700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 0x563870c1f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 -- 172.21.15.73:0/3723933460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 msgr2=0x563870c1fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c1fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 -- 172.21.15.73:0/3723933460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 msgr2=0x563870c1f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:39.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 0x563870c1f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 -- 172.21.15.73:0/3723933460 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563870c2ee00 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595909e700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 0x563870c1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f595989f700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f594c005680 tx=0x7f594c00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c4c67704704d6b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f593cff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f594c005e70 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f594c0058c0 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595007f070 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f594c03a050 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f5965b85b80 10 client.25267 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.341+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563870c91ce0 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f594c02a070 con 0x563870c2e2f0 2024-06-06T07:02:39.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f594c011070 con 0x563870c2e2f0 2024-06-06T07:02:39.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 10 client.25267 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:39.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 10 client.25267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:39.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 20 client.25267 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c923f0 0x563870c94810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 4 client.25267 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f595007f070 con 0x563870c923f0 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.342+0000 7f595909e700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c923f0 0x563870c94810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.343+0000 7f593cff9700 10 client.25267 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.343+0000 7f595909e700 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c923f0 0x563870c94810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d44dd95db4bf596f server_cookie=7d545d62e62e706b in_seq=0 out_seq=0 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.343+0000 7f593cff9700 1 -- 172.21.15.73:0/3723933460 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f595007f070 con 0x563870c923f0 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.343+0000 7f593cff9700 10 client.25267 handle_command_reply: tid=0 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.343+0000 7f5965b85b80 1 client.25267 shutdown 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 msgr2=0x563870c2e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f594c005680 tx=0x7f594c00bf60 comp rx=0 tx=0).stop 2024-06-06T07:02:39.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 shutdown_connections 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563870c923f0 0x563870c94810 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563870c2e2f0 0x563870c2e770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563870c0a640 0x563870c1f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 --2- 172.21.15.73:0/3723933460 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563870c0af60 0x563870c1fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 >> 172.21.15.73:0/3723933460 conn(0x563870b2fcc0 msgr2=0x563870bf8e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f593cff9700 0 client.25267 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 shutdown_connections 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 1 -- 172.21.15.73:0/3723933460 wait complete. 2024-06-06T07:02:39.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.344+0000 7f5965b85b80 20 client.25267 trim_cache size 0 max 16384 2024-06-06T07:02:39.348 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:02:39.348 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "no active scrubs running", 2024-06-06T07:02:39.348 INFO:teuthology.orchestra.run.smithi073.stdout: "scrubs": {} 2024-06-06T07:02:39.348 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 msgr2=0x7f59400a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 0x7f59400a5470 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f59541998b0 tx=0x7f5944009260 comp rx=0 tx=0).stop 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 msgr2=0x7f5954197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f595000e9c0 tx=0x7f595000ecd0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 shutdown_connections 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f59400a2fb0 0x7f59400a5470 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.345+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59541073d0 0x7f59541981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.346+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5954101390 0x7f5954197750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.349 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.346+0000 7f595b302700 1 --2- 172.21.15.73:0/2839776147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5954101d30 0x7f5954197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:39.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.346+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 >> 172.21.15.73:0/2839776147 conn(0x7f59540782c0 msgr2=0x7f59541001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:39.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.346+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 shutdown_connections 2024-06-06T07:02:39.350 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:39.346+0000 7f595b302700 1 -- 172.21.15.73:0/2839776147 wait complete. 2024-06-06T07:02:39.362 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:39 smithi073 ceph-mon[32303]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:39.362 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:39 smithi073 ceph-mon[32303]: scrub summary: idle 2024-06-06T07:02:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:39 smithi136 ceph-mon[35901]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:39.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:39 smithi136 ceph-mon[35901]: scrub summary: idle 2024-06-06T07:02:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:39 smithi170 ceph-mon[36108]: pgmap v42: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:39 smithi170 ceph-mon[36108]: scrub summary: idle 2024-06-06T07:02:40.306 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:02:40.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:02:40.659 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:41 smithi073 ceph-mon[32303]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:41 smithi136 ceph-mon[35901]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:41 smithi170 ceph-mon[36108]: pgmap v43: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.707+0000 7fbcad6ff700 1 -- 172.21.15.73:0/706061393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca81085d0 msgr2=0x7fbca81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.707+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/706061393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca81085d0 0x7fbca81089b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fbc980099c0 tx=0x7fbc98009cd0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 -- 172.21.15.73:0/706061393 shutdown_connections 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/706061393 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102ef0 0x7fbca810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/706061393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca81085d0 0x7fbca81089b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/706061393 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8102530 0x7fbca81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 -- 172.21.15.73:0/706061393 >> 172.21.15.73:0/706061393 conn(0x7fbca8079640 msgr2=0x7fbca8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:42.709 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 -- 172.21.15.73:0/706061393 shutdown_connections 2024-06-06T07:02:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.708+0000 7fbcad6ff700 1 -- 172.21.15.73:0/706061393 wait complete. 2024-06-06T07:02:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbcad6ff700 1 Processor -- start 2024-06-06T07:02:42.710 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbcad6ff700 1 -- start start 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbcad6ff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102530 0x7fbca819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbcad6ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbcad6ff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8198ae0 0x7fbca8198f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.709+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48372/0 (socket says 172.21.15.73:48372) 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 -- 172.21.15.73:0/1126075550 learned_addr learned my addr 172.21.15.73:0/1126075550 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbca81126c0 con 0x7fbca8102ef0 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8112540 con 0x7fbca8198ae0 2024-06-06T07:02:42.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbca8112840 con 0x7fbca8102530 2024-06-06T07:02:42.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca77fe700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8198ae0 0x7fbca8198f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:42.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca6ffd700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102530 0x7fbca819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:42.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 -- 172.21.15.73:0/1126075550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102530 msgr2=0x7fbca819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102530 0x7fbca819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 -- 172.21.15.73:0/1126075550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8198ae0 msgr2=0x7fbca8198f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8198ae0 0x7fbca8198f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbca8199660 con 0x7fbca8102ef0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.710+0000 7fbca67fc700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c920 tx=0x7fbc9c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.711+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c011b60 con 0x7fbca8102ef0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.711+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc9c011d00 con 0x7fbca8102ef0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.711+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbca80746c0 con 0x7fbca8102ef0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.711+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9c005600 con 0x7fbca8102ef0 2024-06-06T07:02:42.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.712+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbca8074b40 con 0x7fbca8102ef0 2024-06-06T07:02:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.713+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbca810aa60 con 0x7fbca8102ef0 2024-06-06T07:02:42.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.713+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fbc9c004750 con 0x7fbca8102ef0 2024-06-06T07:02:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.714+0000 7fbc8ffff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 0x7fbc900a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:42.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.715+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fbc9c0d3580 con 0x7fbca8102ef0 2024-06-06T07:02:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.715+0000 7fbca6ffd700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 0x7fbc900a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:42.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.716+0000 7fbca6ffd700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 0x7fbc900a54c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc980098e0 tx=0x7fbc98005fb0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:42.723 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:42.722+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbc9c054850 con 0x7fbca8102ef0 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.197+0000 7ffa47e2c640 1 -- 172.21.15.73:0/1352710890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa401008d0 msgr2=0x7ffa40100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.197+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/1352710890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa401008d0 0x7ffa40100d50 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009930 tx=0x7ffa3002a420 comp rx=0 tx=0).stop 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 -- 172.21.15.73:0/1352710890 shutdown_connections 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/1352710890 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa40101290 0x7ffa4010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/1352710890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa401008d0 0x7ffa40100d50 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/1352710890 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40106890 0x7ffa40106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 -- 172.21.15.73:0/1352710890 >> 172.21.15.73:0/1352710890 conn(0x7ffa40077a20 msgr2=0x7ffa40077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 -- 172.21.15.73:0/1352710890 shutdown_connections 2024-06-06T07:02:43.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.198+0000 7ffa47e2c640 1 -- 172.21.15.73:0/1352710890 wait complete. 2024-06-06T07:02:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 Processor -- start 2024-06-06T07:02:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 -- start start 2024-06-06T07:02:43.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401008d0 0x7ffa40196dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 0x7ffa40197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40110ba0 con 0x7ffa40106890 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40110a20 con 0x7ffa40101290 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa47e2c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa40110d20 con 0x7ffa401008d0 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa463c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa463c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48394/0 (socket says 172.21.15.73:48394) 2024-06-06T07:02:43.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa463c9640 1 -- 172.21.15.73:0/3357637096 learned_addr learned my addr 172.21.15.73:0/3357637096 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa453c7640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 0x7ffa40197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401008d0 0x7ffa40196dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.199+0000 7ffa463c9640 1 -- 172.21.15.73:0/3357637096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401008d0 msgr2=0x7ffa40196dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa463c9640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401008d0 0x7ffa40196dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa463c9640 1 -- 172.21.15.73:0/3357637096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 msgr2=0x7ffa40197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa463c9640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 0x7ffa40197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa463c9640 1 -- 172.21.15.73:0/3357637096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4019bdb0 con 0x7ffa40106890 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa453c7640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 0x7ffa40197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa463c9640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800eaa0 tx=0x7ffa3800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa36ffd640 1 -- 172.21.15.73:0/3357637096 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800cdb0 con 0x7ffa40106890 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4019c0a0 con 0x7ffa40106890 2024-06-06T07:02:43.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa36ffd640 1 -- 172.21.15.73:0/3357637096 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa38005020 con 0x7ffa40106890 2024-06-06T07:02:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa36ffd640 1 -- 172.21.15.73:0/3357637096 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa380116d0 con 0x7ffa40106890 2024-06-06T07:02:43.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.200+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa40069050 con 0x7ffa40106890 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.203+0000 7ffa4ac0e740 1 Processor -- start 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.204+0000 7ffa36ffd640 1 -- 172.21.15.73:0/3357637096 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7ffa380051d0 con 0x7ffa40106890 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 -- start start 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cb747664f0 con 0x55cb74802170 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb74704870 con 0x55cb74802a90 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cb747624b0 con 0x55cb7480d350 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48406/0 (socket says 172.21.15.73:48406) 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa453c7640 1 -- 172.21.15.73:0/322715691 learned_addr learned my addr 172.21.15.73:0/322715691 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.205+0000 7ffa36ffd640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 0x7ffa2c0a5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa36ffd640 1 -- 172.21.15.73:0/3357637096 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa380cff30 con 0x7ffa40106890 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa463c9640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 0x7ffa2c0a5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 -- 172.21.15.73:0/322715691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 msgr2=0x55cb7480f740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480f740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 -- 172.21.15.73:0/322715691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 msgr2=0x55cb7480ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 -- 172.21.15.73:0/322715691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb747209b0 con 0x55cb74802170 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa453c7640 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009900 tx=0x7ffa30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d920d4dc906dcc72 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa463c9640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 0x7ffa2c0a5400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa380cd1b0 tx=0x7ffa380b0040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa2bfff640 1 -- 172.21.15.73:0/322715691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30035030 con 0x55cb74802170 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa2bfff640 1 -- 172.21.15.73:0/322715691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa30004b70 con 0x55cb74802170 2024-06-06T07:02:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.206+0000 7ffa2bfff640 1 -- 172.21.15.73:0/322715691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30031b20 con 0x55cb74802170 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.207+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/322715691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 msgr2=0x55cb74802550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.207+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009900 tx=0x7ffa30000bc0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.207+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/322715691 shutdown_connections 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/322715691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802170 0x55cb74802550 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/322715691 >> 172.21.15.73:0/322715691 conn(0x55cb747fe680 msgr2=0x55cb747fea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/322715691 shutdown_connections 2024-06-06T07:02:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/322715691 wait complete. 2024-06-06T07:02:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.207+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbca8199fd0 con 0x7fbca8102ef0 2024-06-06T07:02:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.208+0000 7ffa4ac0e740 1 -- start start 2024-06-06T07:02:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 Processor -- start 2024-06-06T07:02:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.209+0000 7fbc8ffff700 1 -- 172.21.15.73:0/1126075550 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11932 (secure 0 0 0) 0x7fbc9c09fcb0 con 0x7fbca8102ef0 2024-06-06T07:02:43.213 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:02:43.213 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T07:02:17.129247+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":26,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25076":{"gid":25076,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/887133051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":887133051},{"type":"v1","addr":"172.21.15.136:6835","nonce":887133051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24433,24419,24421,24415,24404]},"id":1}]} 2024-06-06T07:02:43.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb7480a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cb74814330 con 0x55cb74802a90 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cb746efae0 con 0x55cb74802170 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.209+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cb748144b0 con 0x55cb7480d350 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa4ac0e740 10 client.0 _ll_register_callbacks cb 0x55cb747289b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa45bc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb7480a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54634/0 (socket says 172.21.15.73:54634) 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 -- 172.21.15.73:0/256397880 learned_addr learned my addr 172.21.15.73:0/256397880 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 -- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 msgr2=0x55cb7480adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 -- 172.21.15.73:0/256397880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 msgr2=0x55cb7480a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb7480a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 -- 172.21.15.73:0/256397880 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb7447a210 con 0x55cb74802170 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb7480a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa453c7640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa30031800 tx=0x7ffa30031700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4138310a9b57d608 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.210+0000 7ffa297fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30048060 con 0x55cb74802170 2024-06-06T07:02:43.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa30031b40 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa30004b70 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa30057050 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa4ac0e740 10 client.25217 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb74546620 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa300048d0 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.211+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ffa30043400 con 0x55cb74802170 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 10 client.25217 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 10 client.25217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 20 client.25217 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb74810e00 0x55cb74813220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 4 client.25217 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:43.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ffa30004b70 con 0x55cb74810e00 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 msgr2=0x7fbc900a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 0x7fbc900a54c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbc980098e0 tx=0x7fbc98005fb0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 msgr2=0x7fbca819ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fbc9c00c920 tx=0x7fbc9c00cce0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 shutdown_connections 2024-06-06T07:02:43.219 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbc900a3000 0x7fbc900a54c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbca8102530 0x7fbca819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbca8198ae0 0x7fbca8198f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 --2- 172.21.15.73:0/1126075550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbca8102ef0 0x7fbca819ee60 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.220 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.212+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 >> 172.21.15.73:0/1126075550 conn(0x7fbca8079640 msgr2=0x7fbca8101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.212+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb74810e00 0x55cb74813220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.213+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 shutdown_connections 2024-06-06T07:02:43.221 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:43.213+0000 7fbcad6ff700 1 -- 172.21.15.73:0/1126075550 wait complete. 2024-06-06T07:02:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.213+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb74810e00 0x55cb74813220 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11dffa8b47f422d0 server_cookie=5d84faee94da3c9d in_seq=0 out_seq=0 2024-06-06T07:02:43.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.213+0000 7ffa297fa640 10 client.25217 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:43.221 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 32 2024-06-06T07:02:43.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa297fa640 1 -- 172.21.15.73:0/256397880 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ffa30004b70 con 0x55cb74810e00 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa297fa640 10 client.25217 handle_command_reply: tid=0 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 client.25217 shutdown 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 msgr2=0x55cb7480a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa30031800 tx=0x7ffa30031700 comp rx=0 tx=0).stop 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 shutdown_connections 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb74810e00 0x55cb74813220 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa297fa640 0 client.25217 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb7480d350 0x55cb7480adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb7480a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/256397880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb7480a330 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 >> 172.21.15.73:0/256397880 conn(0x55cb747fe680 msgr2=0x55cb74804400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 shutdown_connections 2024-06-06T07:02:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/256397880 wait complete. 2024-06-06T07:02:43.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.214+0000 7ffa4ac0e740 20 client.25217 trim_cache size 0 max 16384 2024-06-06T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.230+0000 7ffa4ac0e740 1 Processor -- start 2024-06-06T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.230+0000 7ffa4ac0e740 1 -- start start 2024-06-06T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74802170 0x55cb74804b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48428/0 (socket says 172.21.15.73:48428) 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 -- 172.21.15.73:0/882683942 learned_addr learned my addr 172.21.15.73:0/882683942 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401ac890 con 0x55cb7480d350 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401ae3d0 con 0x55cb74802a90 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401acf60 con 0x55cb74802170 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74802170 0x55cb74804b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 -- 172.21.15.73:0/882683942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74802170 msgr2=0x55cb74804b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74802170 0x55cb74804b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 -- 172.21.15.73:0/882683942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 msgr2=0x55cb7480abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa453c7640 1 -- 172.21.15.73:0/882683942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb747f8ae0 con 0x55cb7480d350 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.231+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.232+0000 7ffa453c7640 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffa30005820 tx=0x7ffa30000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e14b75aac05736 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.232+0000 7ffa2b7fe640 1 -- 172.21.15.73:0/882683942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30035070 con 0x55cb7480d350 2024-06-06T07:02:43.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.232+0000 7ffa2b7fe640 1 -- 172.21.15.73:0/882683942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa300050d0 con 0x55cb7480d350 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.232+0000 7ffa2b7fe640 1 -- 172.21.15.73:0/882683942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa30043400 con 0x55cb7480d350 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 msgr2=0x55cb747fd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffa30005820 tx=0x7ffa30000bc0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 shutdown_connections 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb7480d350 0x55cb747fd660 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802a90 0x55cb7480abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/882683942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74802170 0x55cb74804b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 >> 172.21.15.73:0/882683942 conn(0x55cb747fe680 msgr2=0x55cb747fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 shutdown_connections 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/882683942 wait complete. 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 -- start start 2024-06-06T07:02:43.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.233+0000 7ffa4ac0e740 1 Processor -- start 2024-06-06T07:02:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb74819700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74810e00 0x55cb748065a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401add00 con 0x55cb74802a90 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401ad630 con 0x55cb74802170 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa401aa870 con 0x55cb74810e00 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa4ac0e740 10 client.0 _ll_register_callbacks cb 0x7ffa3c0097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48442/0 (socket says 172.21.15.73:48442) 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 -- 172.21.15.73:0/2366644599 learned_addr learned my addr 172.21.15.73:0/2366644599 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa453c7640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74810e00 0x55cb748065a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 -- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74810e00 msgr2=0x55cb748065a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74810e00 0x55cb748065a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa453c7640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb74819700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 -- 172.21.15.73:0/2366644599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 msgr2=0x55cb74819700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb74819700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 -- 172.21.15.73:0/2366644599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb74808130 con 0x55cb74802a90 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa453c7640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb74819700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa45bc8640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55cb7480b900 tx=0x7ffa3c012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a588e936e8ed043d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.234+0000 7ffa297fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004230 con 0x55cb74802a90 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa3c0043d0 con 0x55cb74802a90 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa300050d0 con 0x55cb74802a90 2024-06-06T07:02:43.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c012e30 con 0x55cb74802a90 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa4ac0e740 10 client.15732 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cb748901f0 con 0x55cb74802a90 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.235+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c01fd70 con 0x55cb74802a90 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ffa3c003cb0 con 0x55cb74802a90 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 10 client.15732 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 10 client.15732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:43.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 20 client.15732 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb748909a0 0x55cb74892dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 4 client.15732 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7ffa300050d0 con 0x55cb748909a0 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.236+0000 7ffa453c7640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb748909a0 0x55cb74892dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.237+0000 7ffa453c7640 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb748909a0 0x55cb74892dc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30157d9ce57e5b01 server_cookie=674390e2ecadbcc9 in_seq=0 out_seq=0 2024-06-06T07:02:43.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.237+0000 7ffa297fa640 10 client.15732 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.241+0000 7ffa297fa640 1 -- 172.21.15.73:0/2366644599 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7ffa300050d0 con 0x55cb748909a0 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.241+0000 7ffa297fa640 10 client.15732 handle_command_reply: tid=0 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.241+0000 7ffa4ac0e740 1 client.15732 shutdown 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.241+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 msgr2=0x55cb74819c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.241+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x55cb7480b900 tx=0x7ffa3c012b20 comp rx=0 tx=0).stop 2024-06-06T07:02:43.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 shutdown_connections 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cb748909a0 0x55cb74892dc0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cb74810e00 0x55cb748065a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cb74802a90 0x55cb74819c40 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa297fa640 0 client.15732 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 --2- 172.21.15.73:0/2366644599 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cb74802170 0x55cb74819700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 >> 172.21.15.73:0/2366644599 conn(0x55cb747fe680 msgr2=0x55cb747fd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 shutdown_connections 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 1 -- 172.21.15.73:0/2366644599 wait complete. 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.242+0000 7ffa4ac0e740 20 client.15732 trim_cache size 0 max 16384 2024-06-06T07:02:43.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:02:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:02:43.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 msgr2=0x7ffa2c0a5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 0x7ffa2c0a5400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa380cd1b0 tx=0x7ffa380b0040 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 msgr2=0x7ffa4019b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800eaa0 tx=0x7ffa3800ef60 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 shutdown_connections 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa40106890 0x7ffa4019b680 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa40101290 0x7ffa40197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa2c0a2f40 0x7ffa2c0a5400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 --2- 172.21.15.73:0/3357637096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa401008d0 0x7ffa40196dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 >> 172.21.15.73:0/3357637096 conn(0x7ffa40077a20 msgr2=0x7ffa40103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 shutdown_connections 2024-06-06T07:02:43.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:43.243+0000 7ffa47e2c640 1 -- 172.21.15.73:0/3357637096 wait complete. 2024-06-06T07:02:43.451 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:43 smithi073 ceph-mon[32303]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:43 smithi136 ceph-mon[35901]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:43 smithi170 ceph-mon[36108]: pgmap v44: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-06T07:02:43.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-06T07:02:44.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:44 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1126075550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:44.293 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:44.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:44 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1126075550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:44.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:44 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1126075550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:02:45.463 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:45 smithi073 ceph-mon[32303]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:45 smithi136 ceph-mon[35901]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:45 smithi170 ceph-mon[36108]: pgmap v45: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 -- 172.21.15.73:0/430991373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d8101380 msgr2=0x7f70d8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 --2- 172.21.15.73:0/430991373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d8101380 0x7f70d8101800 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009990 tx=0x7f70c4009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 -- 172.21.15.73:0/430991373 shutdown_connections 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 --2- 172.21.15.73:0/430991373 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 --2- 172.21.15.73:0/430991373 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d81073f0 0x7f70d81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 --2- 172.21.15.73:0/430991373 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d8101380 0x7f70d8101800 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 -- 172.21.15.73:0/430991373 >> 172.21.15.73:0/430991373 conn(0x7f70d8078300 msgr2=0x7f70d8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 -- 172.21.15.73:0/430991373 shutdown_connections 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 -- 172.21.15.73:0/430991373 wait complete. 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.290+0000 7f70dedaa700 1 Processor -- start 2024-06-06T07:02:46.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 -- start start 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101380 0x7f70d8073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d806cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 0x7f70d806d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70d8112660 con 0x7f70d81073f0 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81124e0 con 0x7f70d8101380 2024-06-06T07:02:46.293 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dedaa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70d81127e0 con 0x7f70d8101d40 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d806cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dd347700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 0x7f70d806d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d806cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35974/0 (socket says 172.21.15.73:35974) 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70d7fff700 1 -- 172.21.15.73:0/3132098134 learned_addr learned my addr 172.21.15.73:0/3132098134 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.291+0000 7f70dcb46700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101380 0x7f70d8073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 -- 172.21.15.73:0/3132098134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 msgr2=0x7f70d806cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d806cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 -- 172.21.15.73:0/3132098134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101380 msgr2=0x7f70d8073c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101380 0x7f70d8073c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.294 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 -- 172.21.15.73:0/3132098134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d806d830 con 0x7f70d81073f0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dd347700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 0x7f70d806d240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b330 tx=0x7f70d000b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70d5ffb700 1 -- 172.21.15.73:0/3132098134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d000bdd0 con 0x7f70d81073f0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d806dac0 con 0x7f70d81073f0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.292+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d8071b00 con 0x7f70d81073f0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.293+0000 7f70d5ffb700 1 -- 172.21.15.73:0/3132098134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70d00090d0 con 0x7f70d81073f0 2024-06-06T07:02:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.293+0000 7f70d5ffb700 1 -- 172.21.15.73:0/3132098134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d0019670 con 0x7f70d81073f0 2024-06-06T07:02:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.296+0000 7f70d5ffb700 1 -- 172.21.15.73:0/3132098134 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f70d0020020 con 0x7f70d81073f0 2024-06-06T07:02:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.296+0000 7f70e962db80 1 Processor -- start 2024-06-06T07:02:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 -- start start 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70d5ffb700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 0x7f70c80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fd39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 0x561c74fd4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561c74f25810 con 0x561c74fd9640 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c74ecd720 con 0x561c74fd9f60 2024-06-06T07:02:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70e962db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c74f217d0 con 0x561c74fd3f20 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70d5ffb700 1 -- 172.21.15.73:0/3132098134 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f70d00d2fe0 con 0x7f70d81073f0 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70dcb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70dcb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44928/0 (socket says 172.21.15.73:44928) 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70dcb46700 1 -- 172.21.15.73:0/1977188572 learned_addr learned my addr 172.21.15.73:0/1977188572 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70dd347700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 0x561c74fd4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.297+0000 7f70d7fff700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 0x7f70c80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70d7fff700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fd39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 -- 172.21.15.73:0/1977188572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 msgr2=0x561c74fd39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fd39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 -- 172.21.15.73:0/1977188572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 msgr2=0x561c74fd9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 -- 172.21.15.73:0/1977188572 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c74ec3f50 con 0x561c74fd3f20 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dcb46700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.298+0000 7f70dd347700 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 0x561c74fd4340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70d00d01a0 tx=0x7f70d009b250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c04cd06ce4e99e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.299+0000 7f70c2ffd700 1 -- 172.21.15.73:0/1977188572 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d009eea0 con 0x561c74fd3f20 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.299+0000 7f70d7fff700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 0x7f70c80a5470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b780 tx=0x7f70c400b680 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.299+0000 7f70c2ffd700 1 -- 172.21.15.73:0/1977188572 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70d00a3250 con 0x561c74fd3f20 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.299+0000 7f70c2ffd700 1 -- 172.21.15.73:0/1977188572 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70d00a78e0 con 0x561c74fd3f20 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- 172.21.15.73:0/1977188572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 msgr2=0x561c74fd4340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 0x561c74fd4340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f70d00d01a0 tx=0x7f70d009b250 comp rx=0 tx=0).stop 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- 172.21.15.73:0/1977188572 shutdown_connections 2024-06-06T07:02:46.302 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd3f20 0x561c74fd4340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fd9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 --2- 172.21.15.73:0/1977188572 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fd39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- 172.21.15.73:0/1977188572 >> 172.21.15.73:0/1977188572 conn(0x561c74efecc0 msgr2=0x561c74fce850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- 172.21.15.73:0/1977188572 shutdown_connections 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- 172.21.15.73:0/1977188572 wait complete. 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 -- start start 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.300+0000 7f70e962db80 1 Processor -- start 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 0x561c74fdfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9f60 0x561c74fdd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561c74fe09d0 con 0x561c74fd9640 2024-06-06T07:02:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x561c74ec5bf0 con 0x561c74fd3f20 2024-06-06T07:02:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561c74fe0b50 con 0x561c74fd9f60 2024-06-06T07:02:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70e962db80 10 client.0 _ll_register_callbacks cb 0x561c74ebf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44940/0 (socket says 172.21.15.73:44940) 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70d7fff700 1 -- 172.21.15.73:0/972306988 learned_addr learned my addr 172.21.15.73:0/972306988 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70dcb46700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 0x561c74fdfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.301+0000 7f70dd347700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9f60 0x561c74fdd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 -- 172.21.15.73:0/972306988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9f60 msgr2=0x561c74fdd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9f60 0x561c74fdd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 -- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 msgr2=0x561c74fdfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 0x561c74fdfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 -- 172.21.15.73:0/972306988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c74e748d0 con 0x561c74fd9640 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70dcb46700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 0x561c74fdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70d7fff700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c400b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ea5bfcb0d215ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.305 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c402e070 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f70c400f460 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70d00a3250 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f70c4021040 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70e962db80 10 client.15744 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.302+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c74feb300 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70d00a3250 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f70c402a710 con 0x561c74fd9640 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 10 client.15744 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:46.306 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 10 client.15744 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:02:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 20 client.15744 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c74fed9e0 0x561c74fefe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 4 client.15744 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.303+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c74f32d10 con 0x561c74fed9e0 2024-06-06T07:02:46.307 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.304+0000 7f70dcb46700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c74fed9e0 0x561c74fefe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.304+0000 7f70b7fff700 10 client.15744 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.304+0000 7f70dcb46700 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c74fed9e0 0x561c74fefe60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2e12c1cd6a98e076 server_cookie=f420276b057ab0e in_seq=0 out_seq=0 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70b7fff700 1 -- 172.21.15.73:0/972306988 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c74f32d10 con 0x561c74fed9e0 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70b7fff700 10 client.15744 handle_command_reply: tid=0 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 client.15744 shutdown 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 msgr2=0x561c74fdcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f70c40098e0 tx=0x7f70c400b610 comp rx=0 tx=0).stop 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 shutdown_connections 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c74fed9e0 0x561c74fefe60 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9f60 0x561c74fdd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.308 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd3f20 0x561c74fdfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 --2- 172.21.15.73:0/972306988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9640 0x561c74fdcfc0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 >> 172.21.15.73:0/972306988 conn(0x561c74efecc0 msgr2=0x561c74fd29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70b7fff700 0 client.15744 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 shutdown_connections 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 1 -- 172.21.15.73:0/972306988 wait complete. 2024-06-06T07:02:46.309 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.305+0000 7f70e962db80 20 client.15744 trim_cache size 0 max 16384 2024-06-06T07:02:46.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.334+0000 7f70e962db80 1 Processor -- start 2024-06-06T07:02:46.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.334+0000 7f70e962db80 1 -- start start 2024-06-06T07:02:46.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.334+0000 7f70e962db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.334+0000 7f70e962db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fdc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70e962db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fdaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70e962db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc00edb0 con 0x561c74fd3f20 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70e962db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc0108f0 con 0x561c74fd9f60 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70e962db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc00f480 con 0x561c74fd9640 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44942/0 (socket says 172.21.15.73:44942) 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 -- 172.21.15.73:0/1349830401 learned_addr learned my addr 172.21.15.73:0/1349830401 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dd347700 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fdaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 -- 172.21.15.73:0/1349830401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 msgr2=0x561c74fdc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.336 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fdc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 -- 172.21.15.73:0/1349830401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 msgr2=0x561c74fdaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fdaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 -- 172.21.15.73:0/1349830401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c74ff04e0 con 0x561c74fd3f20 2024-06-06T07:02:46.337 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dd347700 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fdaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.335+0000 7f70dcb46700 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x561c74ff06e0 tx=0x7f70cc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=629ed4574893938d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.336+0000 7f70c27fc700 1 -- 172.21.15.73:0/1349830401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc003ce0 con 0x561c74fd3f20 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.336+0000 7f70c27fc700 1 -- 172.21.15.73:0/1349830401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70cc003e80 con 0x561c74fd3f20 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.336+0000 7f70c27fc700 1 -- 172.21.15.73:0/1349830401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70cc011440 con 0x561c74fd3f20 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- 172.21.15.73:0/1349830401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 msgr2=0x561c74fd39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x561c74ff06e0 tx=0x7f70cc00b410 comp rx=0 tx=0).stop 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- 172.21.15.73:0/1349830401 shutdown_connections 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74fd9f60 0x561c74fdaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd3f20 0x561c74fd39e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 --2- 172.21.15.73:0/1349830401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fdc8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- 172.21.15.73:0/1349830401 >> 172.21.15.73:0/1349830401 conn(0x561c74efecc0 msgr2=0x561c74fda340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- 172.21.15.73:0/1349830401 shutdown_connections 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- 172.21.15.73:0/1349830401 wait complete. 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 -- start start 2024-06-06T07:02:46.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.337+0000 7f70e962db80 1 Processor -- start 2024-06-06T07:02:46.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fe6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 0x561c74ff5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc010220 con 0x561c74fd9f60 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc00fb50 con 0x561c74ff5480 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f70bc00cd50 con 0x561c74fd9640 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44950/0 (socket says 172.21.15.73:44950) 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70d7fff700 1 -- 172.21.15.73:0/3543769157 learned_addr learned my addr 172.21.15.73:0/3543769157 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70dcb46700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fe6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.338+0000 7f70e962db80 10 client.0 _ll_register_callbacks cb 0x561c74ebf920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70dd347700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 0x561c74ff5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 -- 172.21.15.73:0/3543769157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 msgr2=0x561c74fe6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fe6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 -- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 msgr2=0x561c74ff5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 0x561c74ff5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 -- 172.21.15.73:0/3543769157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c75057790 con 0x561c74fd9f60 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70dd347700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 0x561c74ff5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70d7fff700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f70c402ad20 tx=0x7f70c402aa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559c55e1bf24d7be server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c402e040 con 0x561c74fd9f60 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f70c400f460 con 0x561c74fd9f60 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.339+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561c74f2d290 con 0x561c74fd9f60 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f70c400f6a0 con 0x561c74fd9f60 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 10 client.15750 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c75058980 con 0x561c74fd9f60 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561c74f2d290 con 0x561c74fd9f60 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f70c40338d0 con 0x561c74fd9f60 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 10 client.15750 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 10 client.15750 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 20 client.15750 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c75059080 0x561c7505b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 4 client.15750 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.340+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c74da7250 con 0x561c75059080 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70dcb46700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c75059080 0x561c7505b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70b7fff700 10 client.15750 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70dcb46700 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c75059080 0x561c7505b4a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d3300ab4a160deca server_cookie=773471aecb572c1b in_seq=0 out_seq=0 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70b7fff700 1 -- 172.21.15.73:0/3543769157 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c74da7250 con 0x561c75059080 2024-06-06T07:02:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70b7fff700 10 client.15750 handle_command_reply: tid=0 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70e962db80 1 client.15750 shutdown 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 msgr2=0x561c74ff4f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f70c402ad20 tx=0x7f70c402aa40 comp rx=0 tx=0).stop 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 shutdown_connections 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.341+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x561c75059080 0x561c7505b4a0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561c74fd9640 0x561c74fe6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70b7fff700 0 client.15750 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561c74ff5480 0x561c74ff5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 --2- 172.21.15.73:0/3543769157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561c74fd9f60 0x561c74ff4f40 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 >> 172.21.15.73:0/3543769157 conn(0x561c74efecc0 msgr2=0x561c74fda340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 shutdown_connections 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 1 -- 172.21.15.73:0/3543769157 wait complete. 2024-06-06T07:02:46.345 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.342+0000 7f70e962db80 20 client.15750 trim_cache size 0 max 16384 2024-06-06T07:02:46.346 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:46.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 msgr2=0x7f70c80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.346 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 0x7f70c80a5470 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f70c400b780 tx=0x7f70c400b680 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 msgr2=0x7f70d806d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 0x7f70d806d240 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f70d000b330 tx=0x7f70d000b640 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 shutdown_connections 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70c80a2fb0 0x7f70c80a5470 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70d81073f0 0x7f70d806d240 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f70d8101380 0x7f70d8073c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 --2- 172.21.15.73:0/3132098134 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f70d8101d40 0x7f70d806cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 >> 172.21.15.73:0/3132098134 conn(0x7f70d8078300 msgr2=0x7f70d80fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 shutdown_connections 2024-06-06T07:02:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:46.343+0000 7f70dedaa700 1 -- 172.21.15.73:0/3132098134 wait complete. 2024-06-06T07:02:47.161 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-06T07:02:47.436 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:47 smithi073 ceph-mon[32303]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T07:02:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:47 smithi136 ceph-mon[35901]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T07:02:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:47 smithi170 ceph-mon[36108]: pgmap v46: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 9 op/s 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2686504555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101d40 msgr2=0x7fdb9810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2686504555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101d40 0x7fdb9810e230 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000b0a0 tx=0x7fdb9000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2686504555 shutdown_connections 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2686504555 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb981073f0 0x7fdb981077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2686504555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101d40 0x7fdb9810e230 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2686504555 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb98101380 0x7fdb98101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.388 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2686504555 >> 172.21.15.73:0/2686504555 conn(0x7fdb98078300 msgr2=0x7fdb98078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2686504555 shutdown_connections 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.339+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2686504555 wait complete. 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 Processor -- start 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 -- start start 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb98101d40 0x7fdb98199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 0x7fdb981a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98111510 con 0x7fdb98101380 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98111390 con 0x7fdb981073f0 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9eaca700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb98111690 con 0x7fdb98101d40 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.389 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44962/0 (socket says 172.21.15.73:44962) 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 -- 172.21.15.73:0/2750335692 learned_addr learned my addr 172.21.15.73:0/2750335692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb97fff700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb98101d40 0x7fdb98199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 -- 172.21.15.73:0/2750335692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb98101d40 msgr2=0x7fdb98199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb98101d40 0x7fdb98199f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9d067700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 0x7fdb981a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 -- 172.21.15.73:0/2750335692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 msgr2=0x7fdb981a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 0x7fdb981a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9c866700 1 -- 172.21.15.73:0/2750335692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb981a4fb0 con 0x7fdb98101380 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.340+0000 7fdb9d067700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 0x7fdb981a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d950 tx=0x7fdb8c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb95ffb700 1 -- 172.21.15.73:0/2750335692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c016070 con 0x7fdb98101380 2024-06-06T07:02:49.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb95ffb700 1 -- 172.21.15.73:0/2750335692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c009df0 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb981a5180 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb95ffb700 1 -- 172.21.15.73:0/2750335692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c00f3c0 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.341+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb981a56c0 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.343+0000 7fdb95ffb700 1 -- 172.21.15.73:0/2750335692 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7fdb8c019070 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.344+0000 7fdb95ffb700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 0x7fdb880a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.344+0000 7fdb95ffb700 1 -- 172.21.15.73:0/2750335692 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb8c0cf610 con 0x7fdb98101380 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.344+0000 7fdb97fff700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 0x7fdb880a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.344+0000 7fdb97fff700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 0x7fdb880a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb98078920 tx=0x7fdb8400b410 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 Processor -- start 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 -- start start 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068a4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5573067f5810 con 0x5573068a9640 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55730679d720 con 0x5573068a3f20 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdba934db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5573067f17d0 con 0x5573068a9f60 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44972/0 (socket says 172.21.15.73:44972) 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdb9c866700 1 -- 172.21.15.73:0/3327041481 learned_addr learned my addr 172.21.15.73:0/3327041481 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.345+0000 7fdb97fff700 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 -- 172.21.15.73:0/3327041481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 msgr2=0x5573068a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 -- 172.21.15.73:0/3327041481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 msgr2=0x5573068a4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068a4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 -- 172.21.15.73:0/3327041481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557306793f50 con 0x5573068a9640 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c09b230 tx=0x7fdb8c010880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dba85676a30e357 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb82ffd700 1 -- 172.21.15.73:0/3327041481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0bb050 con 0x5573068a9640 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb82ffd700 1 -- 172.21.15.73:0/3327041481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c0a41a0 con 0x5573068a9640 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.346+0000 7fdb82ffd700 1 -- 172.21.15.73:0/3327041481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c020070 con 0x5573068a9640 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- 172.21.15.73:0/3327041481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 msgr2=0x5573068a9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c09b230 tx=0x7fdb8c010880 comp rx=0 tx=0).stop 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- 172.21.15.73:0/3327041481 shutdown_connections 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068a4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068a9a20 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 --2- 172.21.15.73:0/3327041481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- 172.21.15.73:0/3327041481 >> 172.21.15.73:0/3327041481 conn(0x5573067cecc0 msgr2=0x55730689e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- 172.21.15.73:0/3327041481 shutdown_connections 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- 172.21.15.73:0/3327041481 wait complete. 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 -- start start 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.347+0000 7fdba934db80 1 Processor -- start 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 0x5573068afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9640 0x5573068ad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5573068b0910 con 0x5573068a9f60 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557306795bf0 con 0x5573068a9640 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5573068b0a90 con 0x5573068a3f20 2024-06-06T07:02:49.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 10 client.0 _ll_register_callbacks cb 0x55730678f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44982/0 (socket says 172.21.15.73:44982) 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 0x5573068afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 -- 172.21.15.73:0/2988134742 learned_addr learned my addr 172.21.15.73:0/2988134742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb97fff700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9640 0x5573068ad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 -- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 msgr2=0x5573068afd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 0x5573068afd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 -- 172.21.15.73:0/2988134742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9640 msgr2=0x5573068ad060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9640 0x5573068ad060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 -- 172.21.15.73:0/2988134742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573067448d0 con 0x5573068a9f60 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 0x5573068afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:49.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb9d067700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000b070 tx=0x7fdb90009dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11666eea72995df server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb77fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb90020030 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb900074e0 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.348+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8c0a41a0 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb90004e40 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 10 client.15762 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5573068bb2c0 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c0a41a0 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb90011070 con 0x5573068a9f60 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 10 client.15762 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 10 client.15762 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:02:49.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 20 client.15762 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573068bd8a0 0x5573068bfce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 4 client.15762 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.349+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557306802d10 con 0x5573068bd8a0 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.350+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573068bd8a0 0x5573068bfce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.350+0000 7fdb9c866700 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573068bd8a0 0x5573068bfce0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8af0f8ab7920def server_cookie=ad5f348dd56c9a13 in_seq=0 out_seq=0 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.350+0000 7fdb77fff700 10 client.15762 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdb77fff700 1 -- 172.21.15.73:0/2988134742 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557306802d10 con 0x5573068bd8a0 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdb77fff700 10 client.15762 handle_command_reply: tid=0 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 client.15762 shutdown 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 msgr2=0x5573068ad5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fdb9000b070 tx=0x7fdb90009dd0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 shutdown_connections 2024-06-06T07:02:49.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573068bd8a0 0x5573068bfce0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a3f20 0x5573068afd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9f60 0x5573068ad5a0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdb77fff700 0 client.15762 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 --2- 172.21.15.73:0/2988134742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9640 0x5573068ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 >> 172.21.15.73:0/2988134742 conn(0x5573067cecc0 msgr2=0x5573068a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 shutdown_connections 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 1 -- 172.21.15.73:0/2988134742 wait complete. 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.351+0000 7fdba934db80 20 client.15762 trim_cache size 0 max 16384 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.366+0000 7fdba934db80 1 Processor -- start 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.366+0000 7fdba934db80 1 -- start start 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 0x5573068a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.398 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9f60 0x5573068a7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981ae5e0 con 0x5573068a3f20 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981b0120 con 0x5573068a9f60 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdba934db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981aecb0 con 0x5573068a9640 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44984/0 (socket says 172.21.15.73:44984) 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 -- 172.21.15.73:0/4155443704 learned_addr learned my addr 172.21.15.73:0/4155443704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9d067700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9f60 0x5573068a7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb97fff700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 0x5573068a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 -- 172.21.15.73:0/4155443704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 msgr2=0x5573068a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 0x5573068a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 -- 172.21.15.73:0/4155443704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9f60 msgr2=0x5573068a7730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.399 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9f60 0x5573068a7730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 -- 172.21.15.73:0/4155443704 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55730689e380 con 0x5573068a3f20 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb97fff700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 0x5573068a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb9c866700 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ccfb0 tx=0x7fdb8c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d6ae74295683e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb827fc700 1 -- 172.21.15.73:0/4155443704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0bb040 con 0x5573068a3f20 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb827fc700 1 -- 172.21.15.73:0/4155443704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c0cf440 con 0x5573068a3f20 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.367+0000 7fdb827fc700 1 -- 172.21.15.73:0/4155443704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c020040 con 0x5573068a3f20 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- 172.21.15.73:0/4155443704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 msgr2=0x5573068b9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0ccfb0 tx=0x7fdb8c086040 comp rx=0 tx=0).stop 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- 172.21.15.73:0/4155443704 shutdown_connections 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a9f60 0x5573068a7730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a3f20 0x5573068b9bb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 --2- 172.21.15.73:0/4155443704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9640 0x5573068a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.400 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- 172.21.15.73:0/4155443704 >> 172.21.15.73:0/4155443704 conn(0x5573067cecc0 msgr2=0x5573068aa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- 172.21.15.73:0/4155443704 shutdown_connections 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- 172.21.15.73:0/4155443704 wait complete. 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.368+0000 7fdba934db80 1 -- start start 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 Processor -- start 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068b6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068c5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981afa50 con 0x5573068a9640 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981af380 con 0x5573068a3f20 2024-06-06T07:02:49.401 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb981ac580 con 0x5573068a9f60 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdba934db80 10 client.0 _ll_register_callbacks cb 0x55730678f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdb97fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdb97fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44986/0 (socket says 172.21.15.73:44986) 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdb97fff700 1 -- 172.21.15.73:0/3344547556 learned_addr learned my addr 172.21.15.73:0/3344547556 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.369+0000 7fdb9d067700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 -- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 msgr2=0x5573068c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 -- 172.21.15.73:0/3344547556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 msgr2=0x5573068b6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068b6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 -- 172.21.15.73:0/3344547556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573068c5680 con 0x5573068a9640 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb97fff700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdb9806b520 tx=0x7fdb840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39449d62aa5fe5a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84029480 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb84029620 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb8c0cf440 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdb84032050 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 10 client.15768 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557306928260 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c0cf440 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fdb840118c0 con 0x5573068a9640 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 10 client.15768 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 10 client.15768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.370+0000 7fdba934db80 20 client.15768 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573069289d0 0x55730692adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 4 client.15768 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:49.403 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5573067fd290 con 0x5573069289d0 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573069289d0 0x55730692adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdb77fff700 10 client.15768 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdb9c866700 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573069289d0 0x55730692adf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=54127e8d441c5a3d server_cookie=c463d7655c184c21 in_seq=0 out_seq=0 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdb77fff700 1 -- 172.21.15.73:0/3344547556 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5573067fd290 con 0x5573069289d0 2024-06-06T07:02:49.404 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdb77fff700 10 client.15768 handle_command_reply: tid=0 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 1 client.15768 shutdown 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 msgr2=0x5573068c4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.371+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdb9806b520 tx=0x7fdb840047c0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 shutdown_connections 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x5573069289d0 0x55730692adf0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5573068a9f60 0x5573068c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5573068a3f20 0x5573068b6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdb77fff700 0 client.15768 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:02:49.405 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 --2- 172.21.15.73:0/3344547556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5573068a9640 0x5573068c4c00 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 >> 172.21.15.73:0/3344547556 conn(0x5573067cecc0 msgr2=0x5573068aabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 shutdown_connections 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 1 -- 172.21.15.73:0/3344547556 wait complete. 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.372+0000 7fdba934db80 20 client.15768 trim_cache size 0 max 16384 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 msgr2=0x7fdb880a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 0x7fdb880a5370 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdb98078920 tx=0x7fdb8400b410 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 msgr2=0x7fdb981999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c00d950 tx=0x7fdb8c00dc60 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 shutdown_connections 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2eb0 0x7fdb880a5370 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb981073f0 0x7fdb981a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb98101380 0x7fdb981999f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.406 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 --2- 172.21.15.73:0/2750335692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb98101d40 0x7fdb98199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:49.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 >> 172.21.15.73:0/2750335692 conn(0x7fdb98078300 msgr2=0x7fdb980fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:49.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 shutdown_connections 2024-06-06T07:02:49.407 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:49.373+0000 7fdb9eaca700 1 -- 172.21.15.73:0/2750335692 wait complete. 2024-06-06T07:02:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:49 smithi073 ceph-mon[32303]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:49 smithi136 ceph-mon[35901]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:49 smithi170 ceph-mon[36108]: pgmap v47: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 255 B/s wr, 13 op/s 2024-06-06T07:02:50.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:02:50.658 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:Note: switching to '3d746ca5346f34eef7e826a1eb71e59e02b682ae'. 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:state without impacting any branches by switching back to a branch. 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-06T07:02:50.948 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: git switch -c 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr:Or undo this operation with: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: git switch - 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr: 2024-06-06T07:02:50.949 INFO:tasks.workunit.client.0.smithi073.stderr:HEAD is now at 3d746ca5346 Merge pull request #57870 from zdover23/wip-doc-2024-06-05-backport-57867-to-squid 2024-06-06T07:02:50.953 DEBUG:teuthology.orchestra.run.smithi073:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-06T07:02:50.991 INFO:tasks.workunit.client.0.smithi073.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-06T07:02:50.993 INFO:tasks.workunit.client.0.smithi073.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T07:02:50.993 INFO:tasks.workunit.client.0.smithi073.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-06T07:02:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:51 smithi073 ceph-mon[32303]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:51 smithi136 ceph-mon[35901]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:51 smithi170 ceph-mon[36108]: pgmap v48: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:52.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 -- 172.21.15.73:0/1713022974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06aa00 msgr2=0x7f52bc06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 --2- 172.21.15.73:0/1713022974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06aa00 0x7f52bc06ae80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009990 tx=0x7f52ac009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 -- 172.21.15.73:0/1713022974 shutdown_connections 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 --2- 172.21.15.73:0/1713022974 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc06b3c0 0x7f52bc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 --2- 172.21.15.73:0/1713022974 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc1084f0 0x7f52bc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 --2- 172.21.15.73:0/1713022974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06aa00 0x7f52bc06ae80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 -- 172.21.15.73:0/1713022974 >> 172.21.15.73:0/1713022974 conn(0x7f52bc078300 msgr2=0x7f52bc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 -- 172.21.15.73:0/1713022974 shutdown_connections 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.676+0000 7f52c310d700 1 -- 172.21.15.73:0/1713022974 wait complete. 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 Processor -- start 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 -- start start 2024-06-06T07:02:52.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 0x7f52bc101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc1084f0 0x7f52bc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc112600 con 0x7f52bc06b3c0 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc112480 con 0x7f52bc06aa00 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c310d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc112780 con 0x7f52bc1084f0 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45006/0 (socket says 172.21.15.73:45006) 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 -- 172.21.15.73:0/1424065329 learned_addr learned my addr 172.21.15.73:0/1424065329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c16aa700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc1084f0 0x7f52bc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 0x7f52bc101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 -- 172.21.15.73:0/1424065329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc1084f0 msgr2=0x7f52bc101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc1084f0 0x7f52bc101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 -- 172.21.15.73:0/1424065329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 msgr2=0x7f52bc101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 0x7f52bc101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52b3fff700 1 -- 172.21.15.73:0/1424065329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52bc1a7080 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.677+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 0x7f52bc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52b3fff700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52b1ffb700 1 -- 172.21.15.73:0/1424065329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac027070 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52b1ffb700 1 -- 172.21.15.73:0/1424065329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52ac00bdc0 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52bc1a7310 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52b1ffb700 1 -- 172.21.15.73:0/1424065329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac022d30 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.678+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52bc1a7760 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.680+0000 7f52b1ffb700 1 -- 172.21.15.73:0/1424065329 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) ==== 138066+0+0 (secure 0 0 0) 0x7f52ac02a070 con 0x7f52bc06b3c0 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.681+0000 7f52b1ffb700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 0x7f52a40a5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.681+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 0x7f52a40a5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.681+0000 7f52b1ffb700 1 -- 172.21.15.73:0/1424065329 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f52ac0d0860 con 0x7f52bc06b3c0 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 0x7f52a40a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009c50 tx=0x7f52b8009040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 Processor -- start 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 -- start start 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 0x55fc3ecdc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ec2e810 con 0x55fc3ecdcf20 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ebd6720 con 0x55fc3ece2f60 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52cd990b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ec2a7d0 con 0x55fc3ece2640 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45008/0 (socket says 172.21.15.73:45008) 2024-06-06T07:02:52.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 -- 172.21.15.73:0/2008244475 learned_addr learned my addr 172.21.15.73:0/2008244475 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52b3fff700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 0x55fc3ecdc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 -- 172.21.15.73:0/2008244475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 msgr2=0x55fc3ece2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 -- 172.21.15.73:0/2008244475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 msgr2=0x55fc3ecdc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 0x55fc3ecdc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52c16aa700 1 -- 172.21.15.73:0/2008244475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3ebccf50 con 0x55fc3ecdcf20 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.682+0000 7f52b3fff700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 0x55fc3ecdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b6c0 tx=0x7f52b400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d4512ac8ea3a97 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52a2ffd700 1 -- 172.21.15.73:0/2008244475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4004a30 con 0x55fc3ecdcf20 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52a2ffd700 1 -- 172.21.15.73:0/2008244475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b4015070 con 0x55fc3ecdcf20 2024-06-06T07:02:52.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52a2ffd700 1 -- 172.21.15.73:0/2008244475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4003980 con 0x55fc3ecdcf20 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52cd990b80 1 -- 172.21.15.73:0/2008244475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 msgr2=0x55fc3ecdd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.683+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f52b400b6c0 tx=0x7f52b400b9d0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- 172.21.15.73:0/2008244475 shutdown_connections 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdd340 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2008244475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2f60 0x55fc3ecdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- 172.21.15.73:0/2008244475 >> 172.21.15.73:0/2008244475 conn(0x55fc3ec07cc0 msgr2=0x55fc3ecd7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- 172.21.15.73:0/2008244475 shutdown_connections 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- 172.21.15.73:0/2008244475 wait complete. 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- start start 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 Processor -- start 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ecdcf20 0x55fc3ece8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 0x55fc3ece5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ece99d0 con 0x55fc3ece2f60 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ebcebf0 con 0x55fc3ece2640 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fc3ece9b50 con 0x55fc3ecdcf20 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.684+0000 7f52cd990b80 10 client.0 _ll_register_callbacks cb 0x55fc3ebc8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 0x55fc3ece5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 0x55fc3ece5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:41146/0 (socket says 172.21.15.73:41146) 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45018/0 (socket says 172.21.15.73:45018) 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52b3fff700 1 -- 172.21.15.73:0/2817400598 learned_addr learned my addr 172.21.15.73:0/2817400598 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ecdcf20 0x55fc3ece8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 -- 172.21.15.73:0/2817400598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ecdcf20 msgr2=0x55fc3ece8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ecdcf20 0x55fc3ece8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 -- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 msgr2=0x55fc3ece5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 0x55fc3ece5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 -- 172.21.15.73:0/2817400598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3eb7d8d0 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52c16aa700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004e20 tx=0x7f52b4004ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138fb66331dff8fb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f529bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4028060 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f52b4005090 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4015070 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52b402f050 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52cd990b80 10 client.15780 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.685+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc3ecf4240 con 0x55fc3ece2f60 2024-06-06T07:02:52.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b4015070 con 0x55fc3ece2f60 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f52b4011440 con 0x55fc3ece2f60 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 10 client.15780 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 10 client.15780 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 20 client.15780 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ecf67c0 0x55fc3ecf8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 4 client.15780 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc3ec3bd10 con 0x55fc3ecf67c0 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.686+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ecf67c0 0x55fc3ecf8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f529bfff700 10 client.15780 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ecf67c0 0x55fc3ecf8bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e395b347642981b server_cookie=dce167af313c05e in_seq=0 out_seq=0 2024-06-06T07:02:52.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f529bfff700 1 -- 172.21.15.73:0/2817400598 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc3ec3bd10 con 0x55fc3ecf67c0 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f529bfff700 10 client.15780 handle_command_reply: tid=0 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 client.15780 shutdown 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 msgr2=0x55fc3ece6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004e20 tx=0x7f52b4004ae0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 shutdown_connections 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ecf67c0 0x55fc3ecf8bb0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ecdcf20 0x55fc3ece8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece6510 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f529bfff700 0 client.15780 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 --2- 172.21.15.73:0/2817400598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ece2640 0x55fc3ece5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.687+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 >> 172.21.15.73:0/2817400598 conn(0x55fc3ec07cc0 msgr2=0x55fc3ecdb9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.688+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 shutdown_connections 2024-06-06T07:02:52.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.688+0000 7f52cd990b80 1 -- 172.21.15.73:0/2817400598 wait complete. 2024-06-06T07:02:52.700 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.688+0000 7f52cd990b80 20 client.15780 trim_cache size 0 max 16384 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 Processor -- start 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 -- start start 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b06b0 con 0x55fc3ece2f60 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b21f0 con 0x55fc3ecdcf20 2024-06-06T07:02:52.705 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52cd990b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b0d80 con 0x55fc3ece2640 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52c16aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45024/0 (socket says 172.21.15.73:45024) 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52c16aa700 1 -- 172.21.15.73:0/1153210369 learned_addr learned my addr 172.21.15.73:0/1153210369 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.703+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52b3fff700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 -- 172.21.15.73:0/1153210369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 msgr2=0x55fc3ece48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.706 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 -- 172.21.15.73:0/1153210369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 msgr2=0x55fc3ecdc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 -- 172.21.15.73:0/1153210369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc3ecd8520 con 0x55fc3ece2f60 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52c16aa700 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004640 tx=0x7f52b400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bc0a6203ad77f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52a27fc700 1 -- 172.21.15.73:0/1153210369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4028070 con 0x55fc3ece2f60 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52a27fc700 1 -- 172.21.15.73:0/1153210369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b4015070 con 0x55fc3ece2f60 2024-06-06T07:02:52.707 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.704+0000 7f52a27fc700 1 -- 172.21.15.73:0/1153210369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b4003740 con 0x55fc3ece2f60 2024-06-06T07:02:52.711 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- 172.21.15.73:0/1153210369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 msgr2=0x55fc3ece4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f52b4004640 tx=0x7f52b400b5b0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- 172.21.15.73:0/1153210369 shutdown_connections 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2f60 0x55fc3ece4e00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecdc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 --2- 172.21.15.73:0/1153210369 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2640 0x55fc3ece48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- 172.21.15.73:0/1153210369 >> 172.21.15.73:0/1153210369 conn(0x55fc3ec07cc0 msgr2=0x55fc3ece00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- 172.21.15.73:0/1153210369 shutdown_connections 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- 172.21.15.73:0/1153210369 wait complete. 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 -- start start 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.705+0000 7f52cd990b80 1 Processor -- start 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecf9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.712 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2f60 0x55fc3ed5f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b1b20 con 0x55fc3ece2640 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1b1450 con 0x55fc3ecdcf20 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f52bc1ae650 con 0x55fc3ece2f60 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 10 client.0 _ll_register_callbacks cb 0x55fc3ebc8920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52c0ea9700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecf9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45036/0 (socket says 172.21.15.73:45036) 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 -- 172.21.15.73:0/4017608913 learned_addr learned my addr 172.21.15.73:0/4017608913 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52c16aa700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2f60 0x55fc3ed5f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 -- 172.21.15.73:0/4017608913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2f60 msgr2=0x55fc3ed5f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2f60 0x55fc3ed5f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.713 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 -- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 msgr2=0x55fc3ecf9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecf9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 -- 172.21.15.73:0/4017608913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc3ed5f760 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52b3fff700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009f90 tx=0x7f52ac0a5240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9379f7b18b03d0a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f529bfff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac0d0630 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52b4015070 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f52ac09dbb0 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.706+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f52ac0a53f0 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 10 client.15786 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc3ed60ab0 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52b4015070 con 0x55fc3ece2640 2024-06-06T07:02:52.714 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f52ac0040b0 con 0x55fc3ece2640 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 20 client.15786 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ed611f0 0x55fc3ed63610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 4 client.15786 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc3ec36290 con 0x55fc3ed611f0 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.707+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ed611f0 0x55fc3ed63610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52c0ea9700 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ed611f0 0x55fc3ed63610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=297ce0e9e5316f7f server_cookie=77b80bd44942ebf9 in_seq=0 out_seq=0 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f529bfff700 10 client.15786 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f529bfff700 1 -- 172.21.15.73:0/4017608913 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc3ec36290 con 0x55fc3ed611f0 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f529bfff700 10 client.15786 handle_command_reply: tid=0 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 client.15786 shutdown 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 msgr2=0x55fc3ecf98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.715 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f52ac009f90 tx=0x7f52ac0a5240 comp rx=0 tx=0).stop 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 shutdown_connections 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55fc3ed611f0 0x55fc3ed63610 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fc3ece2f60 0x55fc3ed5f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fc3ecdcf20 0x55fc3ecf9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 --2- 172.21.15.73:0/4017608913 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fc3ece2640 0x55fc3ecf98b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 >> 172.21.15.73:0/4017608913 conn(0x55fc3ec07cc0 msgr2=0x55fc3ecd0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f529bfff700 0 client.15786 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 shutdown_connections 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 1 -- 172.21.15.73:0/4017608913 wait complete. 2024-06-06T07:02:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.708+0000 7f52cd990b80 20 client.15786 trim_cache size 0 max 16384 2024-06-06T07:02:52.717 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:52.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 msgr2=0x7f52a40a5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.717 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 0x7f52a40a5370 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009c50 tx=0x7f52b8009040 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 msgr2=0x7f52bc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f52ac000c00 tx=0x7f52ac003a80 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 shutdown_connections 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f52a40a2eb0 0x7f52a40a5370 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f52bc1084f0 0x7f52bc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f52bc06aa00 0x7f52bc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 --2- 172.21.15.73:0/1424065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52bc06b3c0 0x7f52bc101990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 >> 172.21.15.73:0/1424065329 conn(0x7f52bc078300 msgr2=0x7f52bc1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 shutdown_connections 2024-06-06T07:02:52.718 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:52.709+0000 7f52c310d700 1 -- 172.21.15.73:0/1424065329 wait complete. 2024-06-06T07:02:53.022 INFO:tasks.workunit.client.0.smithi073.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-06T07:02:53.087 INFO:tasks.workunit.client.0.smithi073.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-06T07:02:53.128 INFO:tasks.workunit.client.0.smithi073.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-06T07:02:53.129 INFO:tasks.workunit.client.0.smithi073.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T07:02:53.129 INFO:tasks.workunit.client.0.smithi073.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-06T07:02:53.168 INFO:tasks.workunit.client.0.smithi073.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-06T07:02:53.171 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T07:02:53.171 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-06T07:02:53.228 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-06T07:02:53.230 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-06T07:02:53.230 DEBUG:teuthology.orchestra.run.smithi073:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=3d746ca5346f34eef7e826a1eb71e59e02b682ae TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-06T07:02:53.294 INFO:tasks.workunit.client.0.smithi073.stderr:+ dbench 1 2024-06-06T07:02:53.319 INFO:tasks.workunit.client.0.smithi073.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-06T07:02:53.319 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:02:53.319 INFO:tasks.workunit.client.0.smithi073.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-06T07:02:53.332 INFO:tasks.workunit.client.0.smithi073.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-06T07:02:53.432 INFO:tasks.workunit.client.0.smithi073.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-06T07:02:53.432 INFO:tasks.workunit.client.0.smithi073.stdout:releasing clients 2024-06-06T07:02:53.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:53 smithi073 ceph-mon[32303]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:53 smithi136 ceph-mon[35901]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:53.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:53 smithi170 ceph-mon[36108]: pgmap v49: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-06-06T07:02:53.671 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:02:53.986 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:54.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 904 21.68 MB/sec warmup 1 sec latency 11.174 ms 2024-06-06T07:02:55.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 1630 13.08 MB/sec warmup 2 sec latency 15.643 ms 2024-06-06T07:02:55.475 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:55 smithi073 ceph-mon[32303]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-06T07:02:55.475 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:55 smithi073 ceph-mon[32303]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-06-06T07:02:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:55 smithi136 ceph-mon[35901]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-06T07:02:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:55 smithi136 ceph-mon[35901]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-06-06T07:02:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:55 smithi170 ceph-mon[36108]: pgmap v50: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-06T07:02:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:55 smithi170 ceph-mon[36108]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-06-06T07:02:55.745 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 -- 172.21.15.73:0/404694977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101380 msgr2=0x7ff1c8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 --2- 172.21.15.73:0/404694977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101380 0x7ff1c8101800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc009990 tx=0x7ff1bc009ca0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 -- 172.21.15.73:0/404694977 shutdown_connections 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 --2- 172.21.15.73:0/404694977 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c8101d40 0x7ff1c810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 --2- 172.21.15.73:0/404694977 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c81073f0 0x7ff1c81077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 --2- 172.21.15.73:0/404694977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101380 0x7ff1c8101800 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 -- 172.21.15.73:0/404694977 >> 172.21.15.73:0/404694977 conn(0x7ff1c8078300 msgr2=0x7ff1c8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 -- 172.21.15.73:0/404694977 shutdown_connections 2024-06-06T07:02:55.746 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.744+0000 7ff1cd627700 1 -- 172.21.15.73:0/404694977 wait complete. 2024-06-06T07:02:55.747 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 Processor -- start 2024-06-06T07:02:55.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 -- start start 2024-06-06T07:02:55.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c8101380 0x7ff1c819e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c81073f0 0x7ff1c8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.748 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c8111540 con 0x7ff1c8101d40 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81113c0 con 0x7ff1c8101380 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.745+0000 7ff1cd627700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81116c0 con 0x7ff1c81073f0 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54046/0 (socket says 172.21.15.73:54046) 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 -- 172.21.15.73:0/4219584772 learned_addr learned my addr 172.21.15.73:0/4219584772 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c81073f0 0x7ff1c8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c8101380 0x7ff1c819e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 -- 172.21.15.73:0/4219584772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c81073f0 msgr2=0x7ff1c8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c81073f0 0x7ff1c8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 -- 172.21.15.73:0/4219584772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c8101380 msgr2=0x7ff1c819e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c8101380 0x7ff1c819e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 -- 172.21.15.73:0/4219584772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c8199090 con 0x7ff1c8101d40 2024-06-06T07:02:55.749 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1affff700 1 -- 172.21.15.73:0/4219584772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc027070 con 0x7ff1c8101d40 2024-06-06T07:02:55.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1affff700 1 -- 172.21.15.73:0/4219584772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1bc022470 con 0x7ff1c8101d40 2024-06-06T07:02:55.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c8199320 con 0x7ff1c8101d40 2024-06-06T07:02:55.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1affff700 1 -- 172.21.15.73:0/4219584772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc017650 con 0x7ff1c8101d40 2024-06-06T07:02:55.750 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.746+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c81a5640 con 0x7ff1c8101d40 2024-06-06T07:02:55.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.749+0000 7ff1affff700 1 -- 172.21.15.73:0/4219584772 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff1bc02a070 con 0x7ff1c8101d40 2024-06-06T07:02:55.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1affff700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 0x7ff1b00a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 0x7ff1b00a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1affff700 1 -- 172.21.15.73:0/4219584772 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff1bc0d04a0 con 0x7ff1c8101d40 2024-06-06T07:02:55.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 0x7ff1b00a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8199a70 tx=0x7ff1b800a380 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1d7eaab80 1 Processor -- start 2024-06-06T07:02:55.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.750+0000 7ff1d7eaab80 1 -- start start 2024-06-06T07:02:55.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c0979af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0986210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0867f60 con 0x55e0c097f470 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0c08c6850 con 0x55e0c0979670 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e0c082e460 con 0x55e0c097a030 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54052/0 (socket says 172.21.15.73:54052) 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3920524252 learned_addr learned my addr 172.21.15.73:0/3920524252 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c0979af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0986210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3920524252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 msgr2=0x55e0c0986210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0986210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3920524252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 msgr2=0x55e0c0979af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c0979af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3920524252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c08158d0 con 0x55e0c097f470 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c0979af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.751+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004530 tx=0x7ff1b8012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90b0ec225c296b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.752+0000 7ff1ad7fa700 1 -- 172.21.15.73:0/3920524252 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8016070 con 0x55e0c097f470 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.752+0000 7ff1ad7fa700 1 -- 172.21.15.73:0/3920524252 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8014070 con 0x55e0c097f470 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.752+0000 7ff1ad7fa700 1 -- 172.21.15.73:0/3920524252 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8010d90 con 0x55e0c097f470 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.752+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3920524252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 msgr2=0x55e0c097f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.757 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8004530 tx=0x7ff1b8012000 comp rx=0 tx=0).stop 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3920524252 shutdown_connections 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0986210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097f470 0x55e0c097f850 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3920524252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c0979af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3920524252 >> 172.21.15.73:0/3920524252 conn(0x55e0c08a2080 msgr2=0x55e0c0970220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3920524252 shutdown_connections 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3920524252 wait complete. 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- start start 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 Processor -- start 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c0979670 0x55e0c0985e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.758 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0983690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0866fc0 con 0x55e0c097a030 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e0c08c2810 con 0x55e0c097f470 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.753+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e0c0986750 con 0x55e0c0979670 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1d7eaab80 10 client.0 _ll_register_callbacks cb 0x55e0c085c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c0979670 0x55e0c0985e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54062/0 (socket says 172.21.15.73:54062) 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 -- 172.21.15.73:0/3256050120 learned_addr learned my addr 172.21.15.73:0/3256050120 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0983690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 -- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c0979670 msgr2=0x55e0c0985e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c0979670 0x55e0c0985e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.759 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 -- 172.21.15.73:0/3256050120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 msgr2=0x55e0c0983690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0983690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 -- 172.21.15.73:0/3256050120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c088c750 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc099070 tx=0x7ff1bc09c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222272d1feb74d04 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc0a0a50 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff1bc085be0 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8014070 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc0a5280 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1d7eaab80 10 client.15798 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.754+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e0c0990d80 con 0x55e0c097a030 2024-06-06T07:02:55.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1bc085de0 con 0x55e0c097a030 2024-06-06T07:02:55.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff1bc09c900 con 0x55e0c097a030 2024-06-06T07:02:55.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 10 client.15798 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:55.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:02:55.762 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 20 client.15798 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c09957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 4 client.15798 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff1b8014070 con 0x55e0c0993390 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.755+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c09957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.756+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c09957b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7263e81930a79fce server_cookie=f85ee9ba5d0dd050 in_seq=0 out_seq=0 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.756+0000 7ff1aaff5700 10 client.15798 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.756+0000 7ff1aaff5700 1 -- 172.21.15.73:0/3256050120 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff1b8014070 con 0x55e0c0993390 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.756+0000 7ff1aaff5700 10 client.15798 handle_command_reply: tid=0 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.756+0000 7ff1d7eaab80 1 client.15798 shutdown 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 msgr2=0x55e0c0983150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.763 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc099070 tx=0x7ff1bc09c080 comp rx=0 tx=0).stop 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 shutdown_connections 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c09957b0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1aaff5700 0 client.15798 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c0979670 0x55e0c0985e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0983690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3256050120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c0983150 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 >> 172.21.15.73:0/3256050120 conn(0x55e0c08a2080 msgr2=0x55e0c096f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.767 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 shutdown_connections 2024-06-06T07:02:55.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3256050120 wait complete. 2024-06-06T07:02:55.768 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.757+0000 7ff1d7eaab80 20 client.15798 trim_cache size 0 max 16384 2024-06-06T07:02:55.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.773+0000 7ff1d7eaab80 1 Processor -- start 2024-06-06T07:02:55.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.773+0000 7ff1d7eaab80 1 -- start start 2024-06-06T07:02:55.775 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0974790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0974cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81ac1e0 con 0x55e0c0979670 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81add20 con 0x55e0c097f470 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81ac8b0 con 0x55e0c097a030 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54066/0 (socket says 172.21.15.73:54066) 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3108530838 learned_addr learned my addr 172.21.15.73:0/3108530838 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0974790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0974cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.776 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3108530838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 msgr2=0x55e0c0974790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0974790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3108530838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 msgr2=0x55e0c0974cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0974cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 -- 172.21.15.73:0/3108530838 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0c09816c0 con 0x55e0c0979670 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800f6a0 tx=0x7ff1b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8a926212f93587 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.774+0000 7ff1acff9700 1 -- 172.21.15.73:0/3108530838 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8016040 con 0x55e0c0979670 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1acff9700 1 -- 172.21.15.73:0/3108530838 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8014050 con 0x55e0c0979670 2024-06-06T07:02:55.777 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1acff9700 1 -- 172.21.15.73:0/3108530838 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8030400 con 0x55e0c0979670 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3108530838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 msgr2=0x55e0c0974250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800f6a0 tx=0x7ff1b800a430 comp rx=0 tx=0).stop 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3108530838 shutdown_connections 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c097f470 0x55e0c0974cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c0979670 0x55e0c0974250 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/3108530838 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097a030 0x55e0c0974790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3108530838 >> 172.21.15.73:0/3108530838 conn(0x55e0c08a2080 msgr2=0x55e0c097d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.781 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3108530838 shutdown_connections 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.775+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/3108530838 wait complete. 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 -- start start 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 Processor -- start 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c098b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 0x55e0c0986e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81ad650 con 0x55e0c097a030 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81acf80 con 0x55e0c0979670 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c81aa180 con 0x55e0c097f470 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1d7eaab80 10 client.0 _ll_register_callbacks cb 0x55e0c085c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1c67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54074/0 (socket says 172.21.15.73:54074) 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1c67fc700 1 -- 172.21.15.73:0/763095570 learned_addr learned my addr 172.21.15.73:0/763095570 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c098b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.776+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 0x55e0c0986e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 -- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 msgr2=0x55e0c0986e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 0x55e0c0986e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 -- 172.21.15.73:0/763095570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 msgr2=0x55e0c098b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c098b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 -- 172.21.15.73:0/763095570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0c0987610 con 0x55e0c097a030 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c77fe700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 0x55e0c0986e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1c67fc700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0098e0 tx=0x7ff1bc098a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3991c95dbe300d86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1bc085de0 con 0x55e0c097a030 2024-06-06T07:02:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff1bc09baf0 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b8014050 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff1bc0a0a50 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 10 client.15804 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e0c09fdad0 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8014050 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7ff1bc09b560 con 0x55e0c097a030 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 10 client.15804 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.777+0000 7ff1d7eaab80 20 client.15804 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c0a00000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 4 client.15804 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e0c08d3d50 con 0x55e0c0993390 2024-06-06T07:02:55.784 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c0a00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1c6ffd700 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c0a00000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9edd94b44f27c82d server_cookie=616cba254cb069df in_seq=0 out_seq=0 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1aaff5700 10 client.15804 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1aaff5700 1 -- 172.21.15.73:0/763095570 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e0c08d3d50 con 0x55e0c0993390 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1aaff5700 10 client.15804 handle_command_reply: tid=0 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 client.15804 shutdown 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 msgr2=0x55e0c098be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc0098e0 tx=0x7ff1bc098a70 comp rx=0 tx=0).stop 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 shutdown_connections 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55e0c0993390 0x55e0c0a00000 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e0c097f470 0x55e0c0986e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e0c0979670 0x55e0c098b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 --2- 172.21.15.73:0/763095570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e0c097a030 0x55e0c098be70 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.785 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.778+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 >> 172.21.15.73:0/763095570 conn(0x55e0c08a2080 msgr2=0x55e0c0980130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1aaff5700 0 client.15804 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:02:55.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 shutdown_connections 2024-06-06T07:02:55.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1d7eaab80 1 -- 172.21.15.73:0/763095570 wait complete. 2024-06-06T07:02:55.786 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1d7eaab80 20 client.15804 trim_cache size 0 max 16384 2024-06-06T07:02:55.786 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 msgr2=0x7ff1b00a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 0x7ff1b00a52d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff1c8199a70 tx=0x7ff1b800a380 comp rx=0 tx=0).stop 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 msgr2=0x7ff1c819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7ff1bc000c00 tx=0x7ff1bc0047b0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 shutdown_connections 2024-06-06T07:02:55.787 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff1b00a2e10 0x7ff1b00a52d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff1c81073f0 0x7ff1c8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff1c8101380 0x7ff1c819e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 --2- 172.21.15.73:0/4219584772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1c8101d40 0x7ff1c819ecb0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.779+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 >> 172.21.15.73:0/4219584772 conn(0x7ff1c8078300 msgr2=0x7ff1c81005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.780+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 shutdown_connections 2024-06-06T07:02:55.788 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:55.780+0000 7ff1cd627700 1 -- 172.21.15.73:0/4219584772 wait complete. 2024-06-06T07:02:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 -- 172.21.15.73:0/4280006351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 msgr2=0x7f9db4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 --2- 172.21.15.73:0/4280006351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db4100d50 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009930 tx=0x7f9da802a420 comp rx=0 tx=0).stop 2024-06-06T07:02:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 -- 172.21.15.73:0/4280006351 shutdown_connections 2024-06-06T07:02:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 --2- 172.21.15.73:0/4280006351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 0x7f9db410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 --2- 172.21.15.73:0/4280006351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db4100d50 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 --2- 172.21.15.73:0/4280006351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 0x7f9db4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 -- 172.21.15.73:0/4280006351 >> 172.21.15.73:0/4280006351 conn(0x7f9db4077a20 msgr2=0x7f9db4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 -- 172.21.15.73:0/4280006351 shutdown_connections 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 -- 172.21.15.73:0/4280006351 wait complete. 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.347+0000 7f9dba89e640 1 Processor -- start 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 -- start start 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 0x7f9db4198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 0x7f9db419cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4110b70 con 0x7f9db41008d0 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41109f0 con 0x7f9db4106890 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9dba89e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9db4110cf0 con 0x7f9db4101290 2024-06-06T07:02:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54102/0 (socket says 172.21.15.73:54102) 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 -- 172.21.15.73:0/1143502687 learned_addr learned my addr 172.21.15.73:0/1143502687 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 0x7f9db4198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 0x7f9db419cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 -- 172.21.15.73:0/1143502687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 msgr2=0x7f9db4198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 0x7f9db4198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 -- 172.21.15.73:0/1143502687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 msgr2=0x7f9db419cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 0x7f9db419cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.348+0000 7f9db3fff640 1 -- 172.21.15.73:0/1143502687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9db419d1e0 con 0x7f9db41008d0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9da400b4a0 tx=0x7f9da400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9db17fa640 1 -- 172.21.15.73:0/1143502687 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4014070 con 0x7f9db41008d0 2024-06-06T07:02:56.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9db17fa640 1 -- 172.21.15.73:0/1143502687 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da4010ac0 con 0x7f9db41008d0 2024-06-06T07:02:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9db17fa640 1 -- 172.21.15.73:0/1143502687 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4017c30 con 0x7f9db41008d0 2024-06-06T07:02:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9db419d4d0 con 0x7f9db41008d0 2024-06-06T07:02:56.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.349+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9db41a4d60 con 0x7f9db41008d0 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.351+0000 7f9db17fa640 1 -- 172.21.15.73:0/1143502687 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9da4017070 con 0x7f9db41008d0 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 Processor -- start 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 -- start start 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db17fa640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 0x7f9d9c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5170 0x556b9a8b5550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 0x556b9a8c2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b9a8194f0 con 0x556b9a8b5a90 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b9a7b7870 con 0x556b9a8c0350 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b9a8154b0 con 0x556b9a8b5170 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 0x7f9d9c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db17fa640 1 -- 172.21.15.73:0/1143502687 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f9da40cedc0 con 0x7f9db41008d0 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54108/0 (socket says 172.21.15.73:54108) 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 -- 172.21.15.73:0/1215995224 learned_addr learned my addr 172.21.15.73:0/1215995224 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5170 0x556b9a8b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 0x556b9a8c2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 -- 172.21.15.73:0/1215995224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5170 msgr2=0x556b9a8b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5170 0x556b9a8b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 -- 172.21.15.73:0/1215995224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 msgr2=0x556b9a8c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 0x556b9a8c2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db37fe640 1 -- 172.21.15.73:0/1215995224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9a7d39b0 con 0x556b9a8b5a90 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.352+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 0x556b9a8c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.353+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 0x7f9d9c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x556b9a8b2aa0 tx=0x7f9dac009dc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.353+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009930 tx=0x7f9da8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a0c304b443f01e2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.353+0000 7f9d9a7fc640 1 -- 172.21.15.73:0/1215995224 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8040070 con 0x556b9a8b5a90 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.353+0000 7f9d9a7fc640 1 -- 172.21.15.73:0/1215995224 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da80316a0 con 0x556b9a8b5a90 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.353+0000 7f9d9a7fc640 1 -- 172.21.15.73:0/1215995224 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da8035040 con 0x556b9a8b5a90 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1215995224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 msgr2=0x556b9a8bfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009930 tx=0x7f9da8005fd0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1215995224 shutdown_connections 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8c0350 0x556b9a8c2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bfe10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1215995224 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5170 0x556b9a8b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1215995224 >> 172.21.15.73:0/1215995224 conn(0x556b9a8b1680 msgr2=0x556b9a8b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1215995224 shutdown_connections 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1215995224 wait complete. 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 -- start start 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 Processor -- start 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8bd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.354+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8bdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b9a8c2f20 con 0x556b9a8b5a90 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556b9a7a2ae0 con 0x556b9a8b5170 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556b9a8c30a0 con 0x556b9a8c0350 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9dbd5ad740 10 client.0 _ll_register_callbacks cb 0x556b9a7db9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54112/0 (socket says 172.21.15.73:54112) 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 -- 172.21.15.73:0/1088625942 learned_addr learned my addr 172.21.15.73:0/1088625942 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8bdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8bd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 -- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 msgr2=0x556b9a8bdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8bdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 -- 172.21.15.73:0/1088625942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 msgr2=0x556b9a8bd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8bd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 -- 172.21.15.73:0/1088625942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b9a52d210 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8bdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9da802a940 tx=0x7f9da80050a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f098ba97b3bd5a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da804b070 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f9da8005530 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.355+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da80316a0 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9da8057050 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 10 client.15816 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b9a5f9620 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da80316a0 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9da803c670 con 0x556b9a8b5a90 2024-06-06T07:02:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 10 client.15816 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 20 client.15816 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a8d02b0 0x556b9a8d26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 4 client.15816 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b9a82cf10 con 0x556b9a8d02b0 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.356+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a8d02b0 0x556b9a8d26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.357+0000 7f9d937fe640 10 client.15816 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.357+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a8d02b0 0x556b9a8d26a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197e458a7c165556 server_cookie=1fb55f407a2bec52 in_seq=0 out_seq=0 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.357+0000 7f9d937fe640 1 -- 172.21.15.73:0/1088625942 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556b9a82cf10 con 0x556b9a8d02b0 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.357+0000 7f9d937fe640 10 client.15816 handle_command_reply: tid=0 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 client.15816 shutdown 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 msgr2=0x556b9a8bd820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f9da802a940 tx=0x7f9da80050a0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 shutdown_connections 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a8d02b0 0x556b9a8d26a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8bdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd820 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1088625942 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8bd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 >> 172.21.15.73:0/1088625942 conn(0x556b9a8b1680 msgr2=0x556b9a8b73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9d937fe640 0 client.15816 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 shutdown_connections 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1088625942 wait complete. 2024-06-06T07:02:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.358+0000 7f9dbd5ad740 20 client.15816 trim_cache size 0 max 16384 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 Processor -- start 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 -- start start 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8d7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41acbe0 con 0x556b9a8b5a90 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ae720 con 0x556b9a8b5170 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ad2b0 con 0x556b9a8c0350 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9db3fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:39724/0 (socket says 172.21.15.73:39724) 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9db3fff640 1 -- 172.21.15.73:0/3224209592 learned_addr learned my addr 172.21.15.73:0/3224209592 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9db8e3b640 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8d7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.377+0000 7f9db37fe640 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 -- 172.21.15.73:0/3224209592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 msgr2=0x556b9a8d7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8d7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 -- 172.21.15.73:0/3224209592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 msgr2=0x556b9a8bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 -- 172.21.15.73:0/3224209592 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9a8b8070 con 0x556b9a8b5170 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9db3fff640 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9da40cc760 tx=0x7f9da40bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4605781f0a49e62 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9d99ffb640 1 -- 172.21.15.73:0/3224209592 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da4079e60 con 0x556b9a8b5170 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9d99ffb640 1 -- 172.21.15.73:0/3224209592 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da409b040 con 0x556b9a8b5170 2024-06-06T07:02:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.378+0000 7f9d99ffb640 1 -- 172.21.15.73:0/3224209592 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da401e040 con 0x556b9a8b5170 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/3224209592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 msgr2=0x556b9a8b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9da40cc760 tx=0x7f9da40bb040 comp rx=0 tx=0).stop 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/3224209592 shutdown_connections 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8c0350 0x556b9a8d7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8b5a90 0x556b9a8bd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/3224209592 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8b7b30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/3224209592 >> 172.21.15.73:0/3224209592 conn(0x556b9a8b1680 msgr2=0x556b9a8bd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/3224209592 shutdown_connections 2024-06-06T07:02:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/3224209592 wait complete. 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.380+0000 7f9dbd5ad740 1 -- start start 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 Processor -- start 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8c0350 0x556b9a8c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ae050 con 0x556b9a8c0350 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41ad980 con 0x556b9a8b5170 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9db41aabc0 con 0x556b9a8b5a90 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9dbd5ad740 10 client.0 _ll_register_callbacks cb 0x7f9da4017360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db8e3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8c0350 0x556b9a8c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55076/0 (socket says 172.21.15.73:55076) 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 -- 172.21.15.73:0/1663119597 learned_addr learned my addr 172.21.15.73:0/1663119597 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 -- 172.21.15.73:0/1663119597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 msgr2=0x556b9a8c32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8c32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 -- 172.21.15.73:0/1663119597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8c0350 msgr2=0x556b9a8c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8c0350 0x556b9a8c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.381+0000 7f9db37fe640 1 -- 172.21.15.73:0/1663119597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b9a938ec0 con 0x556b9a8b5a90 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9db37fe640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9da8031440 tx=0x7f9da80095b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=37d580f964b7b131 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9d937fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9da804b070 con 0x556b9a8b5a90 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9da409b040 con 0x556b9a8b5a90 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f9da803c640 con 0x556b9a8b5a90 2024-06-06T07:02:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9da8064050 con 0x556b9a8b5a90 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9dbd5ad740 10 client.25363 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556b9a93a1c0 con 0x556b9a8b5a90 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.382+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9da409b040 con 0x556b9a8b5a90 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7f9da8050440 con 0x556b9a8b5a90 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 10 client.25363 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 10 client.25363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 20 client.25363 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a93a970 0x556b9a93cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 4 client.25363 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b9a826c20 con 0x556b9a93a970 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a93a970 0x556b9a93cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9d937fe640 10 client.25363 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.383+0000 7f9db3fff640 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a93a970 0x556b9a93cd90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51c61e2f04b82110 server_cookie=8c29f1a4fac9a403 in_seq=0 out_seq=0 2024-06-06T07:02:56.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 2361 9.30 MB/sec warmup 3 sec latency 43.899 ms 2024-06-06T07:02:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.449+0000 7f9d937fe640 1 -- 172.21.15.73:0/1663119597 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b9a826c20 con 0x556b9a93a970 2024-06-06T07:02:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.449+0000 7f9d937fe640 10 client.25363 handle_command_reply: tid=0 2024-06-06T07:02:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.449+0000 7f9dbd5ad740 1 client.25363 shutdown 2024-06-06T07:02:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 msgr2=0x556b9a8c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f9da8031440 tx=0x7f9da80095b0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 shutdown_connections 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556b9a93a970 0x556b9a93cd90 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9a8c0350 0x556b9a8c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556b9a8b5a90 0x556b9a8c37f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9d937fe640 0 client.25363 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 --2- 172.21.15.73:0/1663119597 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556b9a8b5170 0x556b9a8c32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 >> 172.21.15.73:0/1663119597 conn(0x556b9a8b1680 msgr2=0x556b9a8cba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 shutdown_connections 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 1 -- 172.21.15.73:0/1663119597 wait complete. 2024-06-06T07:02:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dbd5ad740 20 client.25363 trim_cache size 0 max 16384 2024-06-06T07:02:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:02:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:02:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 msgr2=0x7f9d9c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 0x7f9d9c0a5260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x556b9a8b2aa0 tx=0x7f9dac009dc0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 msgr2=0x7f9db41982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.450+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9da400b4a0 tx=0x7f9da400b960 comp rx=0 tx=0).stop 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 shutdown_connections 2024-06-06T07:02:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9db4106890 0x7f9db419cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9d9c0a2da0 0x7f9d9c0a5260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9db4101290 0x7f9db4198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 --2- 172.21.15.73:0/1143502687 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db41008d0 0x7f9db41982e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 >> 172.21.15.73:0/1143502687 conn(0x7f9db4077a20 msgr2=0x7f9db4069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 shutdown_connections 2024-06-06T07:02:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:02:56.451+0000 7f9dba89e640 1 -- 172.21.15.73:0/1143502687 wait complete. 2024-06-06T07:02:56.625 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:02:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:02:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:57 smithi073 ceph-mon[32303]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T07:02:57.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 3205 8.57 MB/sec warmup 4 sec latency 62.959 ms 2024-06-06T07:02:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:57 smithi136 ceph-mon[35901]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T07:02:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:57 smithi170 ceph-mon[36108]: pgmap v51: 129 pgs: 129 active+clean; 1.3 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.2 KiB/s wr, 9 op/s 2024-06-06T07:02:58.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 3790 7.95 MB/sec warmup 5 sec latency 16.268 ms 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 -- 172.21.15.73:0/2522484103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 msgr2=0x7fab8410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 --2- 172.21.15.73:0/2522484103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8410e240 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fab7c00b0a0 tx=0x7fab7c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 -- 172.21.15.73:0/2522484103 shutdown_connections 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 --2- 172.21.15.73:0/2522484103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8410e240 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 --2- 172.21.15.73:0/2522484103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 0x7fab841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.658 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 --2- 172.21.15.73:0/2522484103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab84101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 -- 172.21.15.73:0/2522484103 >> 172.21.15.73:0/2522484103 conn(0x7fab84078340 msgr2=0x7fab84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 -- 172.21.15.73:0/2522484103 shutdown_connections 2024-06-06T07:02:58.659 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 -- 172.21.15.73:0/2522484103 wait complete. 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.657+0000 7fab8980d700 1 Processor -- start 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 -- start start 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab8419d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 0x7fab84197890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab84111550 con 0x7fab84101d50 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab841113d0 con 0x7fab84107400 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab8980d700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab841116d0 con 0x7fab84101390 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab827fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab827fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54138/0 (socket says 172.21.15.73:54138) 2024-06-06T07:02:58.661 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab827fc700 1 -- 172.21.15.73:0/2667288629 learned_addr learned my addr 172.21.15.73:0/2667288629 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab8419d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab837fe700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 0x7fab84197890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 -- 172.21.15.73:0/2667288629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 msgr2=0x7fab84197890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 0x7fab84197890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 -- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 msgr2=0x7fab8419dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab82ffd700 1 -- 172.21.15.73:0/2667288629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab84197ff0 con 0x7fab84101390 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.658+0000 7fab827fc700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab8419d6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab7400d750 tx=0x7fab7400da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab8880b700 1 -- 172.21.15.73:0/2667288629 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74014030 con 0x7fab84101390 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab8880b700 1 -- 172.21.15.73:0/2667288629 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab740044d0 con 0x7fab84101390 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab841982e0 con 0x7fab84101390 2024-06-06T07:02:58.662 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab8880b700 1 -- 172.21.15.73:0/2667288629 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74005420 con 0x7fab84101390 2024-06-06T07:02:58.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.659+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab8406a170 con 0x7fab84101390 2024-06-06T07:02:58.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.661+0000 7fab8880b700 1 -- 172.21.15.73:0/2667288629 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fab74004750 con 0x7fab84101390 2024-06-06T07:02:58.663 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.662+0000 7fab8880b700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 0x7fab6c0a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.662+0000 7fab827fc700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 0x7fab6c0a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.662+0000 7fab8880b700 1 -- 172.21.15.73:0/2667288629 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fab740cf340 con 0x7fab84101390 2024-06-06T07:02:58.664 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.663+0000 7fab827fc700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 0x7fab6c0a52d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab84078980 tx=0x7fab78009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.665 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.663+0000 7fab94090b80 1 Processor -- start 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.663+0000 7fab94090b80 1 -- start start 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 0x555da67c26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67c2c30 0x555da67c3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555da670b810 con 0x555da67bdfd0 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555da66b3720 con 0x555da67be8f0 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab94090b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555da67077d0 con 0x555da67c2c30 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54144/0 (socket says 172.21.15.73:54144) 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 -- 172.21.15.73:0/3748963259 learned_addr learned my addr 172.21.15.73:0/3748963259 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab837fe700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67c2c30 0x555da67c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab827fc700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 0x555da67c26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.666 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 -- 172.21.15.73:0/3748963259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67c2c30 msgr2=0x555da67c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67c2c30 0x555da67c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 -- 172.21.15.73:0/3748963259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 msgr2=0x555da67c26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 0x555da67c26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 -- 172.21.15.73:0/3748963259 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da66a9f50 con 0x555da67bdfd0 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab827fc700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 0x555da67c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fab7400ddb0 tx=0x7fab7409bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c93c2911f5aafb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.664+0000 7fab69ffb700 1 -- 172.21.15.73:0/3748963259 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7401d030 con 0x555da67bdfd0 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab69ffb700 1 -- 172.21.15.73:0/3748963259 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab740a40c0 con 0x555da67bdfd0 2024-06-06T07:02:58.667 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab69ffb700 1 -- 172.21.15.73:0/3748963259 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740055c0 con 0x555da67bdfd0 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 -- 172.21.15.73:0/3748963259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 msgr2=0x555da67be3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fab7400ddb0 tx=0x7fab7409bc50 comp rx=0 tx=0).stop 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 -- 172.21.15.73:0/3748963259 shutdown_connections 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67c2c30 0x555da67c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67bdfd0 0x555da67be3b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 --2- 172.21.15.73:0/3748963259 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67be8f0 0x555da67c26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 -- 172.21.15.73:0/3748963259 >> 172.21.15.73:0/3748963259 conn(0x555da66e4cc0 msgr2=0x555da67b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.665+0000 7fab94090b80 1 -- 172.21.15.73:0/3748963259 shutdown_connections 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 -- 172.21.15.73:0/3748963259 wait complete. 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 -- start start 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 Processor -- start 2024-06-06T07:02:58.671 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 0x555da67ca170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67c2c30 0x555da67c7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555da67cadd0 con 0x555da67be8f0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555da66abbf0 con 0x555da67c2c30 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555da67caf50 con 0x555da67bdfd0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.666+0000 7fab94090b80 10 client.0 _ll_register_callbacks cb 0x555da66a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab827fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 0x555da67ca170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab827fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54160/0 (socket says 172.21.15.73:54160) 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab827fc700 1 -- 172.21.15.73:0/3277239801 learned_addr learned my addr 172.21.15.73:0/3277239801 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab837fe700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67c2c30 0x555da67c7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 -- 172.21.15.73:0/3277239801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67c2c30 msgr2=0x555da67c7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67c2c30 0x555da67c7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.672 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 -- 172.21.15.73:0/3277239801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 msgr2=0x555da67c74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 -- 172.21.15.73:0/3277239801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da665a8d0 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab827fc700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab82ffd700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 0x555da67ca170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab7400ddb0 tx=0x7fab74097cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be4d46070d2ee510 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab740b8070 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fab740055c0 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.667+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab740a40c0 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab7401d040 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 10 client.25375 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555da67cadd0 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab740b4260 con 0x555da67bdfd0 2024-06-06T07:02:58.673 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fab740a3c30 con 0x555da67bdfd0 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 10 client.25375 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 10 client.25375 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 20 client.25375 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da67d7a60 0x555da67d9e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 4 client.25375 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab740a40c0 con 0x555da67d7a60 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.668+0000 7fab827fc700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da67d7a60 0x555da67d9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab827fc700 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da67d7a60 0x555da67d9e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cb6e5ff24bbcf4 server_cookie=5747244ea2260a50 in_seq=0 out_seq=0 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab62ffd700 10 client.25375 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:58.674 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab62ffd700 1 -- 172.21.15.73:0/3277239801 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fab740a40c0 con 0x555da67d7a60 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab62ffd700 10 client.25375 handle_command_reply: tid=0 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab94090b80 1 client.25375 shutdown 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.669+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 msgr2=0x555da67ca170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 0x555da67ca170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab7400ddb0 tx=0x7fab74097cd0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 shutdown_connections 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da67d7a60 0x555da67d9e50 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67bdfd0 0x555da67ca170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67c2c30 0x555da67c7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 --2- 172.21.15.73:0/3277239801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67be8f0 0x555da67c74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 >> 172.21.15.73:0/3277239801 conn(0x555da66e4cc0 msgr2=0x555da67b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab62ffd700 0 client.25375 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 shutdown_connections 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 1 -- 172.21.15.73:0/3277239801 wait complete. 2024-06-06T07:02:58.675 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.670+0000 7fab94090b80 20 client.25375 trim_cache size 0 max 16384 2024-06-06T07:02:58.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.685+0000 7fab94090b80 1 Processor -- start 2024-06-06T07:02:58.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.685+0000 7fab94090b80 1 -- start start 2024-06-06T07:02:58.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67ae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.687 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab841ac430 con 0x555da67c2c30 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab841adf70 con 0x555da67bdfd0 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab94090b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab841acb00 con 0x555da67be8f0 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54162/0 (socket says 172.21.15.73:54162) 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 -- 172.21.15.73:0/1407259830 learned_addr learned my addr 172.21.15.73:0/1407259830 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab827fc700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab82ffd700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67ae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 -- 172.21.15.73:0/1407259830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 msgr2=0x555da67c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 -- 172.21.15.73:0/1407259830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 msgr2=0x555da67ae510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.688 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67ae510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 -- 172.21.15.73:0/1407259830 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555da67aea50 con 0x555da67c2c30 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab827fc700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.686+0000 7fab837fe700 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009560 tx=0x7fab7c009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1781cf1238cf40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab697fa700 1 -- 172.21.15.73:0/1407259830 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00bc90 con 0x555da67c2c30 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab697fa700 1 -- 172.21.15.73:0/1407259830 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab7c0074e0 con 0x555da67c2c30 2024-06-06T07:02:58.689 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab697fa700 1 -- 172.21.15.73:0/1407259830 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c00be30 con 0x555da67c2c30 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 -- 172.21.15.73:0/1407259830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 msgr2=0x555da67df4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fab7c009560 tx=0x7fab7c009590 comp rx=0 tx=0).stop 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 -- 172.21.15.73:0/1407259830 shutdown_connections 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67df4a0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67ae510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 --2- 172.21.15.73:0/1407259830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 -- 172.21.15.73:0/1407259830 >> 172.21.15.73:0/1407259830 conn(0x555da66e4cc0 msgr2=0x555da67ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.692 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 -- 172.21.15.73:0/1407259830 shutdown_connections 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.687+0000 7fab94090b80 1 -- 172.21.15.73:0/1407259830 wait complete. 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 -- start start 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 Processor -- start 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67de890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab841ad8a0 con 0x555da67c2c30 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fab841ad1d0 con 0x555da67bdfd0 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fab841aa3d0 con 0x555da67be8f0 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab94090b80 10 client.0 _ll_register_callbacks cb 0x555da66a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab827fc700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55140/0 (socket says 172.21.15.73:55140) 2024-06-06T07:02:58.693 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab827fc700 1 -- 172.21.15.73:0/2865055199 learned_addr learned my addr 172.21.15.73:0/2865055199 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab837fe700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67de890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.688+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 -- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 msgr2=0x555da67d0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 -- 172.21.15.73:0/2865055199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 msgr2=0x555da67d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67d0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 -- 172.21.15.73:0/2865055199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555da67df0d0 con 0x555da67c2c30 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab827fc700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab837fe700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67de890 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fab7c007fd0 tx=0x7fab7c007940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697ba731444e1b9b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab62ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab7c006000 con 0x555da67c2c30 2024-06-06T07:02:58.694 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab7c0074e0 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fab7c00f040 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fab7c038050 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab94090b80 10 client.15840 fetch_fsmap learned FSMap version 32 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.689+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555da6841f50 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab7c0074e0 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 32) ==== 8362+0+0 (secure 0 0 0) 0x7fab7c011070 con 0x555da67c2c30 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 32 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 20 client.15840 populate_metadata read hostname 'smithi073' 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da6842520 0x555da6844940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 4 client.15840 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555da6718d10 con 0x555da6842520 2024-06-06T07:02:58.695 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da6842520 0x555da6844940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab62ffd700 10 client.15840 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.690+0000 7fab82ffd700 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da6842520 0x555da6844940 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e9bd70f781b76ce server_cookie=a5fc663ba348b726 in_seq=0 out_seq=0 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab62ffd700 1 -- 172.21.15.73:0/2865055199 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555da6718d10 con 0x555da6842520 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab62ffd700 10 client.15840 handle_command_reply: tid=0 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 client.15840 shutdown 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 msgr2=0x555da67de890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67de890 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fab7c007fd0 tx=0x7fab7c007940 comp rx=0 tx=0).stop 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 shutdown_connections 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555da6842520 0x555da6844940 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555da67c2c30 0x555da67de890 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555da67bdfd0 0x555da67d0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 --2- 172.21.15.73:0/2865055199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555da67be8f0 0x555da67d0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.696 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 >> 172.21.15.73:0/2865055199 conn(0x555da66e4cc0 msgr2=0x555da67c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab62ffd700 0 client.15840 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:02:58.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 shutdown_connections 2024-06-06T07:02:58.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 1 -- 172.21.15.73:0/2865055199 wait complete. 2024-06-06T07:02:58.697 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.691+0000 7fab94090b80 20 client.15840 trim_cache size 0 max 16384 2024-06-06T07:02:58.697 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 msgr2=0x7fab6c0a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 0x7fab6c0a52d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fab84078980 tx=0x7fab78009260 comp rx=0 tx=0).stop 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 msgr2=0x7fab8419d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab8419d6d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fab7400d750 tx=0x7fab7400da60 comp rx=0 tx=0).stop 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 shutdown_connections 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fab6c0a2e10 0x7fab6c0a52d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fab84101390 0x7fab8419d6d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.698 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fab84107400 0x7fab84197890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 --2- 172.21.15.73:0/2667288629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab84101d50 0x7fab8419dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:02:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 >> 172.21.15.73:0/2667288629 conn(0x7fab84078340 msgr2=0x7fab840fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:02:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 shutdown_connections 2024-06-06T07:02:58.699 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:02:58.692+0000 7fab8980d700 1 -- 172.21.15.73:0/2667288629 wait complete. 2024-06-06T07:02:59.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 4186 6.68 MB/sec warmup 6 sec latency 18.197 ms 2024-06-06T07:02:59.433 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:02:59 smithi073 ceph-mon[32303]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-06-06T07:02:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:02:59 smithi136 ceph-mon[35901]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-06-06T07:02:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:02:59 smithi170 ceph-mon[36108]: pgmap v52: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 86 KiB/s rd, 2.2 MiB/s wr, 84 op/s 2024-06-06T07:03:00.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 4811 6.03 MB/sec warmup 7 sec latency 14.341 ms 2024-06-06T07:03:00.465 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-06T07:03:00.466 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-06T07:03:00.490 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:00 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:00.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:00 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:00 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:00.743 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:01.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 5575 5.70 MB/sec warmup 8 sec latency 25.053 ms 2024-06-06T07:03:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:01 smithi073 ceph-mon[32303]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:01 smithi136 ceph-mon[35901]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:01 smithi170 ceph-mon[36108]: pgmap v53: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:02.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 6149 5.35 MB/sec warmup 9 sec latency 67.684 ms 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 -- 172.21.15.73:0/2812624506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806b3c0 msgr2=0x7f47a810f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 --2- 172.21.15.73:0/2812624506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806b3c0 0x7f47a810f310 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f47a000b0a0 tx=0x7f47a000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 -- 172.21.15.73:0/2812624506 shutdown_connections 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 --2- 172.21.15.73:0/2812624506 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806b3c0 0x7f47a810f310 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 --2- 172.21.15.73:0/2812624506 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 0x7f47a81088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 --2- 172.21.15.73:0/2812624506 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806aa00 0x7f47a806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 -- 172.21.15.73:0/2812624506 >> 172.21.15.73:0/2812624506 conn(0x7f47a8078300 msgr2=0x7f47a8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.437 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 -- 172.21.15.73:0/2812624506 shutdown_connections 2024-06-06T07:03:02.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.435+0000 7f47ad19b700 1 -- 172.21.15.73:0/2812624506 wait complete. 2024-06-06T07:03:02.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.436+0000 7f47ad19b700 1 Processor -- start 2024-06-06T07:03:02.438 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.436+0000 7f47ad19b700 1 -- start start 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 0x7f47a819ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 0x7f47a8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8112620 con 0x7f47a806aa00 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81124a0 con 0x7f47a81084f0 2024-06-06T07:03:02.439 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81127a0 con 0x7f47a806b3c0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54200/0 (socket says 172.21.15.73:54200) 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 0x7f47a819ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a6d9d700 1 -- 172.21.15.73:0/3727379508 learned_addr learned my addr 172.21.15.73:0/3727379508 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a759e700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 0x7f47a8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 -- 172.21.15.73:0/3727379508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 msgr2=0x7f47a8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 0x7f47a8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 -- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 msgr2=0x7f47a819e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 -- 172.21.15.73:0/3727379508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81990e0 con 0x7f47a806b3c0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47a659c700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 0x7f47a819ed00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e9f0 tx=0x7f479c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f478ffff700 1 -- 172.21.15.73:0/3727379508 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004220 con 0x7f47a806b3c0 2024-06-06T07:03:02.440 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81993d0 con 0x7f47a806b3c0 2024-06-06T07:03:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.437+0000 7f478ffff700 1 -- 172.21.15.73:0/3727379508 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c0043c0 con 0x7f47a806b3c0 2024-06-06T07:03:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.438+0000 7f478ffff700 1 -- 172.21.15.73:0/3727379508 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c005020 con 0x7f47a806b3c0 2024-06-06T07:03:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.438+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a5690 con 0x7f47a806b3c0 2024-06-06T07:03:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.440+0000 7f478ffff700 1 -- 172.21.15.73:0/3727379508 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f479c004750 con 0x7f47a806b3c0 2024-06-06T07:03:02.441 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.441+0000 7f478ffff700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 0x7f47900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.441+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 0x7f47900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.441+0000 7f478ffff700 1 -- 172.21.15.73:0/3727379508 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f479c0cf750 con 0x7f47a806b3c0 2024-06-06T07:03:02.442 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.441+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 0x7f47900a5250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f47a8199ac0 tx=0x7f4798009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.443 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 Processor -- start 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 -- start start 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e11bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55915e009fe0 con 0x55915e1214f0 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55915e0688d0 con 0x55915e11b6f0 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55915dfd04e0 con 0x55915e11c0b0 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54202/0 (socket says 172.21.15.73:54202) 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 -- 172.21.15.73:0/721811744 learned_addr learned my addr 172.21.15.73:0/721811744 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a759e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55160/0 (socket says 172.21.15.73:55160) 2024-06-06T07:03:02.444 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a659c700 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e11bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 -- 172.21.15.73:0/721811744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 msgr2=0x55915e128290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 -- 172.21.15.73:0/721811744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 msgr2=0x55915e11bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e11bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.442+0000 7f47a6d9d700 1 -- 172.21.15.73:0/721811744 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915dfb7950 con 0x55915e1214f0 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.443+0000 7f47a759e700 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.443+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f47980043b0 tx=0x7f4798014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6286c5537228dca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.443+0000 7f478d7fa700 1 -- 172.21.15.73:0/721811744 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798011e60 con 0x55915e1214f0 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.443+0000 7f478d7fa700 1 -- 172.21.15.73:0/721811744 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4798003ce0 con 0x55915e1214f0 2024-06-06T07:03:02.445 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.443+0000 7f478d7fa700 1 -- 172.21.15.73:0/721811744 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798026620 con 0x55915e1214f0 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/721811744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 msgr2=0x55915e1218d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f47980043b0 tx=0x7f4798014000 comp rx=0 tx=0).stop 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/721811744 shutdown_connections 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e128290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e1214f0 0x55915e1218d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/721811744 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e11bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/721811744 >> 172.21.15.73:0/721811744 conn(0x55915e044100 msgr2=0x55915e1122a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/721811744 shutdown_connections 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/721811744 wait complete. 2024-06-06T07:03:02.448 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 -- start start 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 Processor -- start 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e125290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.444+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e1214f0 0x55915e1257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e125290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54212/0 (socket says 172.21.15.73:54212) 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 -- 172.21.15.73:0/1839461106 learned_addr learned my addr 172.21.15.73:0/1839461106 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55915e009040 con 0x55915e11b6f0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55915e064890 con 0x55915e1214f0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55915e1287d0 con 0x55915e11c0b0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47b7a1eb80 10 client.0 _ll_register_callbacks cb 0x55915dffe630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 -- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 msgr2=0x55915e125290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.449 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e125290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 -- 172.21.15.73:0/1839461106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e1214f0 msgr2=0x55915e1257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a759e700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e1214f0 0x55915e1257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e1214f0 0x55915e1257d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915e02e7d0 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4798009840 tx=0x7f479801ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5de2b0d2247af7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.445+0000 7f478aff5700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798011e60 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f479802c070 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4798003ce0 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f479803a050 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 10 client.15852 fetch_fsmap learned FSMap version 33 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55915e132ec0 con 0x55915e11b6f0 2024-06-06T07:03:02.450 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4798003ce0 con 0x55915e11b6f0 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f4798004a20 con 0x55915e11b6f0 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 10 client.15852 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 20 client.15852 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e135440 0x55915e137830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 4 client.15852 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.446+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55915e075dd0 con 0x55915e135440 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.447+0000 7f47a659c700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e135440 0x55915e137830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.447+0000 7f47a659c700 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e135440 0x55915e137830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b8a2a4ba9ceab8 server_cookie=3fd95fa630c89e6c in_seq=0 out_seq=0 2024-06-06T07:03:02.451 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.447+0000 7f478aff5700 10 client.15852 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:02.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f478aff5700 1 -- 172.21.15.73:0/1839461106 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55915e075dd0 con 0x55915e135440 2024-06-06T07:03:02.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f478aff5700 10 client.15852 handle_command_reply: tid=0 2024-06-06T07:03:02.452 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 client.15852 shutdown 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 msgr2=0x55915e127f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4798009840 tx=0x7f479801ead0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 shutdown_connections 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e135440 0x55915e137830 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e1214f0 0x55915e1257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e127f90 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1839461106 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e11c0b0 0x55915e125290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 >> 172.21.15.73:0/1839461106 conn(0x55915e044100 msgr2=0x55915e111780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f478aff5700 0 client.15852 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 shutdown_connections 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1839461106 wait complete. 2024-06-06T07:03:02.453 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.450+0000 7f47b7a1eb80 20 client.15852 trim_cache size 0 max 16384 2024-06-06T07:03:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.470+0000 7f47b7a1eb80 1 Processor -- start 2024-06-06T07:03:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 -- start start 2024-06-06T07:03:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.472 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e111f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ac390 con 0x55915e11c0b0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81adf00 con 0x55915e11b6f0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81aca60 con 0x55915e1214f0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54214/0 (socket says 172.21.15.73:54214) 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 -- 172.21.15.73:0/2517820770 learned_addr learned my addr 172.21.15.73:0/2517820770 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a759e700 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e111f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 -- 172.21.15.73:0/2517820770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 msgr2=0x55915e111f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e111f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 -- 172.21.15.73:0/2517820770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 msgr2=0x55915e131790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e131790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 -- 172.21.15.73:0/2517820770 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55915e1131c0 con 0x55915e11c0b0 2024-06-06T07:03:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f47a659c700 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f479c09b800 tx=0x7f479c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac2c52d6f336b4f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.471+0000 7f478cff9700 1 -- 172.21.15.73:0/2517820770 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c01c040 con 0x55915e11c0b0 2024-06-06T07:03:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f478cff9700 1 -- 172.21.15.73:0/2517820770 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c0b3070 con 0x55915e11c0b0 2024-06-06T07:03:02.474 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f478cff9700 1 -- 172.21.15.73:0/2517820770 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479c004a40 con 0x55915e11c0b0 2024-06-06T07:03:02.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/2517820770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 msgr2=0x55915e1271d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f479c09b800 tx=0x7f479c086040 comp rx=0 tx=0).stop 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/2517820770 shutdown_connections 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e111f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11b6f0 0x55915e131790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/2517820770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11c0b0 0x55915e1271d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/2517820770 >> 172.21.15.73:0/2517820770 conn(0x55915e044100 msgr2=0x55915e126ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/2517820770 shutdown_connections 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.472+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/2517820770 wait complete. 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 -- start start 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 Processor -- start 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 0x55915e12ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.477 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e12a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad830 con 0x55915e11b6f0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ad130 con 0x55915e11c0b0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81aa330 con 0x55915e1214f0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47b7a1eb80 10 client.0 _ll_register_callbacks cb 0x55915dffe630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47a6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54218/0 (socket says 172.21.15.73:54218) 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47a6d9d700 1 -- 172.21.15.73:0/1886308167 learned_addr learned my addr 172.21.15.73:0/1886308167 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47a759e700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e12a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.473+0000 7f47a659c700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 0x55915e12ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 -- 172.21.15.73:0/1886308167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 msgr2=0x55915e12ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 0x55915e12ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 -- 172.21.15.73:0/1886308167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 msgr2=0x55915e12d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:02.478 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 -- 172.21.15.73:0/1886308167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55915e12acf0 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a659c700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 0x55915e12ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47a759e700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e12a780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a000bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8daa731b2c669cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f478aff5700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0006040 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f47a00074e0 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f479c0b3070 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f47a0007720 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47b7a1eb80 10 client.25399 fetch_fsmap learned FSMap version 33 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55915e1a0630 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.474+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f479c0b3070 con 0x55915e1214f0 2024-06-06T07:03:02.479 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 33) ==== 8366+0+0 (secure 0 0 0) 0x7f47a0011070 con 0x55915e1214f0 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 10 client.25399 fetch_fsmap finished waiting for FSMap version 33 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 20 client.25399 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e1a0d10 0x55915e1a3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 4 client.25399 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55915e070350 con 0x55915e1a0d10 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e1a0d10 0x55915e1a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f47a6d9d700 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e1a0d10 0x55915e1a3130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6be4a7c53225831 server_cookie=4c81c5b605b09daf in_seq=0 out_seq=0 2024-06-06T07:03:02.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.475+0000 7f478aff5700 10 client.25399 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f478aff5700 1 -- 172.21.15.73:0/1886308167 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55915e070350 con 0x55915e1a0d10 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f478aff5700 10 client.25399 handle_command_reply: tid=0 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 client.25399 shutdown 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 msgr2=0x55915e12a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e12a780 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f47a0009fd0 tx=0x7f47a000bf60 comp rx=0 tx=0).stop 2024-06-06T07:03:02.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 shutdown_connections 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55915e1a0d10 0x55915e1a3130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55915e1214f0 0x55915e12a780 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55915e11b6f0 0x55915e12d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f478aff5700 0 client.25399 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 --2- 172.21.15.73:0/1886308167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55915e11c0b0 0x55915e12ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 >> 172.21.15.73:0/1886308167 conn(0x55915e044100 msgr2=0x55915e11f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 shutdown_connections 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 1 -- 172.21.15.73:0/1886308167 wait complete. 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.477+0000 7f47b7a1eb80 20 client.25399 trim_cache size 0 max 16384 2024-06-06T07:03:02.482 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stdout: "return_code": 0, 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_tag": "645bda2b-50d6-4be0-a3a1-d7deedbf542e", 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": "asynchronous" 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 msgr2=0x7f47900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 0x7f47900a5250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f47a8199ac0 tx=0x7f4798009260 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 msgr2=0x7f47a819ed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 0x7f47a819ed00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f479c00e9f0 tx=0x7f479c00ed00 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 shutdown_connections 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f47900a2d90 0x7f47900a5250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f47a81084f0 0x7f47a8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47a806aa00 0x7f47a819e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 --2- 172.21.15.73:0/3727379508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f47a806b3c0 0x7f47a819ed00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 >> 172.21.15.73:0/3727379508 conn(0x7f47a8078300 msgr2=0x7f47a81000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 shutdown_connections 2024-06-06T07:03:02.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:02.478+0000 7f47ad19b700 1 -- 172.21.15.73:0/3727379508 wait complete. 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.487+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020b [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC auth v167 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.544702+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15294 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa4fb80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020b:[<0x10000000208/FACTS.DOC v46>,<0x10000000009/WORD v390>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x1000000020b:[<0x10000000208/FACTS.DOC v167>,<0x10000000009/WORD v1327>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.488+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000209 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC auth v104 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.528328+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15294 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa50100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000209:[<0x10000000208/CHAP10.DOC v8>,<0x10000000009/WORD v390>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000209:[<0x10000000208/CHAP10.DOC v104>,<0x10000000009/WORD v1327>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.488+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020a [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC auth v140 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.537095+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@19},l=15294 | ptrwaiter=0 request=0 lock=0 importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa4f600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x1000000020a:[<0x10000000208/BASEMACH.DOC v30>,<0x10000000009/WORD v390>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x1000000020a:[<0x10000000208/BASEMACH.DOC v140>,<0x10000000009/WORD v1327>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.488+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000222 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP auth v8 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.699627+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa45180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000222:[<0x10000000221/BENCHS.LWP v2>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000222:[<0x10000000221/BENCHS.LWP v8>,<0x10000000009/WORDPRO v1097>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.488+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000223 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS auth v22 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.707653+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa45700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000223:[<0x10000000221/RESULTS.XLS v4>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000223:[<0x10000000221/RESULTS.XLS v22>,<0x10000000009/WORDPRO v1097>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.488+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000224 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP auth v46 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.714872+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa4f080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000224:[<0x10000000221/NEWS1_1.LWP v6>,<0x10000000009/WORDPRO v58>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000224:[<0x10000000221/NEWS1_1.LWP v46>,<0x10000000009/WORDPRO v1097>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.489+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000226 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS auth v82 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.727288+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa50c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000226:[<0x10000000225/T1.XLS v2>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000226:[<0x10000000225/T1.XLS v82>,<0x10000000009/PM v1057>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.490+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000228 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC auth v170 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.743430+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa82000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v6>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000228:[<0x10000000225/MOVED.DOC v170>,<0x10000000009/PM v1057>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:02.848 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:03:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:03:02.490+0000 7f2b15f17700 -1 mds.1.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000227 [3,head] /volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC auth v164 ap=1 s=260096 n(v0 rc2024-06-06T07:02:53.735576+0000 b260096 1=1+0) (iauth excl) (ifile excl) (ixattr excl) cr={15294=0-4194304@2} caps={15294=pAsxLsXsxFsxcrwb/pAsxXsxFsxcrwb@2},l=15294 | importingcaps=0 caps=1 replicated=0 dirty=1 authpin=1 scrubqueue=0 0x559f2aa51180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v4>,<0x10000000009/PM v66>,<0x10000000008/~dmtmp v34>,<0x10000000007/client0 v18>,<0x10000000006/clients v8>,<0x10000000005/tmp v8>,<0x10000000003/client.0 v10>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v26>,<0x10000000001/sv_0 v49>,<0x10000000000/qa v36>,<0x1/volumes v37>]//[3]","memoryvalue":"(4)0x10000000227:[<0x10000000225/EVENTS.DOC v164>,<0x10000000009/PM v1057>,<0x10000000008/~dmtmp v41>,<0x10000000007/client0 v29>,<0x10000000006/clients v19>,<0x10000000005/tmp v21>,<0x10000000003/client.0 v20>,<0x10000000002/f625dafc-a4d8-4858-b05b-5ad87ae2f66d v32>,<0x10000000001/sv_0 v51>,<0x10000000000/qa v26>,<0x1/volumes v24>]//[3]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-06-06T07:03:03.313 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:03:03.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 6873 5.45 MB/sec warmup 10 sec latency 15.100 ms 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: pgmap v54: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub queued for path: / 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: active paths [/] 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: active paths [/] 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: active paths [/] 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:03 smithi073 ceph-mon[32303]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: pgmap v54: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub queued for path: / 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: active paths [/] 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: active paths [/] 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: active paths [/] 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.553 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:03 smithi136 ceph-mon[35901]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: pgmap v54: 129 pgs: 129 active+clean; 25 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 2.2 MiB/s wr, 79 op/s 2024-06-06T07:03:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub queued for path: / 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: active paths [/] 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: active paths [/] 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000209 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x1000000020a (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000222 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000223 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000224 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: active paths [/] 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000226 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000228 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: Scrub error on inode 0x10000000227 (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:03:03.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:03 smithi170 ceph-mon[36108]: scrub summary: idle+waiting paths [/] 2024-06-06T07:03:03.614 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:04.271 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:03:04.218+0000 7f12df37d700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:03:04.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 7402 5.35 MB/sec warmup 11 sec latency 14.653 ms 2024-06-06T07:03:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:04 smithi073 ceph-mon[32303]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:03:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:04 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T07:03:04.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:04 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:04.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:04 smithi136 ceph-mon[35901]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:03:04.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:04 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T07:03:04.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:04 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:04.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:04 smithi170 ceph-mon[36108]: Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-06-06T07:03:04.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:04 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] up:active 2024-06-06T07:03:04.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:04 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:03:05.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 7685 4.93 MB/sec warmup 12 sec latency 46.587 ms 2024-06-06T07:03:05.473 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:05 smithi073 ceph-mon[32303]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-06-06T07:03:05.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 -- 172.21.15.73:0/891845125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 msgr2=0x7f4fcc1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/891845125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc1077e0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009990 tx=0x7f4fc8009ca0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 -- 172.21.15.73:0/891845125 shutdown_connections 2024-06-06T07:03:05.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/891845125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 0x7f4fcc10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/891845125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc1077e0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/891845125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 0x7f4fcc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.472+0000 7f4fd32c3700 1 -- 172.21.15.73:0/891845125 >> 172.21.15.73:0/891845125 conn(0x7f4fcc078340 msgr2=0x7f4fcc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.473+0000 7f4fd32c3700 1 -- 172.21.15.73:0/891845125 shutdown_connections 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.473+0000 7f4fd32c3700 1 -- 172.21.15.73:0/891845125 wait complete. 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.473+0000 7f4fd32c3700 1 Processor -- start 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 -- start start 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 0x7f4fcc19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 0x7f4fcc19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc111550 con 0x7f4fcc107400 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1113d0 con 0x7f4fcc101d50 2024-06-06T07:03:05.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd32c3700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1116d0 con 0x7f4fcc101390 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50462/0 (socket says 172.21.15.73:50462) 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 -- 172.21.15.73:0/575717519 learned_addr learned my addr 172.21.15.73:0/575717519 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd085e700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 0x7f4fcc19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd105f700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 0x7f4fcc19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 -- 172.21.15.73:0/575717519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 msgr2=0x7f4fcc19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 0x7f4fcc19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 -- 172.21.15.73:0/575717519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 msgr2=0x7f4fcc19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 0x7f4fcc19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.474+0000 7f4fd1860700 1 -- 172.21.15.73:0/575717519 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc1990f0 con 0x7f4fcc107400 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fd1860700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007d70 tx=0x7f4fc400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fc27fc700 1 -- 172.21.15.73:0/575717519 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4016020 con 0x7f4fcc107400 2024-06-06T07:03:05.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fc27fc700 1 -- 172.21.15.73:0/575717519 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc400ac90 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc1993e0 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fc27fc700 1 -- 172.21.15.73:0/575717519 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40115e0 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.475+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc1a5650 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.477+0000 7f4fc27fc700 1 -- 172.21.15.73:0/575717519 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4fc4019020 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.478+0000 7f4fc27fc700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 0x7f4fb80a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.478+0000 7f4fd105f700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 0x7f4fb80a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.478+0000 7f4fc27fc700 1 -- 172.21.15.73:0/575717519 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4fc40cf610 con 0x7f4fcc107400 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.479+0000 7f4fd105f700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 0x7f4fb80a5250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b2e0 tx=0x7f4fc801a040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.479+0000 7f4fddb46b80 1 Processor -- start 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.479+0000 7f4fddb46b80 1 -- start start 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.479+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa1a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa1b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56334a96c810 con 0x56334aa20640 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56334a914720 con 0x56334aa20f60 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56334a9687d0 con 0x56334aa1af20 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd105f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd105f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50466/0 (socket says 172.21.15.73:50466) 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd105f700 1 -- 172.21.15.73:0/1759067679 learned_addr learned my addr 172.21.15.73:0/1759067679 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa1b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd085e700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa1a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 -- 172.21.15.73:0/1759067679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 msgr2=0x56334aa1a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa1a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 -- 172.21.15.73:0/1759067679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 msgr2=0x56334aa20a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 -- 172.21.15.73:0/1759067679 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334a90af50 con 0x56334aa1af20 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd105f700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:05.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa1b340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40ccac0 tx=0x7f4fc409b1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e648adbb41eca767 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.480+0000 7f4fb77fe700 1 -- 172.21.15.73:0/1759067679 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc409b890 con 0x56334aa1af20 2024-06-06T07:03:05.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fb77fe700 1 -- 172.21.15.73:0/1759067679 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc409fc40 con 0x56334aa1af20 2024-06-06T07:03:05.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fb77fe700 1 -- 172.21.15.73:0/1759067679 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc409baf0 con 0x56334aa1af20 2024-06-06T07:03:05.489 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1759067679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 msgr2=0x56334aa1b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa1b340 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40ccac0 tx=0x7f4fc409b1f0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1759067679 shutdown_connections 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa1b340 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa20a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1759067679 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa1a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1759067679 >> 172.21.15.73:0/1759067679 conn(0x56334a945cc0 msgr2=0x56334aa15850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.481+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1759067679 shutdown_connections 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1759067679 wait complete. 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 -- start start 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 Processor -- start 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa26d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa24080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56334aa279d0 con 0x56334aa20f60 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56334a90cbf0 con 0x56334aa1af20 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56334aa27b50 con 0x56334aa20640 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.482+0000 7f4fddb46b80 10 client.0 _ll_register_callbacks cb 0x56334a906920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50476/0 (socket says 172.21.15.73:50476) 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 -- 172.21.15.73:0/1308802436 learned_addr learned my addr 172.21.15.73:0/1308802436 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd105f700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa26d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd085e700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa24080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 -- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 msgr2=0x56334aa24080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa24080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 -- 172.21.15.73:0/1308802436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 msgr2=0x56334aa26d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.491 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa26d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 -- 172.21.15.73:0/1308802436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334a8bb8d0 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fd1860700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400a690 tx=0x7f4fc4018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1face66921e2f924 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc40b8070 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4fc40cf240 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.483+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fc409fc40 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4fc40cf440 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 10 client.15870 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56334aa279d0 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fc409fc40 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4fc40a4070 con 0x56334aa20f60 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 10 client.15870 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:05.492 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 20 client.15870 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aa34680 0x56334aa36ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 4 client.15870 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.484+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56334a979d10 con 0x56334aa34680 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.485+0000 7f4fd105f700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aa34680 0x56334aa36ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.485+0000 7f4fb4ff9700 10 client.15870 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.485+0000 7f4fd105f700 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aa34680 0x56334aa36ac0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbbad9ade169c40d server_cookie=3f3797d538c4e85c in_seq=0 out_seq=0 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/1308802436 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56334a979d10 con 0x56334aa34680 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fb4ff9700 10 client.15870 handle_command_reply: tid=0 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 client.15870 shutdown 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 msgr2=0x56334aa245c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4fc400a690 tx=0x7f4fc4018fb0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 shutdown_connections 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aa34680 0x56334aa36ac0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa245c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa26d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/1308802436 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa24080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fb4ff9700 0 client.15870 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 >> 172.21.15.73:0/1308802436 conn(0x56334a945cc0 msgr2=0x56334aa199d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 shutdown_connections 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 1 -- 172.21.15.73:0/1308802436 wait complete. 2024-06-06T07:03:05.494 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.486+0000 7f4fddb46b80 20 client.15870 trim_cache size 0 max 16384 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 Processor -- start 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 -- start start 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa30c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa1a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa21af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1ac300 con 0x56334aa20f60 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1adea0 con 0x56334aa1af20 2024-06-06T07:03:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1ac9d0 con 0x56334aa20640 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.511+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa21af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa1a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd1860700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa21af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50490/0 (socket says 172.21.15.73:50490) 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd1860700 1 -- 172.21.15.73:0/20067213 learned_addr learned my addr 172.21.15.73:0/20067213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd105f700 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa30c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 -- 172.21.15.73:0/20067213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 msgr2=0x56334aa30c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa30c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 -- 172.21.15.73:0/20067213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 msgr2=0x56334aa21af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa21af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 -- 172.21.15.73:0/20067213 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56334aa22030 con 0x56334aa20640 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fd085e700 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa1a9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56334aa25650 tx=0x7f4fbc009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3c63c964850fa8a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fb6ffd700 1 -- 172.21.15.73:0/20067213 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc013a00 con 0x56334aa20640 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fb6ffd700 1 -- 172.21.15.73:0/20067213 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fbc004ce0 con 0x56334aa20640 2024-06-06T07:03:05.514 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.512+0000 7f4fb6ffd700 1 -- 172.21.15.73:0/20067213 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc0055b0 con 0x56334aa20640 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- 172.21.15.73:0/20067213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 msgr2=0x56334aa1a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa1a9e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x56334aa25650 tx=0x7f4fbc009f60 comp rx=0 tx=0).stop 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- 172.21.15.73:0/20067213 shutdown_connections 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20f60 0x56334aa21af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa1af20 0x56334aa30c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/20067213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa20640 0x56334aa1a9e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- 172.21.15.73:0/20067213 >> 172.21.15.73:0/20067213 conn(0x56334a945cc0 msgr2=0x56334aa213e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.518 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- 172.21.15.73:0/20067213 shutdown_connections 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- 172.21.15.73:0/20067213 wait complete. 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.513+0000 7f4fddb46b80 1 -- start start 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 Processor -- start 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa2d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa29db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1ad7d0 con 0x56334aa20640 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1ad0a0 con 0x56334aa20f60 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc1aa2c0 con 0x56334aa1af20 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fddb46b80 10 client.0 _ll_register_callbacks cb 0x56334a906920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd085e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa2d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.519 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:49846/0 (socket says 172.21.15.73:49846) 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 -- 172.21.15.73:0/2504813650 learned_addr learned my addr 172.21.15.73:0/2504813650 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd1860700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa29db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 -- 172.21.15.73:0/2504813650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 msgr2=0x56334aa29db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa29db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 -- 172.21.15.73:0/2504813650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 msgr2=0x56334aa2d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa2d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.514+0000 7f4fd105f700 1 -- 172.21.15.73:0/2504813650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56334aa2d910 con 0x56334aa1af20 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fd085e700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fd105f700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80098e0 tx=0x7f4fc8009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=861a86a056911f59 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fc8038020 con 0x56334aa1af20 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4fbc004ce0 con 0x56334aa1af20 2024-06-06T07:03:05.520 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4fc8003b50 con 0x56334aa1af20 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4fc8043050 con 0x56334aa1af20 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fddb46b80 10 client.25417 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.515+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56334aa9f930 con 0x56334aa1af20 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fbc004ce0 con 0x56334aa1af20 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4fc8034450 con 0x56334aa1af20 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 10 client.25417 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 10 client.25417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 20 client.25417 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aaa00a0 0x56334aaa24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 4 client.25417 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56334a974290 con 0x56334aaa00a0 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fd085e700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aaa00a0 0x56334aaa24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fd085e700 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aaa00a0 0x56334aaa24c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baaf6398b6ace1e9 server_cookie=38207008d2b8ce5 in_seq=0 out_seq=0 2024-06-06T07:03:05.521 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.516+0000 7f4fb4ff9700 10 client.25417 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fb4ff9700 1 -- 172.21.15.73:0/2504813650 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56334a974290 con 0x56334aaa00a0 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fb4ff9700 10 client.25417 handle_command_reply: tid=0 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 client.25417 shutdown 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 msgr2=0x56334aa2cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4fc80098e0 tx=0x7f4fc8009310 comp rx=0 tx=0).stop 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 shutdown_connections 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56334aaa00a0 0x56334aaa24c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56334aa1af20 0x56334aa2cde0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fb4ff9700 0 client.25417 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56334aa20f60 0x56334aa29db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 --2- 172.21.15.73:0/2504813650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56334aa20640 0x56334aa2d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 >> 172.21.15.73:0/2504813650 conn(0x56334a945cc0 msgr2=0x56334aa1f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.522 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 shutdown_connections 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 1 -- 172.21.15.73:0/2504813650 wait complete. 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.517+0000 7f4fddb46b80 20 client.25417 trim_cache size 0 max 16384 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "scrubs": { 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "645bda2b-50d6-4be0-a3a1-d7deedbf542e": { 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/", 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tag": "645bda2b-50d6-4be0-a3a1-d7deedbf542e", 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: "options": "recursive,force" 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:03:05.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 msgr2=0x7f4fb80a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 0x7f4fb80a5250 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b2e0 tx=0x7f4fc801a040 comp rx=0 tx=0).stop 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 msgr2=0x7f4fcc198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007d70 tx=0x7f4fc400a340 comp rx=0 tx=0).stop 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 shutdown_connections 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4fb80a2d90 0x7f4fb80a5250 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4fcc101390 0x7f4fcc19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.524 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fcc107400 0x7f4fcc198990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 --2- 172.21.15.73:0/575717519 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4fcc101d50 0x7f4fcc19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 >> 172.21.15.73:0/575717519 conn(0x7f4fcc078340 msgr2=0x7f4fcc100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 shutdown_connections 2024-06-06T07:03:05.525 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:05.518+0000 7f4fd32c3700 1 -- 172.21.15.73:0/575717519 wait complete. 2024-06-06T07:03:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:05 smithi136 ceph-mon[35901]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-06-06T07:03:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:05 smithi170 ceph-mon[36108]: pgmap v55: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 2.8 MiB/s wr, 151 op/s 2024-06-06T07:03:06.348 INFO:tasks.cephfs.filesystem:scrub status for tag:645bda2b-50d6-4be0-a3a1-d7deedbf542e - {'path': '/', 'tag': '645bda2b-50d6-4be0-a3a1-d7deedbf542e', 'options': 'recursive,force'} 2024-06-06T07:03:06.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 8279 4.71 MB/sec warmup 13 sec latency 17.491 ms 2024-06-06T07:03:07.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 9014 4.62 MB/sec warmup 14 sec latency 16.745 ms 2024-06-06T07:03:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:07 smithi073 ceph-mon[32303]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-06-06T07:03:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:07 smithi136 ceph-mon[35901]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-06-06T07:03:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:07 smithi170 ceph-mon[36108]: pgmap v56: 129 pgs: 129 active+clean; 26 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 2.8 MiB/s wr, 146 op/s 2024-06-06T07:03:08.432 INFO:tasks.workunit.client.0.smithi073.stdout: 1 9608 4.46 MB/sec warmup 15 sec latency 20.036 ms 2024-06-06T07:03:09.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 4.54 MB/sec warmup 16 sec latency 142.517 ms 2024-06-06T07:03:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 -- 172.21.15.73:0/513746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88106890 msgr2=0x7fee88106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/513746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88106890 0x7fee88106c70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fee78009930 tx=0x7fee7802a420 comp rx=0 tx=0).stop 2024-06-06T07:03:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 -- 172.21.15.73:0/513746252 shutdown_connections 2024-06-06T07:03:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/513746252 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee88101290 0x7fee8810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/513746252 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee881008d0 0x7fee88100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/513746252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88106890 0x7fee88106c70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 -- 172.21.15.73:0/513746252 >> 172.21.15.73:0/513746252 conn(0x7fee88077a20 msgr2=0x7fee88077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 -- 172.21.15.73:0/513746252 shutdown_connections 2024-06-06T07:03:09.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.542+0000 7fee8e3f9640 1 -- 172.21.15.73:0/513746252 wait complete. 2024-06-06T07:03:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 Processor -- start 2024-06-06T07:03:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 -- start start 2024-06-06T07:03:09.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee881008d0 0x7fee881981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 0x7fee8819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee88110b10 con 0x7fee88101290 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee88110990 con 0x7fee88106890 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee8e3f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee88110c90 con 0x7fee881008d0 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.543+0000 7fee877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50518/0 (socket says 172.21.15.73:50518) 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 -- 172.21.15.73:0/1054251538 learned_addr learned my addr 172.21.15.73:0/1054251538 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee8c996640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 0x7fee8819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee87fff640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee881008d0 0x7fee881981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 -- 172.21.15.73:0/1054251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee881008d0 msgr2=0x7fee881981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee881008d0 0x7fee881981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 -- 172.21.15.73:0/1054251538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 msgr2=0x7fee8819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 0x7fee8819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 -- 172.21.15.73:0/1054251538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8819d160 con 0x7fee88101290 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee8c996640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 0x7fee8819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee877fe640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9f0 tx=0x7fee7c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee857fa640 1 -- 172.21.15.73:0/1054251538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c00cd80 con 0x7fee88101290 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8819d450 con 0x7fee88101290 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee857fa640 1 -- 172.21.15.73:0/1054251538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee7c0108e0 con 0x7fee88101290 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee857fa640 1 -- 172.21.15.73:0/1054251538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c018920 con 0x7fee88101290 2024-06-06T07:03:09.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.544+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881a4ce0 con 0x7fee88101290 2024-06-06T07:03:09.556 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:09 smithi136 ceph-mon[35901]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-06T07:03:09.556 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:09 smithi136 ceph-mon[35901]: scrub summary: idle 2024-06-06T07:03:09.556 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:09 smithi136 ceph-mon[35901]: scrub summary: idle 2024-06-06T07:03:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee857fa640 1 -- 172.21.15.73:0/1054251538 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fee7c010a90 con 0x7fee88101290 2024-06-06T07:03:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 Processor -- start 2024-06-06T07:03:09.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 -- start start 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x555937495550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5559373f94f0 con 0x555937495a90 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555937397870 con 0x5559374a0350 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.547+0000 7fee911e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5559373f54b0 con 0x555937495170 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50534/0 (socket says 172.21.15.73:50534) 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 -- 172.21.15.73:0/3542786514 learned_addr learned my addr 172.21.15.73:0/3542786514 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee8c996640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x555937495550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee857fa640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 0x7fee700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee877fe640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 -- 172.21.15.73:0/3542786514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 msgr2=0x555937495550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x555937495550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 -- 172.21.15.73:0/3542786514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 msgr2=0x5559374a2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 -- 172.21.15.73:0/3542786514 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559373b39b0 con 0x555937495a90 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee877fe640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee8c996640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 0x7fee700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee857fa640 1 -- 172.21.15.73:0/1054251538 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fee7c0cfad0 con 0x7fee88101290 2024-06-06T07:03:09.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee87fff640 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee7802a940 tx=0x7fee78005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c226bc37f3089b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee6e7fc640 1 -- 172.21.15.73:0/3542786514 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78040070 con 0x555937495a90 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee6e7fc640 1 -- 172.21.15.73:0/3542786514 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee78004030 con 0x555937495a90 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.548+0000 7fee6e7fc640 1 -- 172.21.15.73:0/3542786514 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78035040 con 0x555937495a90 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- 172.21.15.73:0/3542786514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 msgr2=0x55593749fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee8c996640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 0x7fee700a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee800094d0 tx=0x7fee80014040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fee7802a940 tx=0x7fee78005be0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- 172.21.15.73:0/3542786514 shutdown_connections 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495a90 0x55593749fe10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 --2- 172.21.15.73:0/3542786514 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x555937495550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- 172.21.15.73:0/3542786514 >> 172.21.15.73:0/3542786514 conn(0x555937491680 msgr2=0x555937491a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:09.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- 172.21.15.73:0/3542786514 shutdown_connections 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- 172.21.15.73:0/3542786514 wait complete. 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.549+0000 7fee911e4740 1 -- start start 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 Processor -- start 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x55593749d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x55593749dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5559374a7330 con 0x555937495170 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555937382ae0 con 0x5559374a0350 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5559374a74b0 con 0x555937495a90 2024-06-06T07:03:09.562 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee911e4740 10 client.0 _ll_register_callbacks cb 0x5559373bb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50536/0 (socket says 172.21.15.73:50536) 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 -- 172.21.15.73:0/602570120 learned_addr learned my addr 172.21.15.73:0/602570120 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee8c996640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x55593749dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee877fe640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x55593749d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 -- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 msgr2=0x55593749d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x55593749d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 -- 172.21.15.73:0/602570120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 msgr2=0x55593749dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x55593749dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.550+0000 7fee87fff640 1 -- 172.21.15.73:0/602570120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593710d210 con 0x555937495170 2024-06-06T07:03:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee87fff640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee78004490 tx=0x7fee780044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765aa9641ae4e48d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78040030 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fee780051d0 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee78004030 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee78056050 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 10 client.15888 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5559371d9620 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee78004030 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fee780035f0 con 0x555937495170 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 10 client.15888 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 20 client.15888 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5559374b45e0 0x5559374b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 4 client.15888 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55593740cf10 con 0x5559374b45e0 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.551+0000 7fee877fe640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5559374b45e0 0x5559374b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.552+0000 7fee877fe640 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5559374b45e0 0x5559374b69d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0a0bf68d06dd850 server_cookie=fa4a29b4cccb8031 in_seq=0 out_seq=0 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.552+0000 7fee677fe640 10 client.15888 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.552+0000 7fee677fe640 1 -- 172.21.15.73:0/602570120 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55593740cf10 con 0x5559374b45e0 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.552+0000 7fee677fe640 10 client.15888 handle_command_reply: tid=0 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.552+0000 7fee911e4740 1 client.15888 shutdown 2024-06-06T07:03:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 msgr2=0x55593749d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fee78004490 tx=0x7fee780044c0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 shutdown_connections 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5559374b45e0 0x5559374b69d0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x55593749dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x55593749d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee677fe640 0 client.15888 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 --2- 172.21.15.73:0/602570120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x55593749d200 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 >> 172.21.15.73:0/602570120 conn(0x555937491680 msgr2=0x5559374973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 shutdown_connections 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 1 -- 172.21.15.73:0/602570120 wait complete. 2024-06-06T07:03:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.553+0000 7fee911e4740 20 client.15888 trim_cache size 0 max 16384 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 Processor -- start 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 -- start start 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x55593749c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555937495a90 0x5559374bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee881acac0 con 0x5559374a0350 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ae660 con 0x555937495a90 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee911e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ad190 con 0x555937495170 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50552/0 (socket says 172.21.15.73:50552) 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 -- 172.21.15.73:0/1910558710 learned_addr learned my addr 172.21.15.73:0/1910558710 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee87fff640 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x55593749c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 -- 172.21.15.73:0/1910558710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 msgr2=0x55593749c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x55593749c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 -- 172.21.15.73:0/1910558710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555937495a90 msgr2=0x5559374bc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee877fe640 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555937495a90 0x5559374bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555937495a90 0x5559374bc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 -- 172.21.15.73:0/1910558710 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559374958c0 con 0x5559374a0350 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee8c996640 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fee80009eb0 tx=0x7fee80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d420f8043079c4f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.568+0000 7fee6dffb640 1 -- 172.21.15.73:0/1910558710 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80012d50 con 0x5559374a0350 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee6dffb640 1 -- 172.21.15.73:0/1910558710 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee8001c930 con 0x5559374a0350 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee6dffb640 1 -- 172.21.15.73:0/1910558710 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80010a30 con 0x5559374a0350 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- 172.21.15.73:0/1910558710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 msgr2=0x5559374bc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fee80009eb0 tx=0x7fee80017000 comp rx=0 tx=0).stop 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- 172.21.15.73:0/1910558710 shutdown_connections 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5559374a0350 0x5559374bc5a0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555937495a90 0x5559374bc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 --2- 172.21.15.73:0/1910558710 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495170 0x55593749c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- 172.21.15.73:0/1910558710 >> 172.21.15.73:0/1910558710 conn(0x555937491680 msgr2=0x55593749bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- 172.21.15.73:0/1910558710 shutdown_connections 2024-06-06T07:03:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- 172.21.15.73:0/1910558710 wait complete. 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.569+0000 7fee911e4740 1 -- start start 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 Processor -- start 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x5559374acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee881adf90 con 0x555937495170 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee881ad860 con 0x5559374a0350 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee881aaaa0 con 0x555937495a90 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee911e4740 10 client.0 _ll_register_callbacks cb 0x7fee78009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50564/0 (socket says 172.21.15.73:50564) 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 -- 172.21.15.73:0/184734779 learned_addr learned my addr 172.21.15.73:0/184734779 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 -- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 msgr2=0x5559374acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee877fe640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x5559374acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x5559374acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee8c996640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 -- 172.21.15.73:0/184734779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 msgr2=0x5559374a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 -- 172.21.15.73:0/184734779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559374a8640 con 0x555937495170 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee8c996640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee87fff640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee7802ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7366ccaa76742767 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.570+0000 7fee677fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78040040 con 0x555937495170 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fee78003c30 con 0x555937495170 2024-06-06T07:03:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8001c930 con 0x555937495170 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fee78035040 con 0x555937495170 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 10 client.15894 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55593751ebe0 con 0x555937495170 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee8001c930 con 0x555937495170 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fee78004770 con 0x555937495170 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 10 client.15894 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 20 client.15894 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55593751f3e0 0x555937521800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 4 client.15894 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555937406c20 con 0x55593751f3e0 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.571+0000 7fee877fe640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55593751f3e0 0x555937521800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.572+0000 7fee877fe640 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55593751f3e0 0x555937521800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36afdcfecced1a1b server_cookie=7e56f46404339abe in_seq=0 out_seq=0 2024-06-06T07:03:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:09.572+0000 7fee677fe640 10 client.15894 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:09 smithi170 ceph-mon[36108]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-06T07:03:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:09 smithi170 ceph-mon[36108]: scrub summary: idle 2024-06-06T07:03:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:09 smithi170 ceph-mon[36108]: scrub summary: idle 2024-06-06T07:03:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:09 smithi073 ceph-mon[32303]: pgmap v57: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 401 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-06T07:03:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:09 smithi073 ceph-mon[32303]: scrub summary: idle 2024-06-06T07:03:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:09 smithi073 ceph-mon[32303]: scrub summary: idle 2024-06-06T07:03:10.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 4.27 MB/sec warmup 17 sec latency 1142.607 ms 2024-06-06T07:03:11.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 4.03 MB/sec warmup 18 sec latency 2142.705 ms 2024-06-06T07:03:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:11 smithi136 ceph-mon[35901]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:11 smithi170 ceph-mon[36108]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:11.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:11 smithi073 ceph-mon[32303]: pgmap v58: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:12.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 3.82 MB/sec warmup 19 sec latency 3142.823 ms 2024-06-06T07:03:13.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 3.63 MB/sec warmup 20 sec latency 4142.903 ms 2024-06-06T07:03:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:13 smithi136 ceph-mon[35901]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:13 smithi170 ceph-mon[36108]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:13 smithi073 ceph-mon[32303]: pgmap v59: 129 pgs: 129 active+clean; 28 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.3 MiB/s wr, 128 op/s 2024-06-06T07:03:14.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10331 3.46 MB/sec warmup 21 sec latency 5143.042 ms 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.735+0000 7fee677fe640 1 -- 172.21.15.73:0/184734779 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555937406c20 con 0x55593751f3e0 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.735+0000 7fee677fe640 10 client.15894 handle_command_reply: tid=0 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.735+0000 7fee911e4740 1 client.15894 shutdown 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 msgr2=0x5559374aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fee78000c00 tx=0x7fee7802ab50 comp rx=0 tx=0).stop 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 shutdown_connections 2024-06-06T07:03:14.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55593751f3e0 0x555937521800 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5559374a0350 0x5559374a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555937495a90 0x5559374acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 --2- 172.21.15.73:0/184734779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555937495170 0x5559374aca00 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee677fe640 0 client.15894 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 >> 172.21.15.73:0/184734779 conn(0x555937491680 msgr2=0x55593749c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 shutdown_connections 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 1 -- 172.21.15.73:0/184734779 wait complete. 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.736+0000 7fee911e4740 20 client.15894 trim_cache size 0 max 16384 2024-06-06T07:03:14.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:03:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:03:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:03:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 msgr2=0x7fee700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 0x7fee700a5260 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fee800094d0 tx=0x7fee80014040 comp rx=0 tx=0).stop 2024-06-06T07:03:14.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 msgr2=0x7fee88198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fee7c00e9f0 tx=0x7fee7c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 shutdown_connections 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee88106890 0x7fee8819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee88101290 0x7fee88198700 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fee700a2da0 0x7fee700a5260 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 --2- 172.21.15.73:0/1054251538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee881008d0 0x7fee881981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 >> 172.21.15.73:0/1054251538 conn(0x7fee88077a20 msgr2=0x7fee88069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 shutdown_connections 2024-06-06T07:03:14.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:14.737+0000 7fee8e3f9640 1 -- 172.21.15.73:0/1054251538 wait complete. 2024-06-06T07:03:15.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10493 3.35 MB/sec warmup 22 sec latency 5448.116 ms 2024-06-06T07:03:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:15 smithi136 ceph-mon[35901]: pgmap v60: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-06-06T07:03:15.576 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:15 smithi073 ceph-mon[32303]: pgmap v60: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-06-06T07:03:15.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:15 smithi170 ceph-mon[36108]: pgmap v60: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 1.6 MiB/s wr, 164 op/s 2024-06-06T07:03:16.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 10961 3.39 MB/sec warmup 23 sec latency 13.223 ms 2024-06-06T07:03:17.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 11258 3.26 MB/sec warmup 24 sec latency 14.799 ms 2024-06-06T07:03:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:17 smithi136 ceph-mon[35901]: pgmap v61: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-06-06T07:03:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:17 smithi170 ceph-mon[36108]: pgmap v61: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-06-06T07:03:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:17 smithi073 ceph-mon[32303]: pgmap v61: 129 pgs: 129 active+clean; 30 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 984 KiB/s wr, 92 op/s 2024-06-06T07:03:18.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 11757 3.21 MB/sec warmup 25 sec latency 15.489 ms 2024-06-06T07:03:19.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 12342 3.21 MB/sec warmup 26 sec latency 15.986 ms 2024-06-06T07:03:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:19 smithi136 ceph-mon[35901]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-06T07:03:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:19 smithi170 ceph-mon[36108]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-06T07:03:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:19 smithi073 ceph-mon[32303]: pgmap v62: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 530 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-06T07:03:20.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 13013 3.16 MB/sec warmup 27 sec latency 15.428 ms 2024-06-06T07:03:21.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 13588 3.24 MB/sec warmup 28 sec latency 18.283 ms 2024-06-06T07:03:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:21 smithi136 ceph-mon[35901]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:21 smithi170 ceph-mon[36108]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:21 smithi073 ceph-mon[32303]: pgmap v63: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:22.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 14053 3.21 MB/sec warmup 29 sec latency 15.167 ms 2024-06-06T07:03:23.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 14517 3.24 MB/sec warmup 30 sec latency 14.091 ms 2024-06-06T07:03:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:23 smithi136 ceph-mon[35901]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:23 smithi170 ceph-mon[36108]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:23 smithi073 ceph-mon[32303]: pgmap v64: 129 pgs: 129 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 674 KiB/s wr, 81 op/s 2024-06-06T07:03:24.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 14805 3.14 MB/sec warmup 31 sec latency 12.318 ms 2024-06-06T07:03:25.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 15317 3.11 MB/sec warmup 32 sec latency 11.980 ms 2024-06-06T07:03:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:25 smithi136 ceph-mon[35901]: pgmap v65: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-06T07:03:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:25 smithi170 ceph-mon[36108]: pgmap v65: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-06T07:03:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:25 smithi073 ceph-mon[32303]: pgmap v65: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-06T07:03:26.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 15899 3.11 MB/sec warmup 33 sec latency 14.902 ms 2024-06-06T07:03:27.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 16549 3.07 MB/sec warmup 34 sec latency 16.970 ms 2024-06-06T07:03:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:27 smithi136 ceph-mon[35901]: pgmap v66: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 954 KiB/s wr, 120 op/s 2024-06-06T07:03:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:27 smithi170 ceph-mon[36108]: pgmap v66: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 954 KiB/s wr, 120 op/s 2024-06-06T07:03:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:27 smithi073 ceph-mon[32303]: pgmap v66: 129 pgs: 129 active+clean; 29 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 954 KiB/s wr, 120 op/s 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 -- 172.21.15.73:0/1807629695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec106890 msgr2=0x7f2aec106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 --2- 172.21.15.73:0/1807629695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec106890 0x7f2aec106c70 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009930 tx=0x7f2ae802a420 comp rx=0 tx=0).stop 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 -- 172.21.15.73:0/1807629695 shutdown_connections 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 --2- 172.21.15.73:0/1807629695 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec101290 0x7f2aec10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 --2- 172.21.15.73:0/1807629695 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec1008d0 0x7f2aec100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 --2- 172.21.15.73:0/1807629695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec106890 0x7f2aec106c70 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 -- 172.21.15.73:0/1807629695 >> 172.21.15.73:0/1807629695 conn(0x7f2aec077a20 msgr2=0x7f2aec077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 -- 172.21.15.73:0/1807629695 shutdown_connections 2024-06-06T07:03:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.839+0000 7f2af389a640 1 -- 172.21.15.73:0/1807629695 wait complete. 2024-06-06T07:03:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 Processor -- start 2024-06-06T07:03:27.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 -- start start 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec101290 0x7f2aec1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec106890 0x7f2aec19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110b50 con 0x7f2aec1008d0 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1109d0 con 0x7f2aec106890 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af389a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec110cd0 con 0x7f2aec101290 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39368/0 (socket says 172.21.15.73:39368) 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 -- 172.21.15.73:0/1119504998 learned_addr learned my addr 172.21.15.73:0/1119504998 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af0e35640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec101290 0x7f2aec1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec106890 0x7f2aec19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 -- 172.21.15.73:0/1119504998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec101290 msgr2=0x7f2aec1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec101290 0x7f2aec1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 -- 172.21.15.73:0/1119504998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec106890 msgr2=0x7f2aec19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec106890 0x7f2aec19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.840+0000 7f2af1636640 1 -- 172.21.15.73:0/1119504998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aec19d160 con 0x7f2aec1008d0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2af1636640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004770 tx=0x7f2ae80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2ae27fc640 1 -- 172.21.15.73:0/1119504998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae803f070 con 0x7f2aec1008d0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2ae27fc640 1 -- 172.21.15.73:0/1119504998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ae803b460 con 0x7f2aec1008d0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aec19d3f0 con 0x7f2aec1008d0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2ae27fc640 1 -- 172.21.15.73:0/1119504998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8035070 con 0x7f2aec1008d0 2024-06-06T07:03:27.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.841+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aec1a4ce0 con 0x7f2aec1008d0 2024-06-06T07:03:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.843+0000 7f2ae27fc640 1 -- 172.21.15.73:0/1119504998 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2ae803b610 con 0x7f2aec1008d0 2024-06-06T07:03:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 Processor -- start 2024-06-06T07:03:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 -- start start 2024-06-06T07:03:27.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aed9220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aed9760 0x55ff2aedbb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ae2e4f0 con 0x55ff2aeca070 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2adcc870 con 0x55ff2aed9760 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af65a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ff2ae2a4b0 con 0x55ff2aeca990 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af1e37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af1636640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aed9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af1e37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39378/0 (socket says 172.21.15.73:39378) 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.844+0000 7f2af1e37640 1 -- 172.21.15.73:0/3706988078 learned_addr learned my addr 172.21.15.73:0/3706988078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2ae27fc640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 0x7f2ad80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af0e35640 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aed9760 0x55ff2aedbb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 -- 172.21.15.73:0/3706988078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 msgr2=0x55ff2aed9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aed9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 -- 172.21.15.73:0/3706988078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aed9760 msgr2=0x55ff2aedbb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aed9760 0x55ff2aedbb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 -- 172.21.15.73:0/3706988078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2ade89b0 con 0x55ff2aeca070 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2ae27fc640 1 -- 172.21.15.73:0/1119504998 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2ae80efb70 con 0x7f2aec1008d0 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 0x7f2ad80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c4a0 tx=0x7f2ae400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbbb093a39d26ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2ad77fe640 1 -- 172.21.15.73:0/3706988078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4018640 con 0x55ff2aeca070 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2ad77fe640 1 -- 172.21.15.73:0/3706988078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ae401e070 con 0x55ff2aeca070 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 0x7f2ad80a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ae401d000 tx=0x7f2ae4017000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.845+0000 7f2ad77fe640 1 -- 172.21.15.73:0/3706988078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae400a840 con 0x55ff2aeca070 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 -- 172.21.15.73:0/3706988078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 msgr2=0x55ff2aeca450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400c4a0 tx=0x7f2ae400c960 comp rx=0 tx=0).stop 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 -- 172.21.15.73:0/3706988078 shutdown_connections 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aed9760 0x55ff2aedbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aed9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 --2- 172.21.15.73:0/3706988078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aeca450 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 -- 172.21.15.73:0/3706988078 >> 172.21.15.73:0/3706988078 conn(0x55ff2aec6680 msgr2=0x55ff2aec6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 -- 172.21.15.73:0/3706988078 shutdown_connections 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.846+0000 7f2af65a7740 1 -- 172.21.15.73:0/3706988078 wait complete. 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 -- start start 2024-06-06T07:03:27.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 Processor -- start 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aed6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ff2aedc390 con 0x55ff2aeca070 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ff2adb7ae0 con 0x55ff2aeca990 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ff2aedc510 con 0x55ff2aed9760 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af65a7740 10 client.0 _ll_register_callbacks cb 0x55ff2adf09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39388/0 (socket says 172.21.15.73:39388) 2024-06-06T07:03:27.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 -- 172.21.15.73:0/1538150479 learned_addr learned my addr 172.21.15.73:0/1538150479 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af0e35640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aed6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 -- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 msgr2=0x55ff2aed7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 -- 172.21.15.73:0/1538150479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 msgr2=0x55ff2aed6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aed6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.847+0000 7f2af1636640 1 -- 172.21.15.73:0/1538150479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2ab42210 con 0x55ff2aeca070 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af1636640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b86b0 tx=0x7f2ae8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ba7dc724467984 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8048040 con 0x55ff2aeca070 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2ae80ec3d0 con 0x55ff2aeca070 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae401e070 con 0x55ff2aeca070 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ae80044e0 con 0x55ff2aeca070 2024-06-06T07:03:27.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 10 client.15906 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff2ac0e620 con 0x55ff2aeca070 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ae401e070 con 0x55ff2aeca070 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ae80d3080 con 0x55ff2aeca070 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 10 client.15906 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 20 client.15906 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2aee98e0 0x55ff2aeebd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 4 client.15906 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.848+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff2ae41f10 con 0x55ff2aee98e0 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.849+0000 7f2af0e35640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2aee98e0 0x55ff2aeebd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.849+0000 7f2ad4ff9640 10 client.15906 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.849+0000 7f2af0e35640 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2aee98e0 0x55ff2aeebd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=594bfa4a310bdb3b server_cookie=c8a60b15f7aed4aa in_seq=0 out_seq=0 2024-06-06T07:03:27.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1538150479 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ff2ae41f10 con 0x55ff2aee98e0 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2ad4ff9640 10 client.15906 handle_command_reply: tid=0 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 client.15906 shutdown 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 msgr2=0x55ff2aed66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80b86b0 tx=0x7f2ae8009560 comp rx=0 tx=0).stop 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 shutdown_connections 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2aee98e0 0x55ff2aeebd80 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aed6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2ad4ff9640 0 client.15906 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1538150479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aed66b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 >> 172.21.15.73:0/1538150479 conn(0x55ff2aec6680 msgr2=0x55ff2aecc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 shutdown_connections 2024-06-06T07:03:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 1 -- 172.21.15.73:0/1538150479 wait complete. 2024-06-06T07:03:27.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.850+0000 7f2af65a7740 20 client.15906 trim_cache size 0 max 16384 2024-06-06T07:03:27.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 Processor -- start 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 -- start start 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aef1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1acc30 con 0x55ff2aeca070 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ae7d0 con 0x55ff2aeca990 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af65a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ad300 con 0x55ff2aed9760 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af1636640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39398/0 (socket says 172.21.15.73:39398) 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af1636640 1 -- 172.21.15.73:0/5204920 learned_addr learned my addr 172.21.15.73:0/5204920 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.865+0000 7f2af1e37640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af0e35640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aef1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 -- 172.21.15.73:0/5204920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 msgr2=0x55ff2aed7ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 -- 172.21.15.73:0/5204920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 msgr2=0x55ff2aef1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aef1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 -- 172.21.15.73:0/5204920 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2aeec400 con 0x55ff2aeca070 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af0e35640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aef1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2af1636640 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80bbc70 tx=0x7f2ae8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db1b1ec03699254 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2ad6ffd640 1 -- 172.21.15.73:0/5204920 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8048040 con 0x55ff2aeca070 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2ad6ffd640 1 -- 172.21.15.73:0/5204920 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ae80bc3e0 con 0x55ff2aeca070 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.866+0000 7f2ad6ffd640 1 -- 172.21.15.73:0/5204920 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80b8350 con 0x55ff2aeca070 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- 172.21.15.73:0/5204920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 msgr2=0x55ff2aef1690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2ae80bbc70 tx=0x7f2ae8005c90 comp rx=0 tx=0).stop 2024-06-06T07:03:27.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- 172.21.15.73:0/5204920 shutdown_connections 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aed9760 0x55ff2aed7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca990 0x55ff2aef1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- 172.21.15.73:0/5204920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aeca070 0x55ff2aef1690 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- 172.21.15.73:0/5204920 >> 172.21.15.73:0/5204920 conn(0x55ff2aec6680 msgr2=0x55ff2aed19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- 172.21.15.73:0/5204920 shutdown_connections 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- 172.21.15.73:0/5204920 wait complete. 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- start start 2024-06-06T07:03:27.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 Processor -- start 2024-06-06T07:03:27.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca070 0x55ff2aee1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aedd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ae100 con 0x55ff2aed9760 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1ad9d0 con 0x55ff2aeca070 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aec1aac10 con 0x55ff2aeca990 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.867+0000 7f2af65a7740 10 client.0 _ll_register_callbacks cb 0x7f2ae80f33b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39412/0 (socket says 172.21.15.73:39412) 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 -- 172.21.15.73:0/1449819606 learned_addr learned my addr 172.21.15.73:0/1449819606 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1636640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca070 0x55ff2aee1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af0e35640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aedd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 -- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 msgr2=0x55ff2aedd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aedd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 -- 172.21.15.73:0/1449819606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca070 msgr2=0x55ff2aee1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca070 0x55ff2aee1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 -- 172.21.15.73:0/1449819606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2aeddd50 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af1e37640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ce60 tx=0x7f2ae400a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e16e1386fdd37365 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae4031060 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ae80bc3e0 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2ae4018640 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2ae4009c20 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af65a7740 10 client.15912 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.868+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ff2af544e0 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ae80bc3e0 con 0x55ff2aed9760 2024-06-06T07:03:27.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2ae40224c0 con 0x55ff2aed9760 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 10 client.15912 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 20 client.15912 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2af54af0 0x55ff2af56f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 4 client.15912 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ff2ae3bc20 con 0x55ff2af54af0 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af1636640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2af54af0 0x55ff2af56f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2af1636640 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2af54af0 0x55ff2af56f10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a546dee68a8f18da server_cookie=4c1f028cdf463870 in_seq=0 out_seq=0 2024-06-06T07:03:27.875 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.869+0000 7f2ad4ff9640 10 client.15912 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:27.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2ad4ff9640 1 -- 172.21.15.73:0/1449819606 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ff2ae3bc20 con 0x55ff2af54af0 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2ad4ff9640 10 client.15912 handle_command_reply: tid=0 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 client.15912 shutdown 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 msgr2=0x55ff2aedd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2ae400ce60 tx=0x7f2ae400a7b0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 shutdown_connections 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ff2af54af0 0x55ff2af56f10 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ff2aed9760 0x55ff2aedd660 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ff2aeca990 0x55ff2aedd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 --2- 172.21.15.73:0/1449819606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ff2aeca070 0x55ff2aee1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 >> 172.21.15.73:0/1449819606 conn(0x55ff2aec6680 msgr2=0x55ff2aec4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.891+0000 7f2ad4ff9640 0 client.15912 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 shutdown_connections 2024-06-06T07:03:27.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af65a7740 1 -- 172.21.15.73:0/1449819606 wait complete. 2024-06-06T07:03:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af65a7740 20 client.15912 trim_cache size 0 max 16384 2024-06-06T07:03:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:03:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:03:27.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 msgr2=0x7f2ad80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 0x7f2ad80a5260 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2ae401d000 tx=0x7f2ae4017000 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 msgr2=0x7f2aec198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8004770 tx=0x7f2ae80042f0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 shutdown_connections 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aec106890 0x7f2aec19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2ad80a2da0 0x7f2ad80a5260 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aec101290 0x7f2aec1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 --2- 172.21.15.73:0/1119504998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aec1008d0 0x7f2aec198170 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 >> 172.21.15.73:0/1119504998 conn(0x7f2aec077a20 msgr2=0x7f2aec069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:27.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 shutdown_connections 2024-06-06T07:03:27.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:27.892+0000 7f2af389a640 1 -- 172.21.15.73:0/1119504998 wait complete. 2024-06-06T07:03:28.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 17109 3.14 MB/sec warmup 35 sec latency 75.002 ms 2024-06-06T07:03:29.433 INFO:tasks.workunit.client.0.smithi073.stdout: 1 17564 3.11 MB/sec warmup 36 sec latency 14.829 ms 2024-06-06T07:03:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:29 smithi136 ceph-mon[35901]: pgmap v67: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-06T07:03:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:29 smithi170 ceph-mon[36108]: pgmap v67: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-06T07:03:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:29 smithi073 ceph-mon[32303]: pgmap v67: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 196 op/s 2024-06-06T07:03:30.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 18019 3.14 MB/sec warmup 37 sec latency 15.242 ms 2024-06-06T07:03:31.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 18311 3.07 MB/sec warmup 38 sec latency 14.060 ms 2024-06-06T07:03:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:31 smithi136 ceph-mon[35901]: pgmap v68: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:31 smithi170 ceph-mon[36108]: pgmap v68: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:31 smithi073 ceph-mon[32303]: pgmap v68: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:32.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 18798 3.04 MB/sec warmup 39 sec latency 11.907 ms 2024-06-06T07:03:33.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 19341 3.04 MB/sec warmup 40 sec latency 19.896 ms 2024-06-06T07:03:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:33 smithi136 ceph-mon[35901]: pgmap v69: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:33 smithi170 ceph-mon[36108]: pgmap v69: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:33 smithi073 ceph-mon[32303]: pgmap v69: 129 pgs: 129 active+clean; 28 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:03:34.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 19970 2.99 MB/sec warmup 41 sec latency 25.350 ms 2024-06-06T07:03:34.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:34 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:03:34.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:34 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:03:34.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:34 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:03:35.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 20438 2.99 MB/sec warmup 42 sec latency 15.414 ms 2024-06-06T07:03:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:35 smithi170 ceph-mon[36108]: pgmap v70: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:03:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:35 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:03:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:35 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:03:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:35 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:03:35.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:35 smithi136 ceph-mon[35901]: pgmap v70: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:03:35.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:35 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:03:35.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:35 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:03:35.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:35 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:03:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:35 smithi073 ceph-mon[32303]: pgmap v70: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:03:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:35 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:03:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:35 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:03:35.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:35 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:03:36.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-06T07:03:36.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 21096 3.05 MB/sec warmup 43 sec latency 15.352 ms 2024-06-06T07:03:36.613 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:37.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 21537 3.08 MB/sec warmup 44 sec latency 15.283 ms 2024-06-06T07:03:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:37 smithi170 ceph-mon[36108]: pgmap v71: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-06T07:03:37.743 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:37 smithi073 ceph-mon[32303]: pgmap v71: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-06T07:03:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:37 smithi136 ceph-mon[35901]: pgmap v71: 129 pgs: 129 active+clean; 33 MiB data, 471 MiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-06T07:03:38.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 21782 3.02 MB/sec warmup 45 sec latency 32.277 ms 2024-06-06T07:03:38.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 -- 172.21.15.73:0/1712294896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 msgr2=0x7f03e810e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 --2- 172.21.15.73:0/1712294896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e810e260 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009990 tx=0x7f03e0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 -- 172.21.15.73:0/1712294896 shutdown_connections 2024-06-06T07:03:38.455 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 --2- 172.21.15.73:0/1712294896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 0x7f03e81067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 --2- 172.21.15.73:0/1712294896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e810e260 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 --2- 172.21.15.73:0/1712294896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e81023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 -- 172.21.15.73:0/1712294896 >> 172.21.15.73:0/1712294896 conn(0x7f03e8076730 msgr2=0x7f03e8076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 -- 172.21.15.73:0/1712294896 shutdown_connections 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.438+0000 7f03ece6a700 1 -- 172.21.15.73:0/1712294896 wait complete. 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.439+0000 7f03ece6a700 1 Processor -- start 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.439+0000 7f03ece6a700 1 -- start start 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 0x7f03e8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.456 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03e8111570 con 0x7f03e8102900 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81113f0 con 0x7f03e8101f40 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81116f0 con 0x7f03e81063c0 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60018/0 (socket says 172.21.15.73:60018) 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 -- 172.21.15.73:0/2209626868 learned_addr learned my addr 172.21.15.73:0/2209626868 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e659c700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 0x7f03e8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 -- 172.21.15.73:0/2209626868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 msgr2=0x7f03e8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 0x7f03e8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 -- 172.21.15.73:0/2209626868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 msgr2=0x7f03e819e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e819e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.457 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 -- 172.21.15.73:0/2209626868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03e8199080 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e659c700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f03d000e9a0 tx=0x7f03d000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03df7fe700 1 -- 172.21.15.73:0/2209626868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d000ca10 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03e8199370 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.440+0000 7f03df7fe700 1 -- 172.21.15.73:0/2209626868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03d000cbb0 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.441+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03e81a5630 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.441+0000 7f03df7fe700 1 -- 172.21.15.73:0/2209626868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d0018680 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.443+0000 7f03df7fe700 1 -- 172.21.15.73:0/2209626868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f03d0010430 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.444+0000 7f03df7fe700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 0x7f03d40a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.444+0000 7f03e659c700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 0x7f03d40a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.444+0000 7f03df7fe700 1 -- 172.21.15.73:0/2209626868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f03d00cf990 con 0x7f03e8102900 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.444+0000 7f03e659c700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 0x7f03d40a5250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f03e8199a60 tx=0x7f03d800b040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.458 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.444+0000 7f03f76edb80 1 Processor -- start 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 -- start start 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d414470 0x56448d414850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 0x56448d41b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56448d2fcf60 con 0x56448d40e670 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56448d35b850 con 0x56448d414470 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03f76edb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56448d2c3460 con 0x56448d40f030 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60028/0 (socket says 172.21.15.73:60028) 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e5d9b700 1 -- 172.21.15.73:0/2477803988 learned_addr learned my addr 172.21.15.73:0/2477803988 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 0x56448d41b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e659c700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d414470 0x56448d414850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.459 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 -- 172.21.15.73:0/2477803988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d414470 msgr2=0x56448d414850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d414470 0x56448d414850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 -- 172.21.15.73:0/2477803988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 msgr2=0x56448d40eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e6d9d700 1 -- 172.21.15.73:0/2477803988 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448d2aa8d0 con 0x56448d40f030 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.445+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.446+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 0x56448d41b210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03e00059d0 tx=0x7f03e0005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13cd735b5b6b80ea server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.446+0000 7f03dcff9700 1 -- 172.21.15.73:0/2477803988 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0027070 con 0x56448d40f030 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.446+0000 7f03dcff9700 1 -- 172.21.15.73:0/2477803988 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e000bc90 con 0x56448d40f030 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.446+0000 7f03dcff9700 1 -- 172.21.15.73:0/2477803988 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0003aa0 con 0x56448d40f030 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- 172.21.15.73:0/2477803988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 msgr2=0x56448d41b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 0x56448d41b210 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f03e00059d0 tx=0x7f03e0005be0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- 172.21.15.73:0/2477803988 shutdown_connections 2024-06-06T07:03:38.460 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40f030 0x56448d41b210 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d414470 0x56448d414850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 --2- 172.21.15.73:0/2477803988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40e670 0x56448d40eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- 172.21.15.73:0/2477803988 >> 172.21.15.73:0/2477803988 conn(0x56448d337080 msgr2=0x56448d405220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- 172.21.15.73:0/2477803988 shutdown_connections 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- 172.21.15.73:0/2477803988 wait complete. 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 -- start start 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.447+0000 7f03f76edb80 1 Processor -- start 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d41aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d4181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56448d2fbfc0 con 0x56448d414470 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56448d357810 con 0x56448d40f030 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56448d41b750 con 0x56448d40e670 2024-06-06T07:03:38.461 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03f76edb80 10 client.0 _ll_register_callbacks cb 0x56448d2f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d41aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60036/0 (socket says 172.21.15.73:60036) 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 -- 172.21.15.73:0/906371844 learned_addr learned my addr 172.21.15.73:0/906371844 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d4181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 -- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 msgr2=0x56448d41aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d41aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 -- 172.21.15.73:0/906371844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 msgr2=0x56448d4181f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d4181f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 -- 172.21.15.73:0/906371844 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d321750 con 0x56448d414470 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03e00036c0 tx=0x7f03e000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45a429e3a2e4d44b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:38.462 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.448+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0027040 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e000bc90 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f03e000b9e0 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f03e0034850 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 10 client.15924 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56448d425dd0 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e000bc90 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f03e0021070 con 0x56448d414470 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 10 client.15924 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 10 client.15924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 20 client.15924 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d4282a0 0x56448d42a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 4 client.15924 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.449+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56448d368d50 con 0x56448d4282a0 2024-06-06T07:03:38.463 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.450+0000 7f03e659c700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d4282a0 0x56448d42a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.450+0000 7f03e659c700 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d4282a0 0x56448d42a690 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91223c06789f0dcb server_cookie=49bebc406b78be63 in_seq=0 out_seq=0 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.450+0000 7f03cdffb700 10 client.15924 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03cdffb700 1 -- 172.21.15.73:0/906371844 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56448d368d50 con 0x56448d4282a0 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03cdffb700 10 client.15924 handle_command_reply: tid=0 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 client.15924 shutdown 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 msgr2=0x56448d418730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f03e00036c0 tx=0x7f03e000be40 comp rx=0 tx=0).stop 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 shutdown_connections 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d4282a0 0x56448d42a690 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d41aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d414470 0x56448d418730 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 --2- 172.21.15.73:0/906371844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d4181f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 >> 172.21.15.73:0/906371844 conn(0x56448d337080 msgr2=0x56448d404700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03cdffb700 0 client.15924 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:38.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 shutdown_connections 2024-06-06T07:03:38.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.451+0000 7f03f76edb80 1 -- 172.21.15.73:0/906371844 wait complete. 2024-06-06T07:03:38.465 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.452+0000 7f03f76edb80 20 client.15924 trim_cache size 0 max 16384 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 Processor -- start 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 -- start start 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 0x56448d4174c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40e670 0x56448d404e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ac320 con 0x56448d40f030 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81adec0 con 0x56448d40e670 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03f76edb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ac9f0 con 0x56448d337080 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03e659c700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 0x56448d4174c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60048/0 (socket says 172.21.15.73:60048) 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03e6d9d700 1 -- 172.21.15.73:0/936517158 learned_addr learned my addr 172.21.15.73:0/936517158 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.473+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40e670 0x56448d404e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 -- 172.21.15.73:0/936517158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 msgr2=0x56448d4174c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 0x56448d4174c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 -- 172.21.15.73:0/936517158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40e670 msgr2=0x56448d404e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40e670 0x56448d404e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 -- 172.21.15.73:0/936517158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56448d406c50 con 0x56448d40f030 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e659c700 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 0x56448d4174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009990 tx=0x7f03e00053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588ece2dacca84bc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03cffff700 1 -- 172.21.15.73:0/936517158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e0034440 con 0x56448d40f030 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03cffff700 1 -- 172.21.15.73:0/936517158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e00345e0 con 0x56448d40f030 2024-06-06T07:03:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.474+0000 7f03cffff700 1 -- 172.21.15.73:0/936517158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e000b7e0 con 0x56448d40f030 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- 172.21.15.73:0/936517158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 msgr2=0x56448d4053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009990 tx=0x7f03e00053a0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- 172.21.15.73:0/936517158 shutdown_connections 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d337080 0x56448d4174c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d40f030 0x56448d4053c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 --2- 172.21.15.73:0/936517158 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40e670 0x56448d404e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- 172.21.15.73:0/936517158 >> 172.21.15.73:0/936517158 conn(0x56448d41b050 msgr2=0x56448d4126f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- 172.21.15.73:0/936517158 shutdown_connections 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- 172.21.15.73:0/936517158 wait complete. 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 -- start start 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.475+0000 7f03f76edb80 1 Processor -- start 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d42aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d42afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ad7f0 con 0x56448d337080 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81ad0c0 con 0x56448d40f030 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03e81aa2c0 con 0x56448d40e670 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03f76edb80 10 client.0 _ll_register_callbacks cb 0x56448d2f15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60058/0 (socket says 172.21.15.73:60058) 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 -- 172.21.15.73:0/812038539 learned_addr learned my addr 172.21.15.73:0/812038539 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d42aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e6d9d700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d42afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 -- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 msgr2=0x56448d42aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d42aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 -- 172.21.15.73:0/812038539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 msgr2=0x56448d42afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d42afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 -- 172.21.15.73:0/812038539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56448d42b520 con 0x56448d337080 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03e659c700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e8f0 tx=0x7f03d8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1104983a456e109 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.476+0000 7f03cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03d80238e0 con 0x56448d337080 2024-06-06T07:03:38.482 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f03d8023a80 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03e00345e0 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f03d803b050 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 10 client.15930 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x56448d492960 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03e00345e0 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f03d8020040 con 0x56448d337080 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 10 client.15930 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 10 client.15930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 20 client.15930 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d414470 0x56448d4950f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 4 client.15930 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.477+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56448d3632d0 con 0x56448d414470 2024-06-06T07:03:38.483 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d414470 0x56448d4950f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03cdffb700 10 client.15930 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03e5d9b700 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d414470 0x56448d4950f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d06b7ff1419b041 server_cookie=65739c02fc42522d in_seq=0 out_seq=0 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03cdffb700 1 -- 172.21.15.73:0/812038539 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56448d3632d0 con 0x56448d414470 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03cdffb700 10 client.15930 handle_command_reply: tid=0 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 client.15930 shutdown 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 msgr2=0x56448d42d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f03d800e8f0 tx=0x7f03d8004ad0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 shutdown_connections 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56448d414470 0x56448d4950f0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56448d40f030 0x56448d42afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56448d337080 0x56448d42d700 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03cdffb700 0 client.15930 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:38.484 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 --2- 172.21.15.73:0/812038539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56448d40e670 0x56448d42aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 >> 172.21.15.73:0/812038539 conn(0x56448d41b050 msgr2=0x56448d409250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 shutdown_connections 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.478+0000 7f03f76edb80 1 -- 172.21.15.73:0/812038539 wait complete. 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03f76edb80 20 client.15930 trim_cache size 0 max 16384 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: "status": "no active scrubs running", 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stdout: "scrubs": {} 2024-06-06T07:03:38.485 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 msgr2=0x7f03d40a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 0x7f03d40a5250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f03e8199a60 tx=0x7f03d800b040 comp rx=0 tx=0).stop 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 msgr2=0x7f03e819eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f03d000e9a0 tx=0x7f03d000ecb0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 shutdown_connections 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03d40a2d90 0x7f03d40a5250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f03e81063c0 0x7f03e8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f03e8101f40 0x7f03e819e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.486 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 --2- 172.21.15.73:0/2209626868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03e8102900 0x7f03e819eca0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:38.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 >> 172.21.15.73:0/2209626868 conn(0x7f03e8076730 msgr2=0x7f03e8100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:38.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 shutdown_connections 2024-06-06T07:03:38.487 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:38.479+0000 7f03ece6a700 1 -- 172.21.15.73:0/2209626868 wait complete. 2024-06-06T07:03:39.243 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-06T07:03:39.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-06T07:03:39.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 22192 2.96 MB/sec warmup 46 sec latency 12.374 ms 2024-06-06T07:03:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:39 smithi073 ceph-mon[32303]: pgmap v72: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:03:39.531 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:39 smithi170 ceph-mon[36108]: pgmap v72: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:03:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:39 smithi136 ceph-mon[35901]: pgmap v72: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:03:40.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 22707 3.00 MB/sec warmup 47 sec latency 15.161 ms 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 -- 172.21.15.73:0/257678910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 msgr2=0x7fc8cc105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 --2- 172.21.15.73:0/257678910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc105c50 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009900 tx=0x7fc8bc02a400 comp rx=0 tx=0).stop 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 -- 172.21.15.73:0/257678910 shutdown_connections 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 --2- 172.21.15.73:0/257678910 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 0x7fc8cc10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 --2- 172.21.15.73:0/257678910 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 --2- 172.21.15.73:0/257678910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc105c50 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 -- 172.21.15.73:0/257678910 >> 172.21.15.73:0/257678910 conn(0x7fc8cc076e30 msgr2=0x7fc8cc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 -- 172.21.15.73:0/257678910 shutdown_connections 2024-06-06T07:03:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.995+0000 7fc8d117d640 1 -- 172.21.15.73:0/257678910 wait complete. 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 Processor -- start 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 -- start start 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 0x7fc8cc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc110c20 con 0x7fc8cc105870 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc110aa0 con 0x7fc8cc101010 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8d117d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc110da0 con 0x7fc8cc1019b0 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8cb59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8cb59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60074/0 (socket says 172.21.15.73:60074) 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8cb59e640 1 -- 172.21.15.73:0/2271628933 learned_addr learned my addr 172.21.15.73:0/2271628933 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.996+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 0x7fc8cc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 -- 172.21.15.73:0/2271628933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 msgr2=0x7fc8cc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 0x7fc8cc1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 -- 172.21.15.73:0/2271628933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 msgr2=0x7fc8cc198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 -- 172.21.15.73:0/2271628933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc1a4560 con 0x7fc8cc105870 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8cb59e640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400eb10 tx=0x7fc8c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8c8d99640 1 -- 172.21.15.73:0/2271628933 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4004b20 con 0x7fc8cc105870 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc1a4730 con 0x7fc8cc105870 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc1a4ca0 con 0x7fc8cc105870 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.997+0000 7fc8c8d99640 1 -- 172.21.15.73:0/2271628933 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8c4004cc0 con 0x7fc8cc105870 2024-06-06T07:03:41.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:40.998+0000 7fc8c8d99640 1 -- 172.21.15.73:0/2271628933 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4010690 con 0x7fc8cc105870 2024-06-06T07:03:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.000+0000 7fc8c8d99640 1 -- 172.21.15.73:0/2271628933 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8c4005000 con 0x7fc8cc105870 2024-06-06T07:03:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 Processor -- start 2024-06-06T07:03:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 -- start start 2024-06-06T07:03:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c08440a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c08440a760 0x55c08440cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c08435f4f0 con 0x55c08440a760 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0842fd870 con 0x55c0843ffea0 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c08435b4b0 con 0x55c0843ff580 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43614/0 (socket says 172.21.15.73:43614) 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.001+0000 7fc8c3fff640 1 -- 172.21.15.73:0/117780161 learned_addr learned my addr 172.21.15.73:0/117780161 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c08440a760 0x55c08440cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c08440a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 -- 172.21.15.73:0/117780161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 msgr2=0x55c08440a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c8d99640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 0x7fc8b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c08440a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 -- 172.21.15.73:0/117780161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c08440a760 msgr2=0x55c08440cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c08440a760 0x55c08440cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 -- 172.21.15.73:0/117780161 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0843199b0 con 0x55c0843ff580 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c8d99640 1 -- 172.21.15.73:0/2271628933 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc8c40cfd40 con 0x7fc8cc105870 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8cb59e640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 0x7fc8b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009fd0 tx=0x7fc8b4005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f3930d54a85bfe5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c17fa640 1 -- 172.21.15.73:0/117780161 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004030 con 0x55c0843ff580 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.002+0000 7fc8c17fa640 1 -- 172.21.15.73:0/117780161 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8b4005020 con 0x55c0843ff580 2024-06-06T07:03:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8c17fa640 1 -- 172.21.15.73:0/117780161 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004b00 con 0x55c0843ff580 2024-06-06T07:03:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8cb59e640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 0x7fc8b80a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40049d0 tx=0x7fc8c4009a70 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 -- 172.21.15.73:0/117780161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 msgr2=0x55c0843ff960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4009fd0 tx=0x7fc8b4005850 comp rx=0 tx=0).stop 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 -- 172.21.15.73:0/117780161 shutdown_connections 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c08440a760 0x55c08440cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c08440a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/117780161 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ff580 0x55c0843ff960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.003+0000 7fc8d3e85740 1 -- 172.21.15.73:0/117780161 >> 172.21.15.73:0/117780161 conn(0x55c0843fba90 msgr2=0x55c0843fbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.004+0000 7fc8d3e85740 1 -- 172.21.15.73:0/117780161 shutdown_connections 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.004+0000 7fc8d3e85740 1 -- 172.21.15.73:0/117780161 wait complete. 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.004+0000 7fc8d3e85740 1 -- start start 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.004+0000 7fc8d3e85740 1 Processor -- start 2024-06-06T07:03:41.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.004+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084403280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c0844037c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c08440d390 con 0x55c0843ff580 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c0842e8ae0 con 0x55c08440a760 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c08440d510 con 0x55c0843ffea0 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 10 client.0 _ll_register_callbacks cb 0x55c0843219b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60092/0 (socket says 172.21.15.73:60092) 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 -- 172.21.15.73:0/3699172174 learned_addr learned my addr 172.21.15.73:0/3699172174 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084403280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 -- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 msgr2=0x55c084403280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084403280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 -- 172.21.15.73:0/3699172174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 msgr2=0x55c0844037c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c0844037c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 -- 172.21.15.73:0/3699172174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c084073210 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4005710 tx=0x7fc8b4005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261645e3e8348a46 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b40048a0 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc8b4004a40 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.005+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8b4005020 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8b4004c40 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 10 client.15942 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c08413f620 con 0x55c0843ff580 2024-06-06T07:03:41.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8b4005020 con 0x55c0843ff580 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc8b4026070 con 0x55c0843ff580 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 10 client.15942 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 10 client.15942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 20 client.15942 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c084420f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 4 client.15942 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c084372f10 con 0x55c08441eb70 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.006+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c084420f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.007+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c084420f60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787f6a7a88533ac1 server_cookie=1069bbf5d9bf8999 in_seq=0 out_seq=0 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.007+0000 7fc8b27fc640 10 client.15942 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.007+0000 7fc8b27fc640 1 -- 172.21.15.73:0/3699172174 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c084372f10 con 0x55c08441eb70 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.007+0000 7fc8b27fc640 10 client.15942 handle_command_reply: tid=0 2024-06-06T07:03:41.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.007+0000 7fc8d3e85740 1 client.15942 shutdown 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 msgr2=0x55c084409fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fc8b4005710 tx=0x7fc8b4005970 comp rx=0 tx=0).stop 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 shutdown_connections 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c084420f60 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c0844037c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8b27fc640 0 client.15942 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084403280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/3699172174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c084409fd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 >> 172.21.15.73:0/3699172174 conn(0x55c0843fba90 msgr2=0x55c084400f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 shutdown_connections 2024-06-06T07:03:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 1 -- 172.21.15.73:0/3699172174 wait complete. 2024-06-06T07:03:41.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.008+0000 7fc8d3e85740 20 client.15942 trim_cache size 0 max 16384 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 Processor -- start 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 -- start start 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084401640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c084409c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1acc20 con 0x55c0843ff580 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1ae7c0 con 0x55c08440a760 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1ad2f0 con 0x55c0843ffea0 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60108/0 (socket says 172.21.15.73:60108) 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8c3fff640 1 -- 172.21.15.73:0/548996550 learned_addr learned my addr 172.21.15.73:0/548996550 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.025+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084401640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c084409c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 -- 172.21.15.73:0/548996550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 msgr2=0x55c084409c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c084409c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 -- 172.21.15.73:0/548996550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 msgr2=0x55c0843f6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 -- 172.21.15.73:0/548996550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c084400610 con 0x55c0843ffea0 2024-06-06T07:03:41.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084401640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005940 tx=0x7fc8bc032800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a989f6dc9b36ec55 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8c0ff9640 1 -- 172.21.15.73:0/548996550 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc035040 con 0x55c0843ffea0 2024-06-06T07:03:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8c0ff9640 1 -- 172.21.15.73:0/548996550 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8bc0328f0 con 0x55c0843ffea0 2024-06-06T07:03:41.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.026+0000 7fc8c0ff9640 1 -- 172.21.15.73:0/548996550 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8bc004a60 con 0x55c0843ffea0 2024-06-06T07:03:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- 172.21.15.73:0/548996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 msgr2=0x55c084401640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084401640 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005940 tx=0x7fc8bc032800 comp rx=0 tx=0).stop 2024-06-06T07:03:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- 172.21.15.73:0/548996550 shutdown_connections 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c08440a760 0x55c084409c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c0843ffea0 0x55c084401640 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/548996550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0843f6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- 172.21.15.73:0/548996550 >> 172.21.15.73:0/548996550 conn(0x55c0843fba90 msgr2=0x55c084400f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- 172.21.15.73:0/548996550 shutdown_connections 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- 172.21.15.73:0/548996550 wait complete. 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 -- start start 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.027+0000 7fc8d3e85740 1 Processor -- start 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c084416d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c08440a760 0x55c0844876f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1ae0f0 con 0x55c0843ff580 2024-06-06T07:03:41.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1ad9c0 con 0x55c0843ffea0 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1aac00 con 0x55c08440a760 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 10 client.0 _ll_register_callbacks cb 0x7fc8b4009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60118/0 (socket says 172.21.15.73:60118) 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 -- 172.21.15.73:0/1101830356 learned_addr learned my addr 172.21.15.73:0/1101830356 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c084416d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c08440a760 0x55c0844876f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 -- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c08440a760 msgr2=0x55c0844876f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c08440a760 0x55c0844876f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 -- 172.21.15.73:0/1101830356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 msgr2=0x55c084416d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c084416d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 -- 172.21.15.73:0/1101830356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c084487d00 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8c3fff640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400a010 tx=0x7fc8b40115f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4bf4e30e45317e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8b4004fe0 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc8b4011070 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c08436cc20 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.028+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fc8b4003ed0 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 10 client.15948 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c084489010 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c08436cc20 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fc8b4026040 con 0x55c0843ff580 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 10 client.15948 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 20 client.15948 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c08448b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 4 client.15948 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c084366f70 con 0x55c08441eb70 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c08448b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8cad9d640 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c08448b780 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c57b67283b78a49e server_cookie=cd864c39a1d6d86d in_seq=0 out_seq=0 2024-06-06T07:03:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.029+0000 7fc8b27fc640 10 client.15948 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:41.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8b27fc640 1 -- 172.21.15.73:0/1101830356 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c084366f70 con 0x55c08441eb70 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8b27fc640 10 client.15948 handle_command_reply: tid=0 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 client.15948 shutdown 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 msgr2=0x55c0844167d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400a010 tx=0x7fc8b40115f0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 shutdown_connections 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c08441eb70 0x55c08448b780 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c08440a760 0x55c0844876f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8b27fc640 0 client.15948 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c0843ffea0 0x55c084416d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 --2- 172.21.15.73:0/1101830356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c0843ff580 0x55c0844167d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 >> 172.21.15.73:0/1101830356 conn(0x55c0843fba90 msgr2=0x55c084402a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 shutdown_connections 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.051+0000 7fc8d3e85740 1 -- 172.21.15.73:0/1101830356 wait complete. 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d3e85740 20 client.15948 trim_cache size 0 max 16384 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:03:41.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 msgr2=0x7fc8b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 0x7fc8b80a52e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8c40049d0 tx=0x7fc8c4009a70 comp rx=0 tx=0).stop 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 msgr2=0x7fc8cc1a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400eb10 tx=0x7fc8c400c2d0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 shutdown_connections 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc105870 0x7fc8cc1a4020 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc8b80a2e20 0x7fc8b80a52e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8cc1019b0 0x7fc8cc1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 --2- 172.21.15.73:0/2271628933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc8cc101010 0x7fc8cc198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 >> 172.21.15.73:0/2271628933 conn(0x7fc8cc076e30 msgr2=0x7fc8cc103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 shutdown_connections 2024-06-06T07:03:41.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:41.052+0000 7fc8d117d640 1 -- 172.21.15.73:0/2271628933 wait complete. 2024-06-06T07:03:41.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 23250 2.94 MB/sec warmup 48 sec latency 22.719 ms 2024-06-06T07:03:41.487 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:41 smithi073 ceph-mon[32303]: pgmap v73: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 -- 172.21.15.73:0/182319571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 msgr2=0x7fdeac06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 --2- 172.21.15.73:0/182319571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac06a5f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fdea0009990 tx=0x7fdea0009ca0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.495 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 -- 172.21.15.73:0/182319571 shutdown_connections 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 --2- 172.21.15.73:0/182319571 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac06ab30 0x7fdeac10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 --2- 172.21.15.73:0/182319571 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac108500 0x7fdeac1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 --2- 172.21.15.73:0/182319571 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac06a5f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 -- 172.21.15.73:0/182319571 >> 172.21.15.73:0/182319571 conn(0x7fdeac078340 msgr2=0x7fdeac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 -- 172.21.15.73:0/182319571 shutdown_connections 2024-06-06T07:03:41.496 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.494+0000 7fdeb214e700 1 -- 172.21.15.73:0/182319571 wait complete. 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.495+0000 7fdeb214e700 1 Processor -- start 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.495+0000 7fdeb214e700 1 -- start start 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.495+0000 7fdeb214e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.495+0000 7fdeb214e700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 0x7fdeac1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeb214e700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac108500 0x7fdeac101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeb214e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac112610 con 0x7fdeac06a170 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeb214e700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac112490 con 0x7fdeac06ab30 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeb214e700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdeac112790 con 0x7fdeac108500 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60126/0 (socket says 172.21.15.73:60126) 2024-06-06T07:03:41.498 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 -- 172.21.15.73:0/3075862125 learned_addr learned my addr 172.21.15.73:0/3075862125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeaaffd700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 0x7fdeac1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeabfff700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac108500 0x7fdeac101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 -- 172.21.15.73:0/3075862125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac108500 msgr2=0x7fdeac101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac108500 0x7fdeac101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 -- 172.21.15.73:0/3075862125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 msgr2=0x7fdeac1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 0x7fdeac1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 -- 172.21.15.73:0/3075862125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdeac1a7090 con 0x7fdeac06a170 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeaaffd700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 0x7fdeac1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdeab7fe700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c920 tx=0x7fde9c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.496+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c004a30 con 0x7fdeac06a170 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.497+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde9c0054d0 con 0x7fdeac06a170 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.497+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdeac1a7380 con 0x7fdeac06a170 2024-06-06T07:03:41.499 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.497+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9c007c30 con 0x7fdeac06a170 2024-06-06T07:03:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.497+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdeac1a7890 con 0x7fdeac06a170 2024-06-06T07:03:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.498+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdeac103bc0 con 0x7fdeac06a170 2024-06-06T07:03:41.500 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.499+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fde9c004bd0 con 0x7fdeac06a170 2024-06-06T07:03:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.500+0000 7fdea8ff9700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 0x7fde940a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.500+0000 7fdeaaffd700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 0x7fde940a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.500+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fde9c0cf3c0 con 0x7fdeac06a170 2024-06-06T07:03:41.501 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.501+0000 7fdeaaffd700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 0x7fde940a5250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b2e0 tx=0x7fdea001a040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:41.505 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.505+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fde9c014030 con 0x7fdeac06a170 2024-06-06T07:03:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:41 smithi170 ceph-mon[36108]: pgmap v73: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:41.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.796+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdeac02dca0 con 0x7fdeac06a170 2024-06-06T07:03:41.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.797+0000 7fdea8ff9700 1 -- 172.21.15.73:0/3075862125 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) ==== 94+0+11935 (secure 0 0 0) 0x7fde9c09bbd0 con 0x7fdeac06a170 2024-06-06T07:03:41.797 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:03:41.798 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14505,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.170:6833/3776485988","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6832","nonce":3776485988},{"type":"v1","addr":"172.21.15.170:6833","nonce":3776485988}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":14523,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2227329730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2227329730},{"type":"v1","addr":"172.21.15.73:6835","nonce":2227329730}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4}],"filesystems":[{"mdsmap":{"epoch":34,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-06T06:57:09.432568+0000","modified":"2024-06-06T07:03:03.742376+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24433,"mds_1":24421,"mds_2":24419,"mds_3":24415,"mds_4":24404},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14529":{"gid":14529,"name":"d","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6837/38418017","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":38418017},{"type":"v1","addr":"172.21.15.73:6837","nonce":38418017}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"g","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6839/2070842628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2070842628},{"type":"v1","addr":"172.21.15.73:6839","nonce":2070842628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14583":{"gid":14583,"name":"a","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6841/552212635","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6840","nonce":552212635},{"type":"v1","addr":"172.21.15.73:6841","nonce":552212635}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24365":{"gid":24365,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6833/464039391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6832","nonce":464039391},{"type":"v1","addr":"172.21.15.136:6833","nonce":464039391}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24404":{"gid":24404,"name":"h","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.136:6837/450719145","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6836","nonce":450719145},{"type":"v1","addr":"172.21.15.136:6837","nonce":450719145}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":3,"incarnation":24,"state":"up:active","state_seq":27,"addr":"172.21.15.170:6835/3477948531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6834","nonce":3477948531},{"type":"v1","addr":"172.21.15.170:6835","nonce":3477948531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"b","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.136:6839/2417852679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6838","nonce":2417852679},{"type":"v1","addr":"172.21.15.136:6839","nonce":2417852679}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":20,"state":"up:active","state_seq":98,"addr":"172.21.15.170:6837/1210322990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6836","nonce":1210322990},{"type":"v1","addr":"172.21.15.170:6837","nonce":1210322990}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":8,"addr":"172.21.15.170:6839/3230829204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6838","nonce":3230829204},{"type":"v1","addr":"172.21.15.170:6839","nonce":3230829204}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25076":{"gid":25076,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.136:6835/887133051","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6834","nonce":887133051},{"type":"v1","addr":"172.21.15.136:6835","nonce":887133051}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24433,"qdb_cluster":[24419,24433,24421,24404,24415]},"id":1}]} 2024-06-06T07:03:41.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 msgr2=0x7fde940a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 0x7fde940a5250 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fdea000b2e0 tx=0x7fdea001a040 comp rx=0 tx=0).stop 2024-06-06T07:03:41.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 msgr2=0x7fdeac101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:41.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fde9c00c920 tx=0x7fde9c00cc30 comp rx=0 tx=0).stop 2024-06-06T07:03:41.800 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 shutdown_connections 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fde940a2d90 0x7fde940a5250 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdeac108500 0x7fdeac101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeac06a170 0x7fdeac101460 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 --2- 172.21.15.73:0/3075862125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdeac06ab30 0x7fdeac1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 >> 172.21.15.73:0/3075862125 conn(0x7fdeac078340 msgr2=0x7fdeac100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 shutdown_connections 2024-06-06T07:03:41.801 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:41.799+0000 7fdeb214e700 1 -- 172.21.15.73:0/3075862125 wait complete. 2024-06-06T07:03:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:41 smithi136 ceph-mon[35901]: pgmap v73: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:41.802 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2024-06-06T07:03:42.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 23793 2.93 MB/sec warmup 49 sec latency 15.431 ms 2024-06-06T07:03:42.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:42 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/3075862125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:03:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:42 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/3075862125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:03:42.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-06T07:03:42.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:42 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/3075862125' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-06T07:03:42.889 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:43.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 24558 2.98 MB/sec warmup 50 sec latency 10.734 ms 2024-06-06T07:03:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:43 smithi170 ceph-mon[36108]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:43 smithi073 ceph-mon[32303]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:43 smithi136 ceph-mon[35901]: pgmap v74: 129 pgs: 129 active+clean; 33 MiB data, 464 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 154 op/s 2024-06-06T07:03:44.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 24944 3.01 MB/sec warmup 51 sec latency 15.503 ms 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/3685147289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4101d80 msgr2=0x7f2aa410e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/3685147289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4101d80 0x7f2aa410e270 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c00b0a0 tx=0x7f2a9c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/3685147289 shutdown_connections 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/3685147289 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4107430 0x7f2aa4107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/3685147289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4101d80 0x7f2aa410e270 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/3685147289 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa4101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/3685147289 >> 172.21.15.73:0/3685147289 conn(0x7f2aa40783d0 msgr2=0x7f2aa40787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/3685147289 shutdown_connections 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.602+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/3685147289 wait complete. 2024-06-06T07:03:44.604 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 Processor -- start 2024-06-06T07:03:44.605 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 -- start start 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa41a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4101d80 0x7f2aa41a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 0x7f2aa419f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa4111570 con 0x7f2aa4107430 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41113f0 con 0x7f2aa41013c0 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.603+0000 7f2aaa2aa700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41116f0 con 0x7f2aa4101d80 2024-06-06T07:03:44.606 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa8847700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 0x7f2aa419f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa41a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa8847700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 0x7f2aa419f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60148/0 (socket says 172.21.15.73:60148) 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa8847700 1 -- 172.21.15.73:0/1907189078 learned_addr learned my addr 172.21.15.73:0/1907189078 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4101d80 0x7f2aa41a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 -- 172.21.15.73:0/1907189078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4101d80 msgr2=0x7f2aa41a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4101d80 0x7f2aa41a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 -- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 msgr2=0x7f2aa419f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 0x7f2aa419f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 -- 172.21.15.73:0/1907189078 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa419fa30 con 0x7f2aa41013c0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa41a3ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400e9f0 tx=0x7f2a9400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa1ffb700 1 -- 172.21.15.73:0/1907189078 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94004220 con 0x7f2aa41013c0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa419fd20 con 0x7f2aa41013c0 2024-06-06T07:03:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa1ffb700 1 -- 172.21.15.73:0/1907189078 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a940043c0 con 0x7f2aa41013c0 2024-06-06T07:03:44.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.604+0000 7f2aa1ffb700 1 -- 172.21.15.73:0/1907189078 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a94005020 con 0x7f2aa41013c0 2024-06-06T07:03:44.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.605+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa41abc10 con 0x7f2aa41013c0 2024-06-06T07:03:44.608 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.607+0000 7f2aa1ffb700 1 -- 172.21.15.73:0/1907189078 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a94004750 con 0x7f2aa41013c0 2024-06-06T07:03:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.608+0000 7f2aa1ffb700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 0x7f2a900a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.608+0000 7f2aa1ffb700 1 -- 172.21.15.73:0/1907189078 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2a940cf6c0 con 0x7f2aa41013c0 2024-06-06T07:03:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.608+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 0x7f2a900a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.609+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 0x7f2a900a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2aa410e780 tx=0x7f2a8c009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.609+0000 7f2ab4b2db80 1 Processor -- start 2024-06-06T07:03:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.609+0000 7f2ab4b2db80 1 -- start start 2024-06-06T07:03:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.609+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3c9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3c4340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55799f315810 con 0x55799f3c9f60 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799f2bd720 con 0x55799f3c9640 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55799f3117d0 con 0x55799f3c3f20 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60150/0 (socket says 172.21.15.73:60150) 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 -- 172.21.15.73:0/202132872 learned_addr learned my addr 172.21.15.73:0/202132872 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2aa8847700 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3c9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 -- 172.21.15.73:0/202132872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 msgr2=0x55799f3c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 -- 172.21.15.73:0/202132872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 msgr2=0x55799f3c9a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3c9a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 -- 172.21.15.73:0/202132872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799f2b3f50 con 0x55799f3c9f60 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c011d50 tx=0x7f2a8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b39f24c3dbd3b218 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9a7fc700 1 -- 172.21.15.73:0/202132872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00eb20 con 0x55799f3c9f60 2024-06-06T07:03:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9a7fc700 1 -- 172.21.15.73:0/202132872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a8c00ecc0 con 0x55799f3c9f60 2024-06-06T07:03:44.613 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.610+0000 7f2a9a7fc700 1 -- 172.21.15.73:0/202132872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c01d670 con 0x55799f3c9f60 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/202132872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 msgr2=0x55799f3c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c011d50 tx=0x7f2a8c014000 comp rx=0 tx=0).stop 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/202132872 shutdown_connections 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3c9a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/202132872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3c39e0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/202132872 >> 172.21.15.73:0/202132872 conn(0x55799f2eecc0 msgr2=0x55799f3be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/202132872 shutdown_connections 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.611+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/202132872 wait complete. 2024-06-06T07:03:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 -- start start 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 Processor -- start 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55799f3d09d0 con 0x55799f3c9f60 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799f2b5bf0 con 0x55799f3c9640 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55799f3d0b50 con 0x55799f3c3f20 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2ab4b2db80 10 client.0 _ll_register_callbacks cb 0x55799f2af920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2aa8847700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2a9bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.612+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa8847700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60152/0 (socket says 172.21.15.73:60152) 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a9bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:34220/0 (socket says 172.21.15.73:34220) 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa8847700 1 -- 172.21.15.73:0/157452465 learned_addr learned my addr 172.21.15.73:0/157452465 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43678/0 (socket says 172.21.15.73:43678) 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 -- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 msgr2=0x55799f3ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 -- 172.21.15.73:0/157452465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 msgr2=0x55799f3cd510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 -- 172.21.15.73:0/157452465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799f2648d0 con 0x55799f3c3f20 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa8847700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a940cca60 tx=0x7f2a9409b210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9538f66dc893ea68 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:03:44.618 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940b3020 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2a940cf4c0 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a8c00ecc0 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2a9401c040 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2ab4b2db80 10 client.25504 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.613+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55799f3db330 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a8c00ecc0 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2a9409ba90 con 0x55799f3c3f20 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 10 client.25504 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 10 client.25504 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 20 client.25504 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f3dfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 4 client.25504 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55799f322d10 con 0x55799f3dd8c0 2024-06-06T07:03:44.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.614+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f3dfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.615+0000 7f2a837fe700 10 client.25504 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:03:44.620 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.615+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f3dfcb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2109dbe3985f9f73 server_cookie=37cc84acc9039d8b in_seq=0 out_seq=0 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2a837fe700 1 -- 172.21.15.73:0/157452465 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55799f322d10 con 0x55799f3dd8c0 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2a837fe700 10 client.25504 handle_command_reply: tid=0 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 client.25504 shutdown 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 msgr2=0x55799f3cfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2a940cca60 tx=0x7f2a9409b210 comp rx=0 tx=0).stop 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 shutdown_connections 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f3dfcb0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2a837fe700 0 client.25504 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c3f20 0x55799f3cfcd0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.621 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c9f60 0x55799f3cd510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/157452465 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 >> 172.21.15.73:0/157452465 conn(0x55799f2eecc0 msgr2=0x55799f3be5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 shutdown_connections 2024-06-06T07:03:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/157452465 wait complete. 2024-06-06T07:03:44.622 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.616+0000 7f2ab4b2db80 20 client.25504 trim_cache size 0 max 16384 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.632+0000 7f2ab4b2db80 1 Processor -- start 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.632+0000 7f2ab4b2db80 1 -- start start 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 0x55799f3e5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b4bc0 con 0x55799f3c3f20 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b6760 con 0x55799f3c9640 2024-06-06T07:03:44.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b5290 con 0x55799f3c9f60 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60156/0 (socket says 172.21.15.73:60156) 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2604140381 learned_addr learned my addr 172.21.15.73:0/2604140381 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa8847700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 0x55799f3e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2604140381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 msgr2=0x55799f3e5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 0x55799f3e5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2604140381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 msgr2=0x55799f3caa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3caa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2604140381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799f3c0330 con 0x55799f3c3f20 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa8847700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 0x55799f3e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ccb10 tx=0x7f2a94097b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b2a349633488f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2a99ffb700 1 -- 172.21.15.73:0/2604140381 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940b3040 con 0x55799f3c3f20 2024-06-06T07:03:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2a99ffb700 1 -- 172.21.15.73:0/2604140381 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a9409b830 con 0x55799f3c3f20 2024-06-06T07:03:44.636 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.633+0000 7f2a99ffb700 1 -- 172.21.15.73:0/2604140381 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940cf520 con 0x55799f3c3f20 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2604140381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 msgr2=0x55799f3c39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f2a940ccb10 tx=0x7f2a94097b30 comp rx=0 tx=0).stop 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2604140381 shutdown_connections 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9f60 0x55799f3e5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3c39e0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2604140381 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9640 0x55799f3caa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2604140381 >> 172.21.15.73:0/2604140381 conn(0x55799f2eecc0 msgr2=0x55799f3ca380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2604140381 shutdown_connections 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2604140381 wait complete. 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.634+0000 7f2ab4b2db80 1 -- start start 2024-06-06T07:03:44.640 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 Processor -- start 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9640 0x55799f3d6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9f60 0x55799f3e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b6090 con 0x55799f3c3f20 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b5960 con 0x55799f3c9f60 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa41b2b60 con 0x55799f3c9640 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2ab4b2db80 10 client.0 _ll_register_callbacks cb 0x55799f2af920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60160/0 (socket says 172.21.15.73:60160) 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2871968031 learned_addr learned my addr 172.21.15.73:0/2871968031 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa8847700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9f60 0x55799f3e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.641 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9640 0x55799f3d6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2871968031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9640 msgr2=0x55799f3d6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9640 0x55799f3d6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9f60 msgr2=0x55799f3e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9f60 0x55799f3e45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.635+0000 7f2aa3fff700 1 -- 172.21.15.73:0/2871968031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799f3e4b10 con 0x55799f3c3f20 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2aa3fff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a94097c90 tx=0x7f2a94004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1528a24668c9533c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a940b3070 con 0x55799f3c3f20 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2a940cf4c0 con 0x55799f3c3f20 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9409b830 con 0x55799f3c3f20 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2a940a42a0 con 0x55799f3c3f20 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 10 client.15972 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:44.642 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55799f447c00 con 0x55799f3c3f20 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a9409b830 con 0x55799f3c3f20 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f2a9409a6f0 con 0x55799f3c3f20 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 10 client.15972 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 10 client.15972 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.h 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 20 client.15972 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f449fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 4 client.15972 mds_command: new command op to 24404 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.636+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 --> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55799f31d290 con 0x55799f3dd8c0 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f449fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a837fe700 10 client.15972 ms_handle_connect on v2:172.21.15.136:6836/450719145 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a9bfff700 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f449fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2d57732adb36fd3d server_cookie=f80cd4b84493b768 in_seq=0 out_seq=0 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a837fe700 1 -- 172.21.15.73:0/2871968031 <== mds.4 v2:172.21.15.136:6836/450719145 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55799f31d290 con 0x55799f3dd8c0 2024-06-06T07:03:44.643 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a837fe700 10 client.15972 handle_command_reply: tid=0 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 client.15972 shutdown 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 msgr2=0x55799f3d5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f2a94097c90 tx=0x7f2a94004e90 comp rx=0 tx=0).stop 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 shutdown_connections 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799f3c9f60 0x55799f3e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799f3c3f20 0x55799f3d5e10 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.136:6836/450719145,v1:172.21.15.136:6837/450719145] conn(0x55799f3dd8c0 0x55799f449fd0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 --2- 172.21.15.73:0/2871968031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799f3c9640 0x55799f3d6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 >> 172.21.15.73:0/2871968031 conn(0x55799f2eecc0 msgr2=0x55799f3ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2a837fe700 0 client.15972 ms_handle_reset on v2:172.21.15.136:6836/450719145 2024-06-06T07:03:44.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 shutdown_connections 2024-06-06T07:03:44.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.637+0000 7f2ab4b2db80 1 -- 172.21.15.73:0/2871968031 wait complete. 2024-06-06T07:03:44.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2ab4b2db80 20 client.15972 trim_cache size 0 max 16384 2024-06-06T07:03:44.645 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:03:44.645 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 msgr2=0x7f2a900a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 0x7f2a900a5250 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2aa410e780 tx=0x7f2a8c009260 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 msgr2=0x7f2aa41a3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa41a3ce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2a9400e9f0 tx=0x7f2a9400ed00 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 shutdown_connections 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a900a2d90 0x7f2a900a5250 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aa4107430 0x7f2aa419f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2aa41013c0 0x7f2aa41a3ce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 --2- 172.21.15.73:0/1907189078 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2aa4101d80 0x7f2aa41a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 >> 172.21.15.73:0/1907189078 conn(0x7f2aa40783d0 msgr2=0x7f2aa406aca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 shutdown_connections 2024-06-06T07:03:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:44.638+0000 7f2aaa2aa700 1 -- 172.21.15.73:0/1907189078 wait complete. 2024-06-06T07:03:45.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-06T07:03:45.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 25256 2.98 MB/sec warmup 52 sec latency 15.122 ms 2024-06-06T07:03:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:45 smithi170 ceph-mon[36108]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:03:45.653 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:45 smithi073 ceph-mon[32303]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:03:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:45 smithi136 ceph-mon[35901]: pgmap v75: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 227 op/s 2024-06-06T07:03:46.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 25592 2.93 MB/sec warmup 53 sec latency 14.717 ms 2024-06-06T07:03:47.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 26054 2.91 MB/sec warmup 54 sec latency 61.945 ms 2024-06-06T07:03:47.515 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:47 smithi073 ceph-mon[32303]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:03:47.540 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 -- 172.21.15.73:0/2192060546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806a170 msgr2=0x7f333806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 --2- 172.21.15.73:0/2192060546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806a170 0x7f333806a5f0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f3328009990 tx=0x7f3328009ca0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 -- 172.21.15.73:0/2192060546 shutdown_connections 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 --2- 172.21.15.73:0/2192060546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f333806ab30 0x7f333810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 --2- 172.21.15.73:0/2192060546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3338108500 0x7f33381088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 --2- 172.21.15.73:0/2192060546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806a170 0x7f333806a5f0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 -- 172.21.15.73:0/2192060546 >> 172.21.15.73:0/2192060546 conn(0x7f3338078340 msgr2=0x7f3338078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 -- 172.21.15.73:0/2192060546 shutdown_connections 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.539+0000 7f333fb51700 1 -- 172.21.15.73:0/2192060546 wait complete. 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 Processor -- start 2024-06-06T07:03:47.541 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 -- start start 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333806a170 0x7f3338101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3338108500 0x7f3338101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3338112630 con 0x7f333806ab30 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f33381124b0 con 0x7f333806a170 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333fb51700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f33381127b0 con 0x7f3338108500 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53522/0 (socket says 172.21.15.73:53522) 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333e0ee700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3338108500 0x7f3338101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333d0ec700 1 -- 172.21.15.73:0/4004057330 learned_addr learned my addr 172.21.15.73:0/4004057330 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:47.543 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.540+0000 7f333d8ed700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333806a170 0x7f3338101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 -- 172.21.15.73:0/4004057330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3338108500 msgr2=0x7f3338101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3338108500 0x7f3338101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 -- 172.21.15.73:0/4004057330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333806a170 msgr2=0x7f3338101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333806a170 0x7f3338101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 -- 172.21.15.73:0/4004057330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33381a7090 con 0x7f333806ab30 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333d0ec700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f3328003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f332effd700 1 -- 172.21.15.73:0/4004057330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3328027070 con 0x7f333806ab30 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33381a7320 con 0x7f333806ab30 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33381a77d0 con 0x7f333806ab30 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f332effd700 1 -- 172.21.15.73:0/4004057330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f332800bdc0 con 0x7f333806ab30 2024-06-06T07:03:47.544 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.541+0000 7f332effd700 1 -- 172.21.15.73:0/4004057330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33280227d0 con 0x7f333806ab30 2024-06-06T07:03:47.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.544+0000 7f332effd700 1 -- 172.21.15.73:0/4004057330 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f332802a070 con 0x7f333806ab30 2024-06-06T07:03:47.545 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.545+0000 7f332effd700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 0x7f33240a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.545+0000 7f333d8ed700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 0x7f33240a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.545+0000 7f332effd700 1 -- 172.21.15.73:0/4004057330 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f33280d0210 con 0x7f333806ab30 2024-06-06T07:03:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 Processor -- start 2024-06-06T07:03:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333d8ed700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 0x7f33240a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3334005950 tx=0x7f333400b410 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.547 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 -- start start 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01d3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dcb00bff60 con 0x55dcb01cd7a0 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dcb011e850 con 0x55dcb01d2f60 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dcb0086460 con 0x55dcb01d3880 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01d3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333e0ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01d3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59864/0 (socket says 172.21.15.73:59864) 2024-06-06T07:03:47.548 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333e0ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53526/0 (socket says 172.21.15.73:53526) 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333d8ed700 1 -- 172.21.15.73:0/4235485982 learned_addr learned my addr 172.21.15.73:0/4235485982 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.546+0000 7f333d0ec700 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 -- 172.21.15.73:0/4235485982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 msgr2=0x55dcb01cd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01cd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 -- 172.21.15.73:0/4235485982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 msgr2=0x55dcb01d3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01d3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 -- 172.21.15.73:0/4235485982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb006d8d0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f333e0ee700 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3330003f20 tx=0x7f3330009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff9d03233c078f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f3323fff700 1 -- 172.21.15.73:0/4235485982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330004bb0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f3323fff700 1 -- 172.21.15.73:0/4235485982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33300048a0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.547+0000 7f3323fff700 1 -- 172.21.15.73:0/4235485982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330007c20 con 0x55dcb01cd7a0 2024-06-06T07:03:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- 172.21.15.73:0/4235485982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 msgr2=0x55dcb01cdbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f3330003f20 tx=0x7f3330009ba0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- 172.21.15.73:0/4235485982 shutdown_connections 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cdbc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01d3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/4235485982 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01cd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- 172.21.15.73:0/4235485982 >> 172.21.15.73:0/4235485982 conn(0x55dcb00fa080 msgr2=0x55dcb01c8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- 172.21.15.73:0/4235485982 shutdown_connections 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- 172.21.15.73:0/4235485982 wait complete. 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 -- start start 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.548+0000 7f334a3d4b80 1 Processor -- start 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d2f60 0x55dcb01d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 0x55dcb01d6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dcb00befc0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dcb011a810 con 0x55dcb01d3880 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dcb01da260 con 0x55dcb01d2f60 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f334a3d4b80 10 client.0 _ll_register_callbacks cb 0x55dcb00b45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53534/0 (socket says 172.21.15.73:53534) 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 -- 172.21.15.73:0/1701707809 learned_addr learned my addr 172.21.15.73:0/1701707809 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333e0ee700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 0x55dcb01d6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d0ec700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d2f60 0x55dcb01d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 -- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d2f60 msgr2=0x55dcb01d6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d2f60 0x55dcb01d6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 -- 172.21.15.73:0/1701707809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 msgr2=0x55dcb01d6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 0x55dcb01d6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 -- 172.21.15.73:0/1701707809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb00e4750 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333e0ee700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 0x55dcb01d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f333d8ed700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33340116f0 tx=0x7f33340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bfe0aae36edbd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.549+0000 7f33217fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334028470 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f3334028610 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33300048a0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f3334031050 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 10 client.15984 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dcb01e49f0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33300048a0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f333401f990 con 0x55dcb01cd7a0 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 10 client.15984 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 10 client.15984 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:03:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 20 client.15984 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb01e7030 0x55dcb01e9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 4 client.15984 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.550+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcb012bd50 con 0x55dcb01e7030 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.551+0000 7f333d0ec700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb01e7030 0x55dcb01e9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.551+0000 7f333d0ec700 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb01e7030 0x55dcb01e9460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=70effe24f46e21c9 server_cookie=ac7952ea7082b564 in_seq=0 out_seq=0 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.551+0000 7f33217fa700 10 client.15984 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f33217fa700 1 -- 172.21.15.73:0/1701707809 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dcb012bd50 con 0x55dcb01e7030 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f33217fa700 10 client.15984 handle_command_reply: tid=0 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 client.15984 shutdown 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 msgr2=0x55dcb01d9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.556 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f33340116f0 tx=0x7f33340047c0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 shutdown_connections 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb01e7030 0x55dcb01e9460 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d3880 0x55dcb01d6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01cd7a0 0x55dcb01d9530 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f33217fa700 0 client.15984 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/1701707809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d2f60 0x55dcb01d6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 >> 172.21.15.73:0/1701707809 conn(0x55dcb00fa080 msgr2=0x55dcb01c7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.557 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 shutdown_connections 2024-06-06T07:03:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 1 -- 172.21.15.73:0/1701707809 wait complete. 2024-06-06T07:03:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.552+0000 7f334a3d4b80 20 client.15984 trim_cache size 0 max 16384 2024-06-06T07:03:47.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.579+0000 7f334a3d4b80 1 Processor -- start 2024-06-06T07:03:47.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 -- start start 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 0x55dcb01d5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01d6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f331c00eda0 con 0x55dcb01d2f60 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f331c010940 con 0x55dcb01cd7a0 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f331c00f470 con 0x55dcb01d3880 2024-06-06T07:03:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 0x55dcb01d5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 0x55dcb01d5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53542/0 (socket says 172.21.15.73:53542) 2024-06-06T07:03:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f333d0ec700 1 -- 172.21.15.73:0/3844078248 learned_addr learned my addr 172.21.15.73:0/3844078248 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.580+0000 7f333e0ee700 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01d6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 -- 172.21.15.73:0/3844078248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 msgr2=0x55dcb01d6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01d6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 -- 172.21.15.73:0/3844078248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 msgr2=0x55dcb01d5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 0x55dcb01d5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 -- 172.21.15.73:0/3844078248 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcb01e9ae0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cd260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f333401ff50 tx=0x7f3334010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21feb2e9a92dda2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f33237fe700 1 -- 172.21.15.73:0/3844078248 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f333403b070 con 0x55dcb01cd7a0 2024-06-06T07:03:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f33237fe700 1 -- 172.21.15.73:0/3844078248 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3334023070 con 0x55dcb01cd7a0 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.581+0000 7f33237fe700 1 -- 172.21.15.73:0/3844078248 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334005510 con 0x55dcb01cd7a0 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3844078248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 msgr2=0x55dcb01cd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cd260 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f333401ff50 tx=0x7f3334010890 comp rx=0 tx=0).stop 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3844078248 shutdown_connections 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01d3880 0x55dcb01d6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01cd7a0 0x55dcb01cd260 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3844078248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d2f60 0x55dcb01d5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3844078248 >> 172.21.15.73:0/3844078248 conn(0x55dcb00fa080 msgr2=0x55dcb01d8060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3844078248 shutdown_connections 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.582+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3844078248 wait complete. 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 -- start start 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 Processor -- start 2024-06-06T07:03:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d3880 0x55dcb01ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f331c010270 con 0x55dcb01d3880 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f331c00fb40 con 0x55dcb01d2f60 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f331c00cd40 con 0x55dcb01cd7a0 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f334a3d4b80 10 client.0 _ll_register_callbacks cb 0x55dcb00b45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f333d8ed700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55494/0 (socket says 172.21.15.73:55494) 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f333d0ec700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59898/0 (socket says 172.21.15.73:59898) 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.583+0000 7f333d8ed700 1 -- 172.21.15.73:0/3841588422 learned_addr learned my addr 172.21.15.73:0/3841588422 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333e0ee700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d3880 0x55dcb01ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 -- 172.21.15.73:0/3841588422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 msgr2=0x55dcb01dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 -- 172.21.15.73:0/3841588422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d3880 msgr2=0x55dcb01ee160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d3880 0x55dcb01ee160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 -- 172.21.15.73:0/3841588422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcb01ee7d0 con 0x55dcb01cd7a0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d0ec700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f333d8ed700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3334000c00 tx=0x7f3334004a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d248015c2185e0c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f33217fa700 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3334043020 con 0x55dcb01cd7a0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3334023070 con 0x55dcb01cd7a0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f3334005020 con 0x55dcb01cd7a0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f333404e050 con 0x55dcb01cd7a0 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f334a3d4b80 10 client.25525 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.584+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55dcb0251680 con 0x55dcb01cd7a0 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3334023070 con 0x55dcb01cd7a0 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f333401f800 con 0x55dcb01cd7a0 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 10 client.25525 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 10 client.25525 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.f 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 20 client.25525 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb0251cc0 0x55dcb02540b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 4 client.25525 mds_command: new command op to 24415 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 --> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcb01262d0 con 0x55dcb0251cc0 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f333d0ec700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb0251cc0 0x55dcb02540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:47.590 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f333d0ec700 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb0251cc0 0x55dcb02540b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=efaf14e1e04f7752 server_cookie=f15f86b3278e5277 in_seq=0 out_seq=0 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.585+0000 7f33217fa700 10 client.25525 ms_handle_connect on v2:172.21.15.170:6834/3477948531 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f33217fa700 1 -- 172.21.15.73:0/3841588422 <== mds.3 v2:172.21.15.170:6834/3477948531 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcb01262d0 con 0x55dcb0251cc0 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f33217fa700 10 client.25525 handle_command_reply: tid=0 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 client.25525 shutdown 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 msgr2=0x55dcb01df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f3334000c00 tx=0x7f3334004a70 comp rx=0 tx=0).stop 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 shutdown_connections 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:6834/3477948531,v1:172.21.15.170:6835/3477948531] conn(0x55dcb0251cc0 0x55dcb02540b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dcb01cd7a0 0x55dcb01df760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dcb01d3880 0x55dcb01ee160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f33217fa700 0 client.25525 ms_handle_reset on v2:172.21.15.170:6834/3477948531 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 --2- 172.21.15.73:0/3841588422 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dcb01d2f60 0x55dcb01dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 >> 172.21.15.73:0/3841588422 conn(0x55dcb00fa080 msgr2=0x55dcb01d4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 shutdown_connections 2024-06-06T07:03:47.591 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 1 -- 172.21.15.73:0/3841588422 wait complete. 2024-06-06T07:03:47.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.586+0000 7f334a3d4b80 20 client.25525 trim_cache size 0 max 16384 2024-06-06T07:03:47.592 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:03:47.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 msgr2=0x7f33240a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.592 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 0x7f33240a5250 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3334005950 tx=0x7f333400b410 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 msgr2=0x7f33381019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f3328003a80 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 shutdown_connections 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f33240a2d90 0x7f33240a5250 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3338108500 0x7f3338101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f333806a170 0x7f3338101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 --2- 172.21.15.73:0/4004057330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f333806ab30 0x7f33381019a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 >> 172.21.15.73:0/4004057330 conn(0x7f3338078340 msgr2=0x7f33381000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 shutdown_connections 2024-06-06T07:03:47.593 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:47.587+0000 7f333fb51700 1 -- 172.21.15.73:0/4004057330 wait complete. 2024-06-06T07:03:47.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:47 smithi170 ceph-mon[36108]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:03:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:47 smithi136 ceph-mon[35901]: pgmap v76: 129 pgs: 129 active+clean; 33 MiB data, 519 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:03:48.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 26588 2.91 MB/sec warmup 55 sec latency 96.391 ms 2024-06-06T07:03:49.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 27211 2.89 MB/sec warmup 56 sec latency 51.453 ms 2024-06-06T07:03:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:49 smithi170 ceph-mon[36108]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:03:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:49 smithi073 ceph-mon[32303]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:03:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:49 smithi136 ceph-mon[35901]: pgmap v77: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:03:50.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 27651 2.91 MB/sec warmup 57 sec latency 82.028 ms 2024-06-06T07:03:51.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-06T07:03:51.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 28193 2.92 MB/sec warmup 58 sec latency 65.504 ms 2024-06-06T07:03:51.454 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:51 smithi073 ceph-mon[32303]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:51 smithi170 ceph-mon[36108]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:51 smithi136 ceph-mon[35901]: pgmap v78: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:52.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 28651 2.94 MB/sec warmup 59 sec latency 15.717 ms 2024-06-06T07:03:53.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 -- 172.21.15.73:0/3232813302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406b5c0 msgr2=0x7fb74410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 --2- 172.21.15.73:0/3232813302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406b5c0 0x7fb74410f370 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fb73c00b0a0 tx=0x7fb73c00b3b0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 -- 172.21.15.73:0/3232813302 shutdown_connections 2024-06-06T07:03:53.356 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 --2- 172.21.15.73:0/3232813302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 0x7fb744108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 --2- 172.21.15.73:0/3232813302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406b5c0 0x7fb74410f370 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 --2- 172.21.15.73:0/3232813302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406ac00 0x7fb74406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 -- 172.21.15.73:0/3232813302 >> 172.21.15.73:0/3232813302 conn(0x7fb744077400 msgr2=0x7fb744077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 -- 172.21.15.73:0/3232813302 shutdown_connections 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.353+0000 7fb74a828700 1 -- 172.21.15.73:0/3232813302 wait complete. 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 Processor -- start 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 -- start start 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 0x7fb744101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 0x7fb7441019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 0x7fb744101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb744112630 con 0x7fb74406ac00 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441124b0 con 0x7fb74406b5c0 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb74a828700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441127b0 con 0x7fb744108530 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb743fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 0x7fb744101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb743fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 0x7fb744101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53558/0 (socket says 172.21.15.73:53558) 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb743fff700 1 -- 172.21.15.73:0/1954893545 learned_addr learned my addr 172.21.15.73:0/1954893545 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 0x7fb7441019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.354+0000 7fb748dc5700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 0x7fb744101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 -- 172.21.15.73:0/1954893545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 msgr2=0x7fb744101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 0x7fb744101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 -- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 msgr2=0x7fb744101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 0x7fb744101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 -- 172.21.15.73:0/1954893545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7441a7040 con 0x7fb74406b5c0 2024-06-06T07:03:53.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 0x7fb7441019d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00e9f0 tx=0x7fb72c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb741ffb700 1 -- 172.21.15.73:0/1954893545 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c00c9c0 con 0x7fb74406b5c0 2024-06-06T07:03:53.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb741ffb700 1 -- 172.21.15.73:0/1954893545 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb72c00cb60 con 0x7fb74406b5c0 2024-06-06T07:03:53.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7441a7330 con 0x7fb74406b5c0 2024-06-06T07:03:53.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb741ffb700 1 -- 172.21.15.73:0/1954893545 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c018680 con 0x7fb74406b5c0 2024-06-06T07:03:53.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.355+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7441a7870 con 0x7fb74406b5c0 2024-06-06T07:03:53.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.358+0000 7fb741ffb700 1 -- 172.21.15.73:0/1954893545 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb72c010430 con 0x7fb74406b5c0 2024-06-06T07:03:53.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.358+0000 7fb741ffb700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 0x7fb7300a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.361 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.358+0000 7fb743fff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 0x7fb7300a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.358+0000 7fb741ffb700 1 -- 172.21.15.73:0/1954893545 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb72c0cf920 con 0x7fb74406b5c0 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb743fff700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 0x7fb7300a5250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb734009ce0 tx=0x7fb734009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb7550abb80 1 Processor -- start 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb7550abb80 1 -- start start 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af23a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.359+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb7550abb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec9ae6f810 con 0x55ec9af1df20 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb7550abb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec9ae17720 con 0x55ec9af23f60 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb7550abb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec9ae6b7d0 con 0x55ec9af23640 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af23a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73bfff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59916/0 (socket says 172.21.15.73:59916) 2024-06-06T07:03:53.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73bfff700 1 -- 172.21.15.73:0/129734248 learned_addr learned my addr 172.21.15.73:0/129734248 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:53.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 -- 172.21.15.73:0/129734248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 msgr2=0x55ec9af1d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 -- 172.21.15.73:0/129734248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 msgr2=0x55ec9af1e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 -- 172.21.15.73:0/129734248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9ae0df50 con 0x55ec9af23640 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73bfff700 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb743fff700 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af23a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb734011ab0 tx=0x7fb734014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bfb907659a482756 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73a7fc700 1 -- 172.21.15.73:0/129734248 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73400fc40 con 0x55ec9af23640 2024-06-06T07:03:53.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73a7fc700 1 -- 172.21.15.73:0/129734248 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb73400fde0 con 0x55ec9af23640 2024-06-06T07:03:53.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.360+0000 7fb73a7fc700 1 -- 172.21.15.73:0/129734248 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73401c6e0 con 0x55ec9af23640 2024-06-06T07:03:53.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 -- 172.21.15.73:0/129734248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 msgr2=0x55ec9af23a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af23a20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb734011ab0 tx=0x7fb734014000 comp rx=0 tx=0).stop 2024-06-06T07:03:53.372 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 -- 172.21.15.73:0/129734248 shutdown_connections 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af23a20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 --2- 172.21.15.73:0/129734248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af1d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 -- 172.21.15.73:0/129734248 >> 172.21.15.73:0/129734248 conn(0x55ec9ae48cc0 msgr2=0x55ec9af18850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 -- 172.21.15.73:0/129734248 shutdown_connections 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.361+0000 7fb7550abb80 1 -- 172.21.15.73:0/129734248 wait complete. 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 -- start start 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 Processor -- start 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af1df20 0x55ec9af29d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af27570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec9af2a9d0 con 0x55ec9af23640 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec9ae0fbf0 con 0x55ec9af23f60 2024-06-06T07:03:53.373 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec9af2ab50 con 0x55ec9af1df20 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb7550abb80 10 client.0 _ll_register_callbacks cb 0x55ec9ae09920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb73bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb73bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53582/0 (socket says 172.21.15.73:53582) 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb743fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af1df20 0x55ec9af29d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb73bfff700 1 -- 172.21.15.73:0/1463071669 learned_addr learned my addr 172.21.15.73:0/1463071669 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.362+0000 7fb748dc5700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af27570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.374 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 -- 172.21.15.73:0/1463071669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af1df20 msgr2=0x55ec9af29d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af1df20 0x55ec9af29d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 -- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 msgr2=0x55ec9af27570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af27570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 -- 172.21.15.73:0/1463071669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9adbe8d0 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb72c09b9c0 tx=0x7fb72c09b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22931047bfd7f1ee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0b4060 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb72c0a4210 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb73400fde0 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb72c09ba30 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 10 client.15999 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ec9af35390 con 0x55ec9af23640 2024-06-06T07:03:53.375 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb73400fde0 con 0x55ec9af23640 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb72c0cf430 con 0x55ec9af23640 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 10 client.15999 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 20 client.15999 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9af37ab0 0x55ec9af39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 4 client.15999 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.363+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec9ae7cd10 con 0x55ec9af37ab0 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.364+0000 7fb743fff700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9af37ab0 0x55ec9af39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.376 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.364+0000 7fb7237fe700 10 client.15999 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.364+0000 7fb743fff700 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9af37ab0 0x55ec9af39ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8754ccca0350e6c0 server_cookie=141d50eb50e23cdc in_seq=0 out_seq=0 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7237fe700 1 -- 172.21.15.73:0/1463071669 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec9ae7cd10 con 0x55ec9af37ab0 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7237fe700 10 client.15999 handle_command_reply: tid=0 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 client.15999 shutdown 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 msgr2=0x55ec9af27030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fb72c09b9c0 tx=0x7fb72c09b400 comp rx=0 tx=0).stop 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 shutdown_connections 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9af37ab0 0x55ec9af39ea0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af1df20 0x55ec9af29d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af27570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1463071669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af27030 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 >> 172.21.15.73:0/1463071669 conn(0x55ec9ae48cc0 msgr2=0x55ec9af1c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.377 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7237fe700 0 client.15999 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:03:53.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 shutdown_connections 2024-06-06T07:03:53.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 1 -- 172.21.15.73:0/1463071669 wait complete. 2024-06-06T07:03:53.378 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.365+0000 7fb7550abb80 20 client.15999 trim_cache size 0 max 16384 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 Processor -- start 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 -- start start 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af249b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af12690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441b0830 con 0x55ec9af1df20 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441b23d0 con 0x55ec9af23f60 2024-06-06T07:03:53.384 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb7550abb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441b0f00 con 0x55ec9af23640 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb743fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb743fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53588/0 (socket says 172.21.15.73:53588) 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.381+0000 7fb743fff700 1 -- 172.21.15.73:0/2825096507 learned_addr learned my addr 172.21.15.73:0/2825096507 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb73bfff700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af249b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb748dc5700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af12690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 -- 172.21.15.73:0/2825096507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 msgr2=0x55ec9af249b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af249b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.385 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 -- 172.21.15.73:0/2825096507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 msgr2=0x55ec9af12690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af12690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 -- 172.21.15.73:0/2825096507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec9af19460 con 0x55ec9af1df20 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb748dc5700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb743fff700 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a090f43a96f989c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb739ffb700 1 -- 172.21.15.73:0/2825096507 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734005040 con 0x55ec9af1df20 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb739ffb700 1 -- 172.21.15.73:0/2825096507 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb734005690 con 0x55ec9af1df20 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.382+0000 7fb739ffb700 1 -- 172.21.15.73:0/2825096507 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb73401e420 con 0x55ec9af1df20 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- 172.21.15.73:0/2825096507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 msgr2=0x55ec9af1d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fb7340098a0 tx=0x7fb734009e20 comp rx=0 tx=0).stop 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- 172.21.15.73:0/2825096507 shutdown_connections 2024-06-06T07:03:53.386 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af23f60 0x55ec9af12690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af1df20 0x55ec9af1d9e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 --2- 172.21.15.73:0/2825096507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23640 0x55ec9af249b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- 172.21.15.73:0/2825096507 >> 172.21.15.73:0/2825096507 conn(0x55ec9ae48cc0 msgr2=0x55ec9af25920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- 172.21.15.73:0/2825096507 shutdown_connections 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- 172.21.15.73:0/2825096507 wait complete. 2024-06-06T07:03:53.387 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 -- start start 2024-06-06T07:03:53.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.383+0000 7fb7550abb80 1 Processor -- start 2024-06-06T07:03:53.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.390 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af2b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23f60 0x55ec9af2c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441b1d00 con 0x55ec9af23640 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441b15d0 con 0x55ec9af1df20 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb7441ae7d0 con 0x55ec9af23f60 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7550abb80 10 client.0 _ll_register_callbacks cb 0x55ec9ae09920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb73bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af2b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59944/0 (socket says 172.21.15.73:59944) 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 -- 172.21.15.73:0/1868667291 learned_addr learned my addr 172.21.15.73:0/1868667291 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb748dc5700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23f60 0x55ec9af2c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 -- 172.21.15.73:0/1868667291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23f60 msgr2=0x55ec9af2c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.391 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23f60 0x55ec9af2c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 -- 172.21.15.73:0/1868667291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 msgr2=0x55ec9af2b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af2b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 -- 172.21.15.73:0/1868667291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec9afa1530 con 0x55ec9af1df20 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb743fff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb734005150 tx=0x7fb734005180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57058514faeb7e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7237fe700 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.384+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb734004710 con 0x55ec9af1df20 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb734005690 con 0x55ec9af1df20 2024-06-06T07:03:53.392 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb734009040 con 0x55ec9af1df20 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb7340048b0 con 0x55ec9af1df20 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 10 client.25439 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ec9afa25e0 con 0x55ec9af1df20 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb734005690 con 0x55ec9af1df20 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb73401e420 con 0x55ec9af1df20 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 10 client.25439 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 10 client.25439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.b 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 20 client.25439 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9afa2940 0x55ec9afa4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:53.393 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 4 client.25439 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 --> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec9ae77290 con 0x55ec9afa2940 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.385+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9afa2940 0x55ec9afa4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb73bfff700 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9afa2940 0x55ec9afa4d30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4ba36dfbfe871307 server_cookie=a39f2edfc102a021 in_seq=0 out_seq=0 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7237fe700 10 client.25439 ms_handle_connect on v2:172.21.15.136:6838/2417852679 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7237fe700 1 -- 172.21.15.73:0/1868667291 <== mds.2 v2:172.21.15.136:6838/2417852679 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec9ae77290 con 0x55ec9afa2940 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7237fe700 10 client.25439 handle_command_reply: tid=0 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 client.25439 shutdown 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 msgr2=0x55ec9af30670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb734005150 tx=0x7fb734005180 comp rx=0 tx=0).stop 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 shutdown_connections 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:6838/2417852679,v1:172.21.15.136:6839/2417852679] conn(0x55ec9afa2940 0x55ec9afa4d30 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.394 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec9af23f60 0x55ec9af2c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec9af1df20 0x55ec9af30670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 --2- 172.21.15.73:0/1868667291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec9af23640 0x55ec9af2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7237fe700 0 client.25439 ms_handle_reset on v2:172.21.15.136:6838/2417852679 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 >> 172.21.15.73:0/1868667291 conn(0x55ec9ae48cc0 msgr2=0x55ec9af12510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 shutdown_connections 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 1 -- 172.21.15.73:0/1868667291 wait complete. 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.386+0000 7fb7550abb80 20 client.25439 trim_cache size 0 max 16384 2024-06-06T07:03:53.395 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:03:53.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 msgr2=0x7fb7300a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 0x7fb7300a5250 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb734009ce0 tx=0x7fb734009260 comp rx=0 tx=0).stop 2024-06-06T07:03:53.396 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 msgr2=0x7fb7441019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 0x7fb7441019d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb72c00e9f0 tx=0x7fb72c00ed00 comp rx=0 tx=0).stop 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 shutdown_connections 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb7300a2d90 0x7fb7300a5250 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb744108530 0x7fb744101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb74406ac00 0x7fb744101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 --2- 172.21.15.73:0/1954893545 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb74406b5c0 0x7fb7441019d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 >> 172.21.15.73:0/1954893545 conn(0x7fb744077400 msgr2=0x7fb744100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 shutdown_connections 2024-06-06T07:03:53.397 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:53.387+0000 7fb74a828700 1 -- 172.21.15.73:0/1954893545 wait complete. 2024-06-06T07:03:53.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 28895 2.90 MB/sec warmup 60 sec latency 38.758 ms 2024-06-06T07:03:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:53 smithi073 ceph-mon[32303]: pgmap v79: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:53 smithi170 ceph-mon[36108]: pgmap v79: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:53 smithi136 ceph-mon[35901]: pgmap v79: 129 pgs: 129 active+clean; 37 MiB data, 533 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 -- 172.21.15.73:0/1077946847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 msgr2=0x7f4edc10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/1077946847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc10d750 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009cd0 tx=0x7f4ed402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 -- 172.21.15.73:0/1077946847 shutdown_connections 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/1077946847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc10d750 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/1077946847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc1008d0 0x7f4edc100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/1077946847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc106890 0x7f4edc106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 -- 172.21.15.73:0/1077946847 >> 172.21.15.73:0/1077946847 conn(0x7f4edc077a20 msgr2=0x7f4edc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 -- 172.21.15.73:0/1077946847 shutdown_connections 2024-06-06T07:03:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.150+0000 7f4ee1c93640 1 -- 172.21.15.73:0/1077946847 wait complete. 2024-06-06T07:03:54.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.151+0000 7f4ee1c93640 1 Processor -- start 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.151+0000 7f4ee1c93640 1 -- start start 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc1008d0 0x7f4edc196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 0x7f4edc19b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc110b50 con 0x7f4edc101290 2024-06-06T07:03:54.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1109d0 con 0x7f4edc106890 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4ee1c93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc110cd0 con 0x7f4edc1008d0 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53608/0 (socket says 172.21.15.73:53608) 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc1008d0 0x7f4edc196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edaffd640 1 -- 172.21.15.73:0/4039309082 learned_addr learned my addr 172.21.15.73:0/4039309082 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 0x7f4edc19b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 -- 172.21.15.73:0/4039309082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc1008d0 msgr2=0x7f4edc196f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc1008d0 0x7f4edc196f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 -- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 msgr2=0x7f4edc1974b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edbfff640 1 -- 172.21.15.73:0/4039309082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4edc19bf60 con 0x7f4edc106890 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.152+0000 7f4edaffd640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4edbfff640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 0x7f4edc19b830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402ae00 tx=0x7f4ed4031dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4ed8ff9640 1 -- 172.21.15.73:0/4039309082 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004040 con 0x7f4edc106890 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4ed8ff9640 1 -- 172.21.15.73:0/4039309082 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed4004690 con 0x7f4edc106890 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4ed8ff9640 1 -- 172.21.15.73:0/4039309082 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4030d30 con 0x7f4edc106890 2024-06-06T07:03:54.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4edc19c1f0 con 0x7f4edc106890 2024-06-06T07:03:54.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.153+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4edc069050 con 0x7f4edc106890 2024-06-06T07:03:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 Processor -- start 2024-06-06T07:03:54.168 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ed8ff9640 1 -- 172.21.15.73:0/4039309082 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4ed4046480 con 0x7f4edc106890 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 -- start start 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224fa350 0x5631224fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5631224534f0 con 0x5631224ef170 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631223f1870 con 0x5631224fa350 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4ee499e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56312244f4b0 con 0x5631224efa90 2024-06-06T07:03:54.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55590/0 (socket says 172.21.15.73:55590) 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 -- 172.21.15.73:0/3413337886 learned_addr learned my addr 172.21.15.73:0/3413337886 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edaffd640 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224fa350 0x5631224fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 -- 172.21.15.73:0/3413337886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224fa350 msgr2=0x5631224fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224fa350 0x5631224fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 -- 172.21.15.73:0/3413337886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 msgr2=0x5631224ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.156+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224ef550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4edb7fe640 1 -- 172.21.15.73:0/3413337886 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312240d9b0 con 0x5631224efa90 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4ed8ff9640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 0x7f4ec40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4edaffd640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 0x7f4ec40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009ca0 tx=0x7f4ecc011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbae9e30bb6fecef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4ed8ff9640 1 -- 172.21.15.73:0/4039309082 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4ed402e030 con 0x7f4edc106890 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4ec1ffb640 1 -- 172.21.15.73:0/3413337886 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc01e070 con 0x5631224efa90 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4ec1ffb640 1 -- 172.21.15.73:0/3413337886 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc009e50 con 0x5631224efa90 2024-06-06T07:03:54.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.157+0000 7f4ec1ffb640 1 -- 172.21.15.73:0/3413337886 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc018710 con 0x5631224efa90 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- 172.21.15.73:0/3413337886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 msgr2=0x5631224f9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4edaffd640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 0x7f4ec40a52e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009710 tx=0x7f4ed0004730 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009ca0 tx=0x7f4ecc011810 comp rx=0 tx=0).stop 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- 172.21.15.73:0/3413337886 shutdown_connections 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224fa350 0x5631224fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224efa90 0x5631224f9e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 --2- 172.21.15.73:0/3413337886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- 172.21.15.73:0/3413337886 >> 172.21.15.73:0/3413337886 conn(0x5631224eb680 msgr2=0x5631224eba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- 172.21.15.73:0/3413337886 shutdown_connections 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- 172.21.15.73:0/3413337886 wait complete. 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.158+0000 7f4ee499e740 1 -- start start 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 Processor -- start 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x5631224f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5631224fcf80 con 0x5631224ef170 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5631223dcae0 con 0x5631224efa90 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5631224fd100 con 0x5631224fa350 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4ee499e740 10 client.0 _ll_register_callbacks cb 0x5631224159b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53616/0 (socket says 172.21.15.73:53616) 2024-06-06T07:03:54.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 -- 172.21.15.73:0/2186297682 learned_addr learned my addr 172.21.15.73:0/2186297682 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edbfff640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x5631224f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edaffd640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 -- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 msgr2=0x5631224f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x5631224f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.159+0000 7f4edb7fe640 1 -- 172.21.15.73:0/2186297682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 msgr2=0x5631224f7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224f7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4edb7fe640 1 -- 172.21.15.73:0/2186297682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563122167210 con 0x5631224ef170 2024-06-06T07:03:54.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc01afd0 tx=0x7f4ecc005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceff43b47572547e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ecc026030 con 0x5631224ef170 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4ecc016040 con 0x5631224ef170 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ecc009e50 con 0x5631224ef170 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ecc030050 con 0x5631224ef170 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 10 client.16017 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x563122233620 con 0x5631224ef170 2024-06-06T07:03:54.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ecc009e50 con 0x5631224ef170 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4ecc021050 con 0x5631224ef170 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 10 client.16017 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 10 client.16017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 20 client.16017 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.160+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56312250a470 0x56312250c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.161+0000 7f4ee499e740 4 client.16017 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.161+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563122466f10 con 0x56312250a470 2024-06-06T07:03:54.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.161+0000 7f4edaffd640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56312250a470 0x56312250c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.161+0000 7f4edaffd640 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56312250a470 0x56312250c860 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd1315bbebcc82 server_cookie=fe09c3acedf7edb9 in_seq=0 out_seq=0 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.161+0000 7f4ebaffd640 10 client.16017 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ebaffd640 1 -- 172.21.15.73:0/2186297682 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563122466f10 con 0x56312250a470 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ebaffd640 10 client.16017 handle_command_reply: tid=0 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 client.16017 shutdown 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 msgr2=0x5631224f7350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc01afd0 tx=0x7f4ecc005720 comp rx=0 tx=0).stop 2024-06-06T07:03:54.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 shutdown_connections 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56312250a470 0x56312250c860 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x5631224f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224f7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ebaffd640 0 client.16017 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 --2- 172.21.15.73:0/2186297682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224ef170 0x5631224f7350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 >> 172.21.15.73:0/2186297682 conn(0x5631224eb680 msgr2=0x5631224f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 shutdown_connections 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 1 -- 172.21.15.73:0/2186297682 wait complete. 2024-06-06T07:03:54.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.162+0000 7f4ee499e740 20 client.16017 trim_cache size 0 max 16384 2024-06-06T07:03:54.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-06T07:03:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 Processor -- start 2024-06-06T07:03:54.190 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 -- start start 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224ef170 0x5631224f5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x563122511fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1aca90 con 0x5631224efa90 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1ae630 con 0x5631224ef170 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4ee499e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1ad160 con 0x5631224fa350 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x563122511fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53628/0 (socket says 172.21.15.73:53628) 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edaffd640 1 -- 172.21.15.73:0/188515706 learned_addr learned my addr 172.21.15.73:0/188515706 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224ef170 0x5631224f5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 -- 172.21.15.73:0/188515706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224ef170 msgr2=0x5631224f5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224ef170 0x5631224f5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 -- 172.21.15.73:0/188515706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 msgr2=0x5631224f6160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.179+0000 7f4edbfff640 1 -- 172.21.15.73:0/188515706 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631225061e0 con 0x5631224fa350 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.180+0000 7f4edaffd640 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.180+0000 7f4edbfff640 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x563122511fa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40ee0c0 tx=0x7f4ed40dd0b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc4d482efa4abf85 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.180+0000 7f4ec17fa640 1 -- 172.21.15.73:0/188515706 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004840 con 0x5631224fa350 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.180+0000 7f4ec17fa640 1 -- 172.21.15.73:0/188515706 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed4036040 con 0x5631224fa350 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.180+0000 7f4ec17fa640 1 -- 172.21.15.73:0/188515706 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40ee230 con 0x5631224fa350 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- 172.21.15.73:0/188515706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 msgr2=0x563122511fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x563122511fa0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40ee0c0 tx=0x7f4ed40dd0b0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- 172.21.15.73:0/188515706 shutdown_connections 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224fa350 0x563122511fa0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224efa90 0x5631224f6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- 172.21.15.73:0/188515706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224ef170 0x5631224f5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- 172.21.15.73:0/188515706 >> 172.21.15.73:0/188515706 conn(0x5631224eb680 msgr2=0x5631224eba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- 172.21.15.73:0/188515706 shutdown_connections 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- 172.21.15.73:0/188515706 wait complete. 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- start start 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 Processor -- start 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 0x5631224fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224fd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1adf60 con 0x5631224fa350 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1ad830 con 0x5631224efa90 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.181+0000 7f4ee499e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4edc1aaa70 con 0x5631224ef170 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ee499e740 10 client.0 _ll_register_callbacks cb 0x7f4ecc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 0x5631224fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edaffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224fd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53632/0 (socket says 172.21.15.73:53632) 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 -- 172.21.15.73:0/509645604 learned_addr learned my addr 172.21.15.73:0/509645604 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 0x5631224fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55614/0 (socket says 172.21.15.73:55614) 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 -- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 msgr2=0x5631224fd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 0x5631224fd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 -- 172.21.15.73:0/509645604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 msgr2=0x5631224fd930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224fd930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 -- 172.21.15.73:0/509645604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631224fe200 con 0x5631224fa350 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4edbfff640 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c55d0 tx=0x7f4ed40048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25688a694f41bdf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ebaffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409f020 con 0x5631224fa350 2024-06-06T07:03:54.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f4ed40b9590 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ed4036040 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4ed40ca050 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ee499e740 10 client.16023 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.182+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5631225745b0 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ed4036040 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f4ed404e070 con 0x5631224fa350 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 10 client.16023 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 10 client.16023 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 20 client.16023 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563122574c40 0x563122577060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 4 client.16023 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563122460c20 con 0x563122574c40 2024-06-06T07:03:54.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563122574c40 0x563122577060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4ebaffd640 10 client.16023 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:54.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.183+0000 7f4edb7fe640 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563122574c40 0x563122577060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70ef0e9379916d39 server_cookie=a25c5608aeb9ebb0 in_seq=0 out_seq=0 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.204+0000 7f4ebaffd640 1 -- 172.21.15.73:0/509645604 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563122460c20 con 0x563122574c40 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.204+0000 7f4ebaffd640 10 client.16023 handle_command_reply: tid=0 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.204+0000 7f4ee499e740 1 client.16023 shutdown 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 msgr2=0x563122572f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40c55d0 tx=0x7f4ed40048f0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 shutdown_connections 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563122574c40 0x563122577060 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ebaffd640 0 client.16023 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5631224fa350 0x563122572f60 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5631224efa90 0x5631224fd930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 --2- 172.21.15.73:0/509645604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5631224ef170 0x5631224fd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 >> 172.21.15.73:0/509645604 conn(0x5631224eb680 msgr2=0x5631224f0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 shutdown_connections 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 1 -- 172.21.15.73:0/509645604 wait complete. 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee499e740 20 client.16023 trim_cache size 0 max 16384 2024-06-06T07:03:54.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:03:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:03:54.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 msgr2=0x7f4ec40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 0x7f4ec40a52e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009710 tx=0x7f4ed0004730 comp rx=0 tx=0).stop 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 msgr2=0x7f4edc19b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.205+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 0x7f4edc19b830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ed402ae00 tx=0x7f4ed4031dd0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 shutdown_connections 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4edc106890 0x7f4edc19b830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4edc101290 0x7f4edc1974b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4ec40a2e20 0x7f4ec40a52e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 --2- 172.21.15.73:0/4039309082 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4edc1008d0 0x7f4edc196f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 >> 172.21.15.73:0/4039309082 conn(0x7f4edc077a20 msgr2=0x7f4edc1033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 shutdown_connections 2024-06-06T07:03:54.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:03:54.206+0000 7f4ee1c93640 1 -- 172.21.15.73:0/4039309082 wait complete. 2024-06-06T07:03:54.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 29337 2.87 MB/sec warmup 61 sec latency 16.668 ms 2024-06-06T07:03:54.447 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:55.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 29869 2.89 MB/sec warmup 62 sec latency 17.179 ms 2024-06-06T07:03:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:55 smithi170 ceph-mon[36108]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-06-06T07:03:55.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:55 smithi073 ceph-mon[32303]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-06-06T07:03:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:55 smithi136 ceph-mon[35901]: pgmap v80: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.6 MiB/s wr, 222 op/s 2024-06-06T07:03:56.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 -- 172.21.15.73:0/3259450092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 msgr2=0x7fdc641077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3259450092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc641077e0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fdc58009990 tx=0x7fdc58009ca0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 -- 172.21.15.73:0/3259450092 shutdown_connections 2024-06-06T07:03:56.228 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3259450092 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3259450092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc641077e0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3259450092 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc64101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 -- 172.21.15.73:0/3259450092 >> 172.21.15.73:0/3259450092 conn(0x7fdc64078340 msgr2=0x7fdc64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 -- 172.21.15.73:0/3259450092 shutdown_connections 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.226+0000 7fdc6b071700 1 -- 172.21.15.73:0/3259450092 wait complete. 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 Processor -- start 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 -- start start 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6419ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc64111520 con 0x7fdc64107400 2024-06-06T07:03:56.229 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641113a0 con 0x7fdc64101390 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6b071700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641116a0 con 0x7fdc64101d50 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.227+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc63fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6419ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54996/0 (socket says 172.21.15.73:54996) 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc63fff700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6419ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34506/0 (socket says 172.21.15.73:34506) 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40544/0 (socket says 172.21.15.73:40544) 2024-06-06T07:03:56.230 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3506050847 learned_addr learned my addr 172.21.15.73:0/3506050847 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3506050847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 msgr2=0x7fdc6419ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6419ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 msgr2=0x7fdc64198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3506050847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc64199060 con 0x7fdc64101390 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:56.231 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc580057d0 tx=0x7fdc58005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc61ffb700 1 -- 172.21.15.73:0/3506050847 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58026070 con 0x7fdc64101390 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc641992f0 con 0x7fdc64101390 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc61ffb700 1 -- 172.21.15.73:0/3506050847 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc58021410 con 0x7fdc64101390 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.228+0000 7fdc61ffb700 1 -- 172.21.15.73:0/3506050847 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58017650 con 0x7fdc64101390 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.229+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc641a5610 con 0x7fdc64101390 2024-06-06T07:03:56.232 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.231+0000 7fdc61ffb700 1 -- 172.21.15.73:0/3506050847 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc58029070 con 0x7fdc64101390 2024-06-06T07:03:56.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.231+0000 7fdc61ffb700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 0x7fdc540a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.232+0000 7fdc61ffb700 1 -- 172.21.15.73:0/3506050847 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdc580d0480 con 0x7fdc64101390 2024-06-06T07:03:56.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.232+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 0x7fdc540a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.233 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.232+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 0x7fdc540a5250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc640788b0 tx=0x7fdc50009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 Processor -- start 2024-06-06T07:03:56.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 -- start start 2024-06-06T07:03:56.234 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb249c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffb23ee810 con 0x55ffb249cf20 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffb2396720 con 0x55ffb24a2f60 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ffb23ea7d0 con 0x55ffb24a2640 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34516/0 (socket says 172.21.15.73:34516) 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc63fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb249c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 -- 172.21.15.73:0/2034473168 learned_addr learned my addr 172.21.15.73:0/2034473168 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 -- 172.21.15.73:0/2034473168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 msgr2=0x55ffb249c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb249c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 -- 172.21.15.73:0/2034473168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 msgr2=0x55ffb249d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.235 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.233+0000 7fdc68e0d700 1 -- 172.21.15.73:0/2034473168 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb238cf50 con 0x55ffb24a2640 2024-06-06T07:03:56.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.234+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc580a4d60 tx=0x7fdc5809c010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668e6b560619ba4d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.234+0000 7fdc4effd700 1 -- 172.21.15.73:0/2034473168 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5809c6b0 con 0x55ffb24a2640 2024-06-06T07:03:56.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.234+0000 7fdc4effd700 1 -- 172.21.15.73:0/2034473168 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc580a0a60 con 0x55ffb24a2640 2024-06-06T07:03:56.236 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.234+0000 7fdc4effd700 1 -- 172.21.15.73:0/2034473168 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc58021930 con 0x55ffb24a2640 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- 172.21.15.73:0/2034473168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 msgr2=0x55ffb24a2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdc580a4d60 tx=0x7fdc5809c010 comp rx=0 tx=0).stop 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- 172.21.15.73:0/2034473168 shutdown_connections 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a2a20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/2034473168 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb249c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- 172.21.15.73:0/2034473168 >> 172.21.15.73:0/2034473168 conn(0x55ffb23c7cc0 msgr2=0x55ffb2497850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- 172.21.15.73:0/2034473168 shutdown_connections 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- 172.21.15.73:0/2034473168 wait complete. 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 -- start start 2024-06-06T07:03:56.237 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.235+0000 7fdc758f4b80 1 Processor -- start 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.236+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 0x55ffb24a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.236+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.236+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.236+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffb24a96f0 con 0x55ffb24a2f60 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40560/0 (socket says 172.21.15.73:40560) 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 -- 172.21.15.73:0/3382659064 learned_addr learned my addr 172.21.15.73:0/3382659064 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ffb238ebf0 con 0x55ffb249cf20 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ffb24a9870 con 0x55ffb24a2640 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc758f4b80 10 client.0 _ll_register_callbacks cb 0x55ffb2388920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 0x55ffb24a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 -- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 msgr2=0x55ffb24a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 -- 172.21.15.73:0/3382659064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 msgr2=0x55ffb24a8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 0x55ffb24a8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc6960e700 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb233d8d0 con 0x55ffb24a2f60 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.237+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 0x55ffb24a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c003f90 tx=0x7fdc5c004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae31e3ed62112b06 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c019d00 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdc5c0075e0 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc580a0a60 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c02a050 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc758f4b80 10 client.16035 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffb24b4080 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc580a0a60 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.238+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdc5c00e9e0 con 0x55ffb24a2f60 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 10 client.16035 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 20 client.16035 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb24b66c0 0x55ffb24b8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 4 client.16035 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:56.243 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffb23fbd10 con 0x55ffb24b66c0 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb24b66c0 0x55ffb24b8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb24b66c0 0x55ffb24b8ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9899ee9d619bfa1a server_cookie=4447744e4d2cd52e in_seq=0 out_seq=0 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.239+0000 7fdc43fff700 10 client.16035 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc43fff700 1 -- 172.21.15.73:0/3382659064 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ffb23fbd10 con 0x55ffb24b66c0 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc43fff700 10 client.16035 handle_command_reply: tid=0 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 client.16035 shutdown 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 msgr2=0x55ffb24a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c003f90 tx=0x7fdc5c004680 comp rx=0 tx=0).stop 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 shutdown_connections 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb24b66c0 0x55ffb24b8ad0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24a65d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb249cf20 0x55ffb24a8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3382659064 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 >> 172.21.15.73:0/3382659064 conn(0x55ffb23c7cc0 msgr2=0x55ffb24975f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc43fff700 0 client.16035 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:03:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 shutdown_connections 2024-06-06T07:03:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3382659064 wait complete. 2024-06-06T07:03:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.240+0000 7fdc758f4b80 20 client.16035 trim_cache size 0 max 16384 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 Processor -- start 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 -- start start 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb24a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac290 con 0x55ffb249cf20 2024-06-06T07:03:56.259 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ade30 con 0x55ffb24a2f60 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.257+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ac960 con 0x55ffb24a2640 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40576/0 (socket says 172.21.15.73:40576) 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3046117250 learned_addr learned my addr 172.21.15.73:0/3046117250 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb24a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3046117250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 msgr2=0x55ffb24b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3046117250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 msgr2=0x55ffb24a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb24a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 -- 172.21.15.73:0/3046117250 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffb24a2c70 con 0x55ffb249cf20 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:56.260 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc58098b10 tx=0x7fdc58009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22a760062a857a94 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc4e7fc700 1 -- 172.21.15.73:0/3046117250 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580b4070 con 0x55ffb249cf20 2024-06-06T07:03:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc4e7fc700 1 -- 172.21.15.73:0/3046117250 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5809c6b0 con 0x55ffb249cf20 2024-06-06T07:03:56.261 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.258+0000 7fdc4e7fc700 1 -- 172.21.15.73:0/3046117250 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc580094b0 con 0x55ffb249cf20 2024-06-06T07:03:56.264 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3046117250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 msgr2=0x55ffb249c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fdc58098b10 tx=0x7fdc58009310 comp rx=0 tx=0).stop 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3046117250 shutdown_connections 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2f60 0x55ffb24a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb249cf20 0x55ffb249c9e0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3046117250 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb24a2640 0x55ffb24b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3046117250 >> 172.21.15.73:0/3046117250 conn(0x55ffb23c7cc0 msgr2=0x55ffb24a49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3046117250 shutdown_connections 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.259+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3046117250 wait complete. 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 -- start start 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 Processor -- start 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 0x55ffb24aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2640 0x55ffb24af000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.265 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ad760 con 0x55ffb24a2f60 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641ad030 con 0x55ffb24a2640 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc641aa250 con 0x55ffb249cf20 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc758f4b80 10 client.0 _ll_register_callbacks cb 0x55ffb2388920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 0x55ffb24aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc6960e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40590/0 (socket says 172.21.15.73:40590) 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc68e0d700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 0x55ffb24aeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34534/0 (socket says 172.21.15.73:34534) 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc6960e700 1 -- 172.21.15.73:0/3953219601 learned_addr learned my addr 172.21.15.73:0/3953219601 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.260+0000 7fdc63fff700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2640 0x55ffb24af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 -- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 msgr2=0x55ffb24aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.266 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 0x55ffb24aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 -- 172.21.15.73:0/3953219601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2640 msgr2=0x55ffb24af000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2640 0x55ffb24af000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 -- 172.21.15.73:0/3953219601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffb24bd9a0 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc6960e700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00ee00 tx=0x7fdc5c01ae40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae2f99b2b577927 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc43fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c032020 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdc5c00e900 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc5809c6b0 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdc5c03a050 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc758f4b80 10 client.16041 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.261+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55ffb2520ae0 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc5809c6b0 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdc5c022450 con 0x55ffb24a2f60 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 10 client.16041 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:56.267 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 10 client.16041 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.c 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 20 client.16041 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb2521120 0x55ffb2523510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 4 client.16041 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 --> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffb23f6290 con 0x55ffb2521120 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb2521120 0x55ffb2523510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc43fff700 10 client.16041 ms_handle_connect on v2:172.21.15.170:6836/1210322990 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.262+0000 7fdc68e0d700 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb2521120 0x55ffb2523510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6649738a275cb50e server_cookie=281ebba7d2e9a15e in_seq=0 out_seq=0 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc43fff700 1 -- 172.21.15.73:0/3953219601 <== mds.1 v2:172.21.15.170:6836/1210322990 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2062 (crc 0 0 0) 0x55ffb23f6290 con 0x55ffb2521120 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc43fff700 10 client.16041 handle_command_reply: tid=0 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 client.16041 shutdown 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 msgr2=0x55ffb24bd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00ee00 tx=0x7fdc5c01ae40 comp rx=0 tx=0).stop 2024-06-06T07:03:56.268 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 shutdown_connections 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:6836/1210322990,v1:172.21.15.170:6837/1210322990] conn(0x55ffb2521120 0x55ffb2523510 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ffb249cf20 0x55ffb24aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffb24a2f60 0x55ffb24bd460 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc43fff700 0 client.16041 ms_handle_reset on v2:172.21.15.170:6836/1210322990 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 --2- 172.21.15.73:0/3953219601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ffb24a2640 0x55ffb24af000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 >> 172.21.15.73:0/3953219601 conn(0x55ffb23c7cc0 msgr2=0x55ffb24a00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 shutdown_connections 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 1 -- 172.21.15.73:0/3953219601 wait complete. 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.263+0000 7fdc758f4b80 20 client.16041 trim_cache size 0 max 16384 2024-06-06T07:03:56.269 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 195046859, 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628298, 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC" 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 868541695, 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628324, 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP" 2024-06-06T07:03:56.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 969409306, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628328, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC" 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 1455232718, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628297, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC" 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.271 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 2213674120, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628299, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC" 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 2298354129, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628323, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS" 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.272 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 2318418918, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628327, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC" 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 3325743081, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628322, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP" 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "damage_type": "backtrace", 2024-06-06T07:03:56.273 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 3440170658, 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1099511628326, 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stdout: "path": "/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS" 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stdout:] 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 msgr2=0x7fdc540a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.274 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 0x7fdc540a5250 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdc640788b0 tx=0x7fdc50009260 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 msgr2=0x7fdc6419e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdc580057d0 tx=0x7fdc58005800 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 shutdown_connections 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc540a2d90 0x7fdc540a5250 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc64107400 0x7fdc64198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc64101390 0x7fdc6419e740 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 --2- 172.21.15.73:0/3506050847 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc64101d50 0x7fdc6419ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 >> 172.21.15.73:0/3506050847 conn(0x7fdc64078340 msgr2=0x7fdc641006b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 shutdown_connections 2024-06-06T07:03:56.275 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:56.264+0000 7fdc6b071700 1 -- 172.21.15.73:0/3506050847 wait complete. 2024-06-06T07:03:56.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 30472 2.85 MB/sec warmup 63 sec latency 42.820 ms 2024-06-06T07:03:57.131 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-06T07:03:57.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 30994 2.84 MB/sec warmup 64 sec latency 12.961 ms 2024-06-06T07:03:57.441 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:03:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:57 smithi073 ceph-mon[32303]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:03:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:57 smithi170 ceph-mon[36108]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:03:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:57 smithi136 ceph-mon[35901]: pgmap v81: 129 pgs: 129 active+clean; 38 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:03:58.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 31681 2.90 MB/sec warmup 65 sec latency 76.934 ms 2024-06-06T07:03:59.325 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 -- 172.21.15.73:0/1109404096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 msgr2=0x7fb4d810e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 --2- 172.21.15.73:0/1109404096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d810e210 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000b0a0 tx=0x7fb4d000b3b0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 -- 172.21.15.73:0/1109404096 shutdown_connections 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 --2- 172.21.15.73:0/1109404096 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 0x7fb4d8107840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 --2- 172.21.15.73:0/1109404096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d810e210 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 --2- 172.21.15.73:0/1109404096 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 -- 172.21.15.73:0/1109404096 >> 172.21.15.73:0/1109404096 conn(0x7fb4d8077430 msgr2=0x7fb4d8077840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 -- 172.21.15.73:0/1109404096 shutdown_connections 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.323+0000 7fb4de679700 1 -- 172.21.15.73:0/1109404096 wait complete. 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 Processor -- start 2024-06-06T07:03:59.326 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 -- start start 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d819d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 0x7fb4d81977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d8111520 con 0x7fb4d8101d80 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81113a0 con 0x7fb4d81013c0 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.324+0000 7fb4de679700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81116a0 con 0x7fb4d8107460 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d819d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.328 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40610/0 (socket says 172.21.15.73:40610) 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d77fe700 1 -- 172.21.15.73:0/1817880024 learned_addr learned my addr 172.21.15.73:0/1817880024 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 0x7fb4d81977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 -- 172.21.15.73:0/1817880024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 msgr2=0x7fb4d81977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 0x7fb4d81977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 -- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 msgr2=0x7fb4d819db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 -- 172.21.15.73:0/1817880024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4d8197fa0 con 0x7fb4d81013c0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d819d640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e9f0 tx=0x7fb4c800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d57fa700 1 -- 172.21.15.73:0/1817880024 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c800c9c0 con 0x7fb4d81013c0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4d8198290 con 0x7fb4d81013c0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d57fa700 1 -- 172.21.15.73:0/1817880024 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4c800cb60 con 0x7fb4d81013c0 2024-06-06T07:03:59.329 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.325+0000 7fb4d57fa700 1 -- 172.21.15.73:0/1817880024 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4c8018680 con 0x7fb4d81013c0 2024-06-06T07:03:59.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.326+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4d81a44a0 con 0x7fb4d81013c0 2024-06-06T07:03:59.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.328+0000 7fb4d57fa700 1 -- 172.21.15.73:0/1817880024 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4c8018820 con 0x7fb4d81013c0 2024-06-06T07:03:59.330 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.329+0000 7fb4d57fa700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 0x7fb4c00a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.329+0000 7fb4d57fa700 1 -- 172.21.15.73:0/1817880024 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb4c80cf930 con 0x7fb4d81013c0 2024-06-06T07:03:59.331 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.329+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 0x7fb4c00a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 Processor -- start 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 -- start start 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 0x7fb4c00a5250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8077b70 tx=0x7fb4cc009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 0x5609065ab340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 0x5609065a5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560906497f60 con 0x5609065ab880 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5609064f6850 con 0x5609065aaf60 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56090645e460 con 0x5609065a57a0 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40624/0 (socket says 172.21.15.73:40624) 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4d77fe700 1 -- 172.21.15.73:0/2752332459 learned_addr learned my addr 172.21.15.73:0/2752332459 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 0x5609065a5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.330+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 0x5609065ab340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 -- 172.21.15.73:0/2752332459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 msgr2=0x5609065a5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 0x5609065a5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 -- 172.21.15.73:0/2752332459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 msgr2=0x5609065ab340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 0x5609065ab340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 -- 172.21.15.73:0/2752332459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609064458d0 con 0x5609065ab880 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 0x5609065ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 0x5609065a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009880 tx=0x7fb4cc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931875458916d904 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.334 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4be7fc700 1 -- 172.21.15.73:0/2752332459 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc026630 con 0x5609065ab880 2024-06-06T07:03:59.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4be7fc700 1 -- 172.21.15.73:0/2752332459 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc01c8c0 con 0x5609065ab880 2024-06-06T07:03:59.335 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.331+0000 7fb4be7fc700 1 -- 172.21.15.73:0/2752332459 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc01d930 con 0x5609065ab880 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/2752332459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 msgr2=0x5609065a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009880 tx=0x7fb4cc014000 comp rx=0 tx=0).stop 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/2752332459 shutdown_connections 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065a57a0 0x5609065a5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065aaf60 0x5609065ab340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/2752332459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x5609065a5260 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/2752332459 >> 172.21.15.73:0/2752332459 conn(0x5609064d2080 msgr2=0x5609065a0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/2752332459 shutdown_connections 2024-06-06T07:03:59.338 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/2752332459 wait complete. 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.332+0000 7fb4e8efcb80 1 -- start start 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 Processor -- start 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560906496fc0 con 0x5609065aaf60 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5609064f2810 con 0x5609065a57a0 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5609065b2160 con 0x5609065ab880 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4e8efcb80 10 client.0 _ll_register_callbacks cb 0x56090648c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40636/0 (socket says 172.21.15.73:40636) 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4d77fe700 1 -- 172.21.15.73:0/293052604 learned_addr learned my addr 172.21.15.73:0/293052604 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:59.339 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.333+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 -- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 msgr2=0x5609065ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 -- 172.21.15.73:0/293052604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 msgr2=0x5609065acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 -- 172.21.15.73:0/293052604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609064bc750 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00ef90 tx=0x7fb4cc01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae7053e7e77e368 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc034060 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb4cc01ca70 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc01c8c0 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4cc03b050 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4e8efcb80 10 client.16053 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.334+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609065c0e60 con 0x5609065aaf60 2024-06-06T07:03:59.340 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc01c8c0 con 0x5609065aaf60 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb4cc004940 con 0x5609065aaf60 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 10 client.16053 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 10 client.16053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 20 client.16053 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065c33e0 0x5609065c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 4 client.16053 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.335+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560906503d50 con 0x5609065c33e0 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.336+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065c33e0 0x5609065c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.336+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065c33e0 0x5609065c57d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3cb4ae9a01d425 server_cookie=57b312627c2d1887 in_seq=0 out_seq=0 2024-06-06T07:03:59.341 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.336+0000 7fb4b77fe700 10 client.16053 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:59.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4b77fe700 1 -- 172.21.15.73:0/293052604 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560906503d50 con 0x5609065c33e0 2024-06-06T07:03:59.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4b77fe700 10 client.16053 handle_command_reply: tid=0 2024-06-06T07:03:59.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 client.16053 shutdown 2024-06-06T07:03:59.342 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 msgr2=0x5609065ad2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00ef90 tx=0x7fb4cc01c410 comp rx=0 tx=0).stop 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 shutdown_connections 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065c33e0 0x5609065c57d0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/293052604 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065ad2a0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 >> 172.21.15.73:0/293052604 conn(0x5609064d2080 msgr2=0x56090659f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.337+0000 7fb4b77fe700 0 client.16053 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.338+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 shutdown_connections 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.338+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/293052604 wait complete. 2024-06-06T07:03:59.343 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.338+0000 7fb4e8efcb80 20 client.16053 trim_cache size 0 max 16384 2024-06-06T07:03:59.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 Processor -- start 2024-06-06T07:03:59.357 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 -- start start 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065b0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ac3a0 con 0x5609065aaf60 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81adf40 con 0x5609065a57a0 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81aca70 con 0x5609065ab880 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40644/0 (socket says 172.21.15.73:40644) 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 -- 172.21.15.73:0/950571386 learned_addr learned my addr 172.21.15.73:0/950571386 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065b0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.358 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 -- 172.21.15.73:0/950571386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 msgr2=0x5609065b0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065b0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 -- 172.21.15.73:0/950571386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 msgr2=0x5609065c57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065c57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.355+0000 7fb4d77fe700 1 -- 172.21.15.73:0/950571386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609065a6190 con 0x5609065aaf60 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.356+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc016040 tx=0x7fb4cc009fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805f6eb9f1413d57 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.356+0000 7fb4bdffb700 1 -- 172.21.15.73:0/950571386 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc044070 con 0x5609065aaf60 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.356+0000 7fb4bdffb700 1 -- 172.21.15.73:0/950571386 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0092d0 con 0x5609065aaf60 2024-06-06T07:03:59.359 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.356+0000 7fb4bdffb700 1 -- 172.21.15.73:0/950571386 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc034050 con 0x5609065aaf60 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/950571386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 msgr2=0x5609065aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc016040 tx=0x7fb4cc009fc0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/950571386 shutdown_connections 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065ab880 0x5609065b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065c57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.362 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/950571386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065aaf60 0x5609065aa540 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/950571386 >> 172.21.15.73:0/950571386 conn(0x5609064d2080 msgr2=0x5609065a4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/950571386 shutdown_connections 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/950571386 wait complete. 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 -- start start 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.357+0000 7fb4e8efcb80 1 Processor -- start 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065aaf60 0x5609065b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ad870 con 0x5609065ab880 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81ad140 con 0x5609065a57a0 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb4d81aa340 con 0x5609065aaf60 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4e8efcb80 10 client.0 _ll_register_callbacks cb 0x56090648c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:03:59.363 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40660/0 (socket says 172.21.15.73:40660) 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 -- 172.21.15.73:0/1882696027 learned_addr learned my addr 172.21.15.73:0/1882696027 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4d77fe700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065aaf60 0x5609065b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 -- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065aaf60 msgr2=0x5609065b8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065aaf60 0x5609065b8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 -- 172.21.15.73:0/1882696027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 msgr2=0x5609065b8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065b8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 -- 172.21.15.73:0/1882696027 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090662c470 con 0x5609065ab880 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4dcc16700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000bf40 tx=0x7fb4d0003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4618a436df4f62b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.358+0000 7fb4b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4d002a060 con 0x5609065ab880 2024-06-06T07:03:59.364 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb4d00074e0 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4cc0092d0 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fb4d0031050 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 10 client.16059 fetch_fsmap learned FSMap version 34 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5609065b44e0 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4cc0092d0 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fb4d0024070 con 0x5609065ab880 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 10 client.16059 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 10 client.16059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.i 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 20 client.16059 populate_metadata read hostname 'smithi073' 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065b4c80 0x5609065b70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 4 client.16059 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.359+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5609064fe2d0 con 0x5609065b4c80 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.360+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065b4c80 0x5609065b70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:03:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.360+0000 7fb4b77fe700 10 client.16059 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:59.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.360+0000 7fb4d7fff700 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065b4c80 0x5609065b70a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96f1b91f53c799a server_cookie=7f41c36a7003c710 in_seq=0 out_seq=0 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4b77fe700 1 -- 172.21.15.73:0/1882696027 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5609064fe2d0 con 0x5609065b4c80 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4b77fe700 10 client.16059 handle_command_reply: tid=0 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 client.16059 shutdown 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 msgr2=0x56090662be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fb4d000bf40 tx=0x7fb4d0003a00 comp rx=0 tx=0).stop 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 shutdown_connections 2024-06-06T07:03:59.367 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5609065b4c80 0x5609065b70a0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5609065ab880 0x56090662be80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4b77fe700 0 client.16059 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5609065a57a0 0x5609065b8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 --2- 172.21.15.73:0/1882696027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5609065aaf60 0x5609065b8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 >> 172.21.15.73:0/1882696027 conn(0x5609064d2080 msgr2=0x5609065a69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 shutdown_connections 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 1 -- 172.21.15.73:0/1882696027 wait complete. 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.361+0000 7fb4e8efcb80 20 client.16059 trim_cache size 0 max 16384 2024-06-06T07:03:59.368 INFO:teuthology.orchestra.run.smithi073.stdout:[] 2024-06-06T07:03:59.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 msgr2=0x7fb4c00a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.369 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 0x7fb4c00a5250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8077b70 tx=0x7fb4cc009260 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 msgr2=0x7fb4d819d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d819d640 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4c800e9f0 tx=0x7fb4c800ed00 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 shutdown_connections 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4c00a2d90 0x7fb4c00a5250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb4d8107460 0x7fb4d81977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb4d81013c0 0x7fb4d819d640 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 --2- 172.21.15.73:0/1817880024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4d8101d80 0x7fb4d819db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 >> 172.21.15.73:0/1817880024 conn(0x7fb4d8077430 msgr2=0x7fb4d8100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 shutdown_connections 2024-06-06T07:03:59.370 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:03:59.362+0000 7fb4de679700 1 -- 172.21.15.73:0/1817880024 wait complete. 2024-06-06T07:03:59.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 32046 2.90 MB/sec warmup 66 sec latency 63.592 ms 2024-06-06T07:03:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:03:59 smithi170 ceph-mon[36108]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:03:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:03:59 smithi073 ceph-mon[32303]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:03:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:03:59 smithi136 ceph-mon[35901]: pgmap v82: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:04:00.250 ERROR:tasks.fwd_scrub:rank 1 damaged: [{'damage_type': 'backtrace', 'id': 195046859, 'ino': 1099511628298, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/BASEMACH.DOC'}, {'damage_type': 'backtrace', 'id': 868541695, 'ino': 1099511628324, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/NEWS1_1.LWP'}, {'damage_type': 'backtrace', 'id': 969409306, 'ino': 1099511628328, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/MOVED.DOC'}, {'damage_type': 'backtrace', 'id': 1455232718, 'ino': 1099511628297, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/CHAP10.DOC'}, {'damage_type': 'backtrace', 'id': 2213674120, 'ino': 1099511628299, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC'}, {'damage_type': 'backtrace', 'id': 2298354129, 'ino': 1099511628323, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/RESULTS.XLS'}, {'damage_type': 'backtrace', 'id': 2318418918, 'ino': 1099511628327, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/EVENTS.DOC'}, {'damage_type': 'backtrace', 'id': 3325743081, 'ino': 1099511628322, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORDPRO/BENCHS.LWP'}, {'damage_type': 'backtrace', 'id': 3440170658, 'ino': 1099511628326, 'path': '/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/PM/T1.XLS'}] 2024-06-06T07:04:00.251 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 48, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 70, in _scrub self._check_damage() File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 81, in _check_damage raise RuntimeError(f"rank damage found: {types}") RuntimeError: rank damage found: {'backtrace'} 2024-06-06T07:04:00.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 32345 2.88 MB/sec warmup 67 sec latency 42.478 ms 2024-06-06T07:04:01.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 32659 2.84 MB/sec warmup 68 sec latency 15.085 ms 2024-06-06T07:04:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:01 smithi170 ceph-mon[36108]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:01.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:01 smithi073 ceph-mon[32303]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:01 smithi136 ceph-mon[35901]: pgmap v83: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:02.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 33152 2.83 MB/sec warmup 69 sec latency 15.064 ms 2024-06-06T07:04:03.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 33796 2.84 MB/sec warmup 70 sec latency 12.017 ms 2024-06-06T07:04:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:03 smithi170 ceph-mon[36108]: pgmap v84: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:03 smithi073 ceph-mon[32303]: pgmap v84: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:03 smithi136 ceph-mon[35901]: pgmap v84: 129 pgs: 129 active+clean; 34 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.0 MiB/s wr, 147 op/s 2024-06-06T07:04:04.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 34373 2.82 MB/sec warmup 71 sec latency 19.397 ms 2024-06-06T07:04:05.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 34980 2.86 MB/sec warmup 72 sec latency 15.574 ms 2024-06-06T07:04:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:05 smithi170 ceph-mon[36108]: pgmap v85: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-06-06T07:04:05.603 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:05 smithi136 ceph-mon[35901]: pgmap v85: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-06-06T07:04:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:05 smithi073 ceph-mon[32303]: pgmap v85: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 237 op/s 2024-06-06T07:04:06.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 35489 2.88 MB/sec warmup 73 sec latency 14.689 ms 2024-06-06T07:04:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 -- 172.21.15.73:0/336953466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2010102c30 msgr2=0x7f2010103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 --2- 172.21.15.73:0/336953466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2010102c30 0x7f2010103010 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f200c009900 tx=0x7f200c02a510 comp rx=0 tx=0).stop 2024-06-06T07:04:07.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 -- 172.21.15.73:0/336953466 shutdown_connections 2024-06-06T07:04:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 --2- 172.21.15.73:0/336953466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f201010bf70 0x7f201010e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 --2- 172.21.15.73:0/336953466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 0x7f201010ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 --2- 172.21.15.73:0/336953466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2010102c30 0x7f2010103010 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 -- 172.21.15.73:0/336953466 >> 172.21.15.73:0/336953466 conn(0x7f2010077a20 msgr2=0x7f2010077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 -- 172.21.15.73:0/336953466 shutdown_connections 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.298+0000 7f201829a640 1 -- 172.21.15.73:0/336953466 wait complete. 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 Processor -- start 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 -- start start 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2010102c30 0x7f2010100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 0x7f2010100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111c20 con 0x7f201010bf70 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111aa0 con 0x7f2010102c30 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f201829a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111da0 con 0x7f2010103550 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52746/0 (socket says 172.21.15.73:52746) 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 -- 172.21.15.73:0/3716118525 learned_addr learned my addr 172.21.15.73:0/3716118525 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2015835640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 0x7f2010100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016036640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2010102c30 0x7f2010100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 -- 172.21.15.73:0/3716118525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 msgr2=0x7f2010100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 0x7f2010100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 -- 172.21.15.73:0/3716118525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2010102c30 msgr2=0x7f2010100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2010102c30 0x7f2010100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.299+0000 7f2016837640 1 -- 172.21.15.73:0/3716118525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20101a6600 con 0x7f201010bf70 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f2016837640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f200800efd0 tx=0x7f200800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f20077fe640 1 -- 172.21.15.73:0/3716118525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008003b70 con 0x7f201010bf70 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f20077fe640 1 -- 172.21.15.73:0/3716118525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2008003d10 con 0x7f201010bf70 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20101a68f0 con 0x7f201010bf70 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f20077fe640 1 -- 172.21.15.73:0/3716118525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2008010980 con 0x7f201010bf70 2024-06-06T07:04:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.300+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a6df0 con 0x7f201010bf70 2024-06-06T07:04:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.302+0000 7f20077fe640 1 -- 172.21.15.73:0/3716118525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2008004020 con 0x7f201010bf70 2024-06-06T07:04:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 Processor -- start 2024-06-06T07:04:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 -- start start 2024-06-06T07:04:07.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 0x5621fae34e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae35350 0x5621fae37740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5621fad8e4f0 con 0x5621fae2aa90 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5621fad2c870 con 0x5621fae35350 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.303+0000 7f201afa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5621fad8a4b0 con 0x5621fae2a170 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f20077fe640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 0x7f1ffc0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2016036640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 0x5621fae34e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34942/0 (socket says 172.21.15.73:34942) 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 -- 172.21.15.73:0/2240661085 learned_addr learned my addr 172.21.15.73:0/2240661085 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2016837640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 0x7f1ffc0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae35350 0x5621fae37740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f20077fe640 1 -- 172.21.15.73:0/3716118525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f20080cfb80 con 0x7f201010bf70 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 -- 172.21.15.73:0/2240661085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae35350 msgr2=0x5621fae37740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae35350 0x5621fae37740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 -- 172.21.15.73:0/2240661085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 msgr2=0x5621fae34e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 0x5621fae34e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 -- 172.21.15.73:0/2240661085 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621fad489b0 con 0x5621fae2a170 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2016036640 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 0x5621fae34e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2016837640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 0x7f1ffc0a5290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20080ccfd0 tx=0x7f200806ac20 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2015835640 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2000009fd0 tx=0x7f2000005850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76063a234d8eba12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2004bfe640 1 -- 172.21.15.73:0/2240661085 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004050 con 0x5621fae2a170 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2004bfe640 1 -- 172.21.15.73:0/2240661085 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000054d0 con 0x5621fae2a170 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.304+0000 7f2004bfe640 1 -- 172.21.15.73:0/2240661085 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000004b00 con 0x5621fae2a170 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 -- 172.21.15.73:0/2240661085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 msgr2=0x5621fae2a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2000009fd0 tx=0x7f2000005850 comp rx=0 tx=0).stop 2024-06-06T07:04:07.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 -- 172.21.15.73:0/2240661085 shutdown_connections 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae35350 0x5621fae37740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2aa90 0x5621fae34e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 --2- 172.21.15.73:0/2240661085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae2a550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 -- 172.21.15.73:0/2240661085 >> 172.21.15.73:0/2240661085 conn(0x5621fae26680 msgr2=0x5621fae26a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 -- 172.21.15.73:0/2240661085 shutdown_connections 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.305+0000 7f201afa5740 1 -- 172.21.15.73:0/2240661085 wait complete. 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 -- start start 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 Processor -- start 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae32830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae35350 0x5621fae32d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5621fae37f80 con 0x5621fae2a170 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5621fad17ae0 con 0x5621fae2aa90 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5621fae38100 con 0x5621fae35350 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f201afa5740 10 client.0 _ll_register_callbacks cb 0x5621fad509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52762/0 (socket says 172.21.15.73:52762) 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 -- 172.21.15.73:0/1561688557 learned_addr learned my addr 172.21.15.73:0/1561688557 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2016036640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae32830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae35350 0x5621fae32d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 -- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae35350 msgr2=0x5621fae32d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae35350 0x5621fae32d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 -- 172.21.15.73:0/1561688557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 msgr2=0x5621fae32830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae32830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.306+0000 7f2015835640 1 -- 172.21.15.73:0/1561688557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621faaa2210 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f2015835640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2000005bc0 tx=0x7f20000097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11d786927253bce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000023070 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f2000004d10 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20000054d0 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f2000035050 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 10 client.16071 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621fab6e620 con 0x5621fae2a170 2024-06-06T07:04:07.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f20000054d0 con 0x5621fae2a170 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f200001f6a0 con 0x5621fae2a170 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 10 client.16071 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 20 client.16071 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621fae47770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 4 client.16071 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.307+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621fada1f10 con 0x5621fae45380 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.308+0000 7f2016036640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621fae47770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.308+0000 7f2016036640 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621fae47770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c21e86e9bbe6c02 server_cookie=c11ed1ce9848a7c6 in_seq=0 out_seq=0 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.308+0000 7f1ff9ffb640 10 client.16071 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/1561688557 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621fada1f10 con 0x5621fae45380 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f1ff9ffb640 10 client.16071 handle_command_reply: tid=0 2024-06-06T07:04:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 client.16071 shutdown 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 msgr2=0x5621fae322f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f2000005bc0 tx=0x7f20000097a0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 shutdown_connections 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621fae47770 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f1ff9ffb640 0 client.16071 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae35350 0x5621fae32d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae32830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 --2- 172.21.15.73:0/1561688557 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae2a170 0x5621fae322f0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 >> 172.21.15.73:0/1561688557 conn(0x5621fae26680 msgr2=0x5621fae2c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 shutdown_connections 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 1 -- 172.21.15.73:0/1561688557 wait complete. 2024-06-06T07:04:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.309+0000 7f201afa5740 20 client.16071 trim_cache size 0 max 16384 2024-06-06T07:04:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 Processor -- start 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 -- start start 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae34230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20101b0f70 con 0x5621fae35350 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20101b2b10 con 0x5621fae2aa90 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f201afa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20101b1640 con 0x5621fae2a170 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52764/0 (socket says 172.21.15.73:52764) 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2016036640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae4cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 -- 172.21.15.73:0/3702752074 learned_addr learned my addr 172.21.15.73:0/3702752074 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae34230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 -- 172.21.15.73:0/3702752074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 msgr2=0x5621fae34230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae34230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 -- 172.21.15.73:0/3702752074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 msgr2=0x5621fae4cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae4cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2015835640 1 -- 172.21.15.73:0/3702752074 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621fae21f70 con 0x5621fae35350 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.326+0000 7f2016036640 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:07.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f2015835640 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20000059f0 tx=0x7f2000000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61014129aeeb0e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f1ffbfff640 1 -- 172.21.15.73:0/3702752074 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000032020 con 0x5621fae35350 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f1ffbfff640 1 -- 172.21.15.73:0/3702752074 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2000005470 con 0x5621fae35350 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f1ffbfff640 1 -- 172.21.15.73:0/3702752074 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000023070 con 0x5621fae35350 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f201afa5740 1 -- 172.21.15.73:0/3702752074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 msgr2=0x5621fae4d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.327+0000 7f201afa5740 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f20000059f0 tx=0x7f2000000bc0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- 172.21.15.73:0/3702752074 shutdown_connections 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae4d490 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- 172.21.15.73:0/3702752074 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae34230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- 172.21.15.73:0/3702752074 >> 172.21.15.73:0/3702752074 conn(0x5621fae26680 msgr2=0x5621fae33b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- 172.21.15.73:0/3702752074 shutdown_connections 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- 172.21.15.73:0/3702752074 wait complete. 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- start start 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 Processor -- start 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae3dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20101b2440 con 0x5621fae35350 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f20101b1d10 con 0x5621fae2aa90 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f20101aef50 con 0x5621fae2a170 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f201afa5740 10 client.0 _ll_register_callbacks cb 0x7f2000005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2016036640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2016036640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:52318/0 (socket says 172.21.15.73:52318) 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2016036640 1 -- 172.21.15.73:0/3326490144 learned_addr learned my addr 172.21.15.73:0/3326490144 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2015835640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52768/0 (socket says 172.21.15.73:52768) 2024-06-06T07:04:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.328+0000 7f2015835640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae3dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 -- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 msgr2=0x5621fae3dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae3dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 -- 172.21.15.73:0/3326490144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 msgr2=0x5621fae39730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 -- 172.21.15.73:0/3326490144 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621fae2b200 con 0x5621fae2aa90 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2015835640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f2016036640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f200c02aa30 tx=0x7f200c02aca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f825fb97aabd529d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f1ff9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200c035040 con 0x5621fae2aa90 2024-06-06T07:04:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f200c004450 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2000005470 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f200c055050 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f201afa5740 10 client.25511 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5621faeaff90 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.329+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2000005470 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f200c004cb0 con 0x5621fae2aa90 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 10 client.25511 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 10 client.25511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 20 client.25511 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621faeb27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 4 client.25511 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621fad9bc20 con 0x5621fae45380 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f2015835640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621faeb27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f2015835640 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621faeb27e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b9f1b2e86e85d94 server_cookie=836b27fdd27216e in_seq=0 out_seq=0 2024-06-06T07:04:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.330+0000 7f1ff9ffb640 10 client.25511 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f1ff9ffb640 1 -- 172.21.15.73:0/3326490144 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5621fad9bc20 con 0x5621fae45380 2024-06-06T07:04:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f1ff9ffb640 10 client.25511 handle_command_reply: tid=0 2024-06-06T07:04:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 client.25511 shutdown 2024-06-06T07:04:07.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 msgr2=0x5621fae38ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f200c02aa30 tx=0x7f200c02aca0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 shutdown_connections 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5621fae45380 0x5621faeb27e0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621fae35350 0x5621fae39730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5621fae2aa90 0x5621fae38ff0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 --2- 172.21.15.73:0/3326490144 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5621fae2a170 0x5621fae3dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f1ff9ffb640 0 client.25511 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 >> 172.21.15.73:0/3326490144 conn(0x5621fae26680 msgr2=0x5621fae2c320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 shutdown_connections 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 1 -- 172.21.15.73:0/3326490144 wait complete. 2024-06-06T07:04:07.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.354+0000 7f201afa5740 20 client.25511 trim_cache size 0 max 16384 2024-06-06T07:04:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:04:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:04:07.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 msgr2=0x7f1ffc0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 0x7f1ffc0a5290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f20080ccfd0 tx=0x7f200806ac20 comp rx=0 tx=0).stop 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 msgr2=0x7f2010101300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f200800efd0 tx=0x7f200800c6a0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 shutdown_connections 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f201010bf70 0x7f2010101300 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1ffc0a2dd0 0x7f1ffc0a5290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2010103550 0x7f2010100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 --2- 172.21.15.73:0/3716118525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2010102c30 0x7f2010100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 >> 172.21.15.73:0/3716118525 conn(0x7f2010077a20 msgr2=0x7f20100fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 shutdown_connections 2024-06-06T07:04:07.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:07.355+0000 7f201829a640 1 -- 172.21.15.73:0/3716118525 wait complete. 2024-06-06T07:04:07.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 35848 2.87 MB/sec warmup 74 sec latency 19.132 ms 2024-06-06T07:04:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:07 smithi170 ceph-mon[36108]: pgmap v86: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-06-06T07:04:07.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:07 smithi073 ceph-mon[32303]: pgmap v86: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-06-06T07:04:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:07 smithi136 ceph-mon[35901]: pgmap v86: 129 pgs: 129 active+clean; 39 MiB data, 575 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 165 op/s 2024-06-06T07:04:08.434 INFO:tasks.workunit.client.0.smithi073.stdout: 1 36127 2.83 MB/sec warmup 75 sec latency 19.345 ms 2024-06-06T07:04:09.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 36651 2.82 MB/sec warmup 76 sec latency 10.969 ms 2024-06-06T07:04:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:09 smithi073 ceph-mon[32303]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2024-06-06T07:04:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:09 smithi136 ceph-mon[35901]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2024-06-06T07:04:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:09 smithi170 ceph-mon[36108]: pgmap v87: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 244 op/s 2024-06-06T07:04:10.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 37221 2.82 MB/sec warmup 77 sec latency 17.252 ms 2024-06-06T07:04:11.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 37851 2.81 MB/sec warmup 78 sec latency 17.214 ms 2024-06-06T07:04:11.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:11 smithi073 ceph-mon[32303]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:11 smithi136 ceph-mon[35901]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:11.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:11 smithi170 ceph-mon[36108]: pgmap v88: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:12.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 38436 2.84 MB/sec warmup 79 sec latency 15.098 ms 2024-06-06T07:04:13.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 38865 2.83 MB/sec warmup 80 sec latency 16.027 ms 2024-06-06T07:04:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:13 smithi073 ceph-mon[32303]: pgmap v89: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:13 smithi136 ceph-mon[35901]: pgmap v89: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:13.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:13 smithi170 ceph-mon[36108]: pgmap v89: 129 pgs: 129 active+clean; 43 MiB data, 560 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 168 op/s 2024-06-06T07:04:14.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 39323 2.85 MB/sec warmup 81 sec latency 15.474 ms 2024-06-06T07:04:15.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 39608 2.82 MB/sec warmup 82 sec latency 15.587 ms 2024-06-06T07:04:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:15 smithi073 ceph-mon[32303]: pgmap v90: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-06-06T07:04:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:15 smithi136 ceph-mon[35901]: pgmap v90: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-06-06T07:04:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:15 smithi170 ceph-mon[36108]: pgmap v90: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 241 op/s 2024-06-06T07:04:16.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 40060 2.80 MB/sec warmup 83 sec latency 15.485 ms 2024-06-06T07:04:17.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 40593 2.82 MB/sec warmup 84 sec latency 17.951 ms 2024-06-06T07:04:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:17 smithi073 ceph-mon[32303]: pgmap v91: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:17 smithi136 ceph-mon[35901]: pgmap v91: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:17 smithi170 ceph-mon[36108]: pgmap v91: 129 pgs: 129 active+clean; 44 MiB data, 582 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:18.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 41260 2.79 MB/sec warmup 85 sec latency 16.167 ms 2024-06-06T07:04:19.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 41807 2.82 MB/sec warmup 86 sec latency 15.729 ms 2024-06-06T07:04:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:19 smithi073 ceph-mon[32303]: pgmap v92: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:04:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:19 smithi136 ceph-mon[35901]: pgmap v92: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:04:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:19 smithi170 ceph-mon[36108]: pgmap v92: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 230 op/s 2024-06-06T07:04:20.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 42380 2.82 MB/sec warmup 87 sec latency 14.947 ms 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- 172.21.15.73:0/233786172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa50102350 msgr2=0x7ffa5010e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/233786172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa50102350 0x7ffa5010e860 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009930 tx=0x7ffa4802a420 comp rx=0 tx=0).stop 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- 172.21.15.73:0/233786172 shutdown_connections 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/233786172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa50102350 0x7ffa5010e860 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/233786172 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 0x7ffa50101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/233786172 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa501079a0 0x7ffa50107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- 172.21.15.73:0/233786172 >> 172.21.15.73:0/233786172 conn(0x7ffa500fd460 msgr2=0x7ffa500ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- 172.21.15.73:0/233786172 shutdown_connections 2024-06-06T07:04:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- 172.21.15.73:0/233786172 wait complete. 2024-06-06T07:04:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 Processor -- start 2024-06-06T07:04:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.452+0000 7ffa56ec3640 1 -- start start 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 0x7ffa501980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 0x7ffa50198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111c60 con 0x7ffa501079a0 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111ae0 con 0x7ffa50102350 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa56ec3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa50111de0 con 0x7ffa50101990 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa55460640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa54c5f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 0x7ffa501980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa55460640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39418/0 (socket says 172.21.15.73:39418) 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa55460640 1 -- 172.21.15.73:0/2235051262 learned_addr learned my addr 172.21.15.73:0/2235051262 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 0x7ffa50198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 -- 172.21.15.73:0/2235051262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 msgr2=0x7ffa501980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 0x7ffa501980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 -- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 msgr2=0x7ffa5019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 -- 172.21.15.73:0/2235051262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5019d0d0 con 0x7ffa50102350 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa55460640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 0x7ffa50198620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.453+0000 7ffa4dffb640 1 -- 172.21.15.73:0/2235051262 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c00ccc0 con 0x7ffa50102350 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.454+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5019d3c0 con 0x7ffa50102350 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.454+0000 7ffa4dffb640 1 -- 172.21.15.73:0/2235051262 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c005020 con 0x7ffa50102350 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.454+0000 7ffa4dffb640 1 -- 172.21.15.73:0/2235051262 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c004a00 con 0x7ffa50102350 2024-06-06T07:04:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.454+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa501a4c50 con 0x7ffa50102350 2024-06-06T07:04:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.456+0000 7ffa4dffb640 1 -- 172.21.15.73:0/2235051262 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ffa3c004ba0 con 0x7ffa50102350 2024-06-06T07:04:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.456+0000 7ffa59bcb740 1 Processor -- start 2024-06-06T07:04:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.456+0000 7ffa4dffb640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 0x7ffa400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 -- start start 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 0x7ffa400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4dffb640 1 -- 172.21.15.73:0/2235051262 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa3c0cf760 con 0x7ffa50102350 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39170 0x55bacbc39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bacbb9d4f0 con 0x55bacbc39a90 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bacbb3b870 con 0x55bacbc39170 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bacbb994b0 con 0x55bacbc44350 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39420/0 (socket says 172.21.15.73:39420) 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3708558288 learned_addr learned my addr 172.21.15.73:0/3708558288 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa55460640 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39170 0x55bacbc39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3708558288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 msgr2=0x55bacbc46740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc46740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 0x7ffa400a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005fd0 tx=0x7ffa44009680 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3708558288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39170 msgr2=0x55bacbc39550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39170 0x55bacbc39550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3708558288 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacbb579b0 con 0x55bacbc39a90 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.457+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c097d30 tx=0x7ffa3c09c110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=650c37da76202e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa3affd640 1 -- 172.21.15.73:0/3708558288 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0b4070 con 0x55bacbc39a90 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa3affd640 1 -- 172.21.15.73:0/3708558288 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c099bd0 con 0x55bacbc39a90 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa3affd640 1 -- 172.21.15.73:0/3708558288 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c01c040 con 0x55bacbc39a90 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3708558288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 msgr2=0x55bacbc43e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c097d30 tx=0x7ffa3c09c110 comp rx=0 tx=0).stop 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3708558288 shutdown_connections 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc43e10 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3708558288 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39170 0x55bacbc39550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3708558288 >> 172.21.15.73:0/3708558288 conn(0x55bacbc35680 msgr2=0x55bacbc35a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3708558288 shutdown_connections 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.458+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3708558288 wait complete. 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 -- start start 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 Processor -- start 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc41890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc41dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bacbc46f80 con 0x55bacbc39170 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55bacbb26ae0 con 0x55bacbc39a90 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55bacbc47100 con 0x55bacbc44350 2024-06-06T07:04:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa59bcb740 10 client.0 _ll_register_callbacks cb 0x55bacbb5f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa54c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa54c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39434/0 (socket says 172.21.15.73:39434) 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa54c5f640 1 -- 172.21.15.73:0/1753387897 learned_addr learned my addr 172.21.15.73:0/1753387897 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc41890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.459+0000 7ffa55460640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc41dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 -- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 msgr2=0x55bacbc41dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc41dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 -- 172.21.15.73:0/1753387897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 msgr2=0x55bacbc41350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 -- 172.21.15.73:0/1753387897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacb8b1210 con 0x55bacbc39a90 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc41890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c099b80 tx=0x7ffa3c0ccae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53e807c57d0dac41 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0b8020 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa3c0051d0 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3c099bd0 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa3c0b4070 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 10 client.25523 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bacb97d620 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c099bd0 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ffa3c0a40b0 con 0x55bacbc39a90 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 10 client.25523 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 10 client.25523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.460+0000 7ffa59bcb740 20 client.25523 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbc543c0 0x55bacbc567b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa59bcb740 4 client.25523 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:04:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bacbbb0f10 con 0x55bacbc543c0 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbc543c0 0x55bacbc567b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa2ffff640 10 client.25523 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.461+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbc543c0 0x55bacbc567b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae62cdb077d38696 server_cookie=56001da7eb36e960 in_seq=0 out_seq=0 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa2ffff640 1 -- 172.21.15.73:0/1753387897 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bacbbb0f10 con 0x55bacbc543c0 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa2ffff640 10 client.25523 handle_command_reply: tid=0 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 client.25523 shutdown 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 msgr2=0x55bacbc41890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc41890 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c099b80 tx=0x7ffa3c0ccae0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 shutdown_connections 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbc543c0 0x55bacbc567b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc41dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc41890 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/1753387897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc41350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa2ffff640 0 client.25523 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.462+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 >> 172.21.15.73:0/1753387897 conn(0x55bacbc35680 msgr2=0x55bacbc3b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.463+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 shutdown_connections 2024-06-06T07:04:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.463+0000 7ffa59bcb740 1 -- 172.21.15.73:0/1753387897 wait complete. 2024-06-06T07:04:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.463+0000 7ffa59bcb740 20 client.25523 trim_cache size 0 max 16384 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 Processor -- start 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 -- start start 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc39170 0x55bacbc3bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 0x55bacbc597d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501aee20 con 0x55bacbc39a90 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b09c0 con 0x55bacbc44350 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501af4f0 con 0x55bacbc39170 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39442/0 (socket says 172.21.15.73:39442) 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3294021287 learned_addr learned my addr 172.21.15.73:0/3294021287 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc39170 0x55bacbc3bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa55460640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 0x55bacbc597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3294021287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc39170 msgr2=0x55bacbc3bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc39170 0x55bacbc3bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3294021287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 msgr2=0x55bacbc597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 0x55bacbc597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 -- 172.21.15.73:0/3294021287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bacbc39880 con 0x55bacbc39a90 2024-06-06T07:04:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa55460640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 0x55bacbc597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.478+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c09b5d0 tx=0x7ffa3c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f236d4d415a16b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa3a7fc640 1 -- 172.21.15.73:0/3294021287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c0b8040 con 0x55bacbc39a90 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa3a7fc640 1 -- 172.21.15.73:0/3294021287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c0ba2c0 con 0x55bacbc39a90 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa3a7fc640 1 -- 172.21.15.73:0/3294021287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa3c01c070 con 0x55bacbc39a90 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3294021287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 msgr2=0x55bacbc4fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c09b5d0 tx=0x7ffa3c009f60 comp rx=0 tx=0).stop 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3294021287 shutdown_connections 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc44350 0x55bacbc597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39a90 0x55bacbc4fa60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3294021287 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc39170 0x55bacbc3bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3294021287 >> 172.21.15.73:0/3294021287 conn(0x55bacbc35680 msgr2=0x55bacbc4f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3294021287 shutdown_connections 2024-06-06T07:04:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.479+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3294021287 wait complete. 2024-06-06T07:04:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 -- start start 2024-06-06T07:04:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 Processor -- start 2024-06-06T07:04:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc56fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501b02f0 con 0x55bacbc39170 2024-06-06T07:04:20.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501afbc0 con 0x55bacbc39a90 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ace00 con 0x55bacbc44350 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa59bcb740 10 client.0 _ll_register_callbacks cb 0x7ffa3c0b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa54c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa54c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39458/0 (socket says 172.21.15.73:39458) 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa55460640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa54c5f640 1 -- 172.21.15.73:0/3942974206 learned_addr learned my addr 172.21.15.73:0/3942974206 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.480+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc56fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 -- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 msgr2=0x55bacbc57500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc57500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 -- 172.21.15.73:0/3942974206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 msgr2=0x55bacbc56fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc56fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 -- 172.21.15.73:0/3942974206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bacbc57a40 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa54c5f640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55bacbc47070 tx=0x7ffa44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b2be49473525e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44020e60 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ffa44032480 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3c0ba2c0 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ffa4403b050 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa59bcb740 10 client.16095 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55bacbcbe9d0 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.481+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa3c0ba2c0 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ffa4400f040 con 0x55bacbc39170 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 10 client.16095 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 20 client.16095 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:20.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbcbf160 0x55bacbcc1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 4 client.16095 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:04:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bacbbaac20 con 0x55bacbcbf160 2024-06-06T07:04:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbcbf160 0x55bacbcc1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa2ffff640 10 client.16095 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:20.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.482+0000 7ffa4ffff640 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbcbf160 0x55bacbcc1580 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d146b91daf43447 server_cookie=49f0d6d39d8eee6 in_seq=0 out_seq=0 2024-06-06T07:04:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa2ffff640 1 -- 172.21.15.73:0/3942974206 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bacbbaac20 con 0x55bacbcbf160 2024-06-06T07:04:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa2ffff640 10 client.16095 handle_command_reply: tid=0 2024-06-06T07:04:20.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 client.16095 shutdown 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 msgr2=0x55bacbc59c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x55bacbc47070 tx=0x7ffa44004ab0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 shutdown_connections 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55bacbcbf160 0x55bacbcc1580 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55bacbc44350 0x55bacbc57500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa2ffff640 0 client.16095 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55bacbc39a90 0x55bacbc56fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 --2- 172.21.15.73:0/3942974206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bacbc39170 0x55bacbc59c30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 >> 172.21.15.73:0/3942974206 conn(0x55bacbc35680 msgr2=0x55bacbc41b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 shutdown_connections 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.502+0000 7ffa59bcb740 1 -- 172.21.15.73:0/3942974206 wait complete. 2024-06-06T07:04:20.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa59bcb740 20 client.16095 trim_cache size 0 max 16384 2024-06-06T07:04:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:04:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:04:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 msgr2=0x7ffa400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 0x7ffa400a5260 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ffa44005fd0 tx=0x7ffa44009680 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 msgr2=0x7ffa50198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 0x7ffa50198620 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c00e9f0 tx=0x7ffa3c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 shutdown_connections 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa501079a0 0x7ffa5019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ffa50102350 0x7ffa50198620 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ffa400a2da0 0x7ffa400a5260 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 --2- 172.21.15.73:0/2235051262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ffa50101990 0x7ffa501980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 >> 172.21.15.73:0/2235051262 conn(0x7ffa500fd460 msgr2=0x7ffa500fdb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 shutdown_connections 2024-06-06T07:04:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:20.503+0000 7ffa56ec3640 1 -- 172.21.15.73:0/2235051262 wait complete. 2024-06-06T07:04:21.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 42839 2.84 MB/sec warmup 88 sec latency 16.153 ms 2024-06-06T07:04:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:21 smithi073 ceph-mon[32303]: pgmap v93: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:21 smithi136 ceph-mon[35901]: pgmap v93: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:21 smithi170 ceph-mon[36108]: pgmap v93: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:22.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 43078 2.81 MB/sec warmup 89 sec latency 15.289 ms 2024-06-06T07:04:23.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 43534 2.79 MB/sec warmup 90 sec latency 15.697 ms 2024-06-06T07:04:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:23 smithi073 ceph-mon[32303]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:23 smithi136 ceph-mon[35901]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:23 smithi170 ceph-mon[36108]: pgmap v94: 129 pgs: 129 active+clean; 44 MiB data, 589 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:24.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 44038 2.81 MB/sec warmup 91 sec latency 16.108 ms 2024-06-06T07:04:25.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 44660 2.78 MB/sec warmup 92 sec latency 18.546 ms 2024-06-06T07:04:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:25 smithi073 ceph-mon[32303]: pgmap v95: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:04:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:25 smithi136 ceph-mon[35901]: pgmap v95: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:04:25.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:25 smithi170 ceph-mon[36108]: pgmap v95: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 226 op/s 2024-06-06T07:04:26.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 45178 2.78 MB/sec warmup 93 sec latency 16.045 ms 2024-06-06T07:04:27.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 45891 2.82 MB/sec warmup 94 sec latency 15.994 ms 2024-06-06T07:04:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:27 smithi073 ceph-mon[32303]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-06T07:04:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:27 smithi136 ceph-mon[35901]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-06T07:04:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:27 smithi170 ceph-mon[36108]: pgmap v96: 129 pgs: 129 active+clean; 44 MiB data, 562 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-06T07:04:28.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 46318 2.83 MB/sec warmup 95 sec latency 14.599 ms 2024-06-06T07:04:29.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 46594 2.81 MB/sec warmup 96 sec latency 15.672 ms 2024-06-06T07:04:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:29 smithi073 ceph-mon[32303]: pgmap v97: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-06-06T07:04:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:29 smithi136 ceph-mon[35901]: pgmap v97: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-06-06T07:04:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:29 smithi170 ceph-mon[36108]: pgmap v97: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 225 op/s 2024-06-06T07:04:30.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 47012 2.79 MB/sec warmup 97 sec latency 15.035 ms 2024-06-06T07:04:31.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 47538 2.80 MB/sec warmup 98 sec latency 14.887 ms 2024-06-06T07:04:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:31 smithi073 ceph-mon[32303]: pgmap v98: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:31 smithi136 ceph-mon[35901]: pgmap v98: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:31 smithi170 ceph-mon[36108]: pgmap v98: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:32.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 48150 2.78 MB/sec warmup 99 sec latency 16.362 ms 2024-06-06T07:04:33.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 48695 2.78 MB/sec warmup 100 sec latency 14.456 ms 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/2266292023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 msgr2=0x7fa6b810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/2266292023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b810e410 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009ca0 tx=0x7fa6b002a9e0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/2266292023 shutdown_connections 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/2266292023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b810e410 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/2266292023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8103550 0x7fa6b810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/2266292023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8102c30 0x7fa6b8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/2266292023 >> 172.21.15.73:0/2266292023 conn(0x7fa6b8077a20 msgr2=0x7fa6b8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/2266292023 shutdown_connections 2024-06-06T07:04:33.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.598+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/2266292023 wait complete. 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 Processor -- start 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 -- start start 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 0x7fa6b8100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b8101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8111c70 con 0x7fa6b810bf70 2024-06-06T07:04:33.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8111af0 con 0x7fa6b8103550 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6bcdf9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b8111df0 con 0x7fa6b8102c30 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 0x7fa6b8100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 0x7fa6b8100880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:38126/0 (socket says 172.21.15.73:38126) 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56328/0 (socket says 172.21.15.73:56328) 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b659c640 1 -- 172.21.15.73:0/4113054192 learned_addr learned my addr 172.21.15.73:0/4113054192 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.599+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b8101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/4113054192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 msgr2=0x7fa6b8100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 0x7fa6b8100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 msgr2=0x7fa6b8101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b8101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/4113054192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6b81a65b0 con 0x7fa6b8103550 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00e9f0 tx=0x7fa6ac00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6a77fe640 1 -- 172.21.15.73:0/4113054192 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac00ccc0 con 0x7fa6b8103550 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6b81a68a0 con 0x7fa6b8103550 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6a77fe640 1 -- 172.21.15.73:0/4113054192 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6ac005020 con 0x7fa6b8103550 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6a77fe640 1 -- 172.21.15.73:0/4113054192 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac004a00 con 0x7fa6b8103550 2024-06-06T07:04:33.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.600+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6b81a6da0 con 0x7fa6b8103550 2024-06-06T07:04:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.604+0000 7fa6bfbe7740 1 Processor -- start 2024-06-06T07:04:33.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.605+0000 7fa6bfbe7740 1 -- start start 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28af4b60 0x559c28af6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559c28a454f0 con 0x559c28af4b60 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c289e3870 con 0x559c28ae19d0 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559c28a414b0 con 0x559c28ae10b0 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:38140/0 (socket says 172.21.15.73:38140) 2024-06-06T07:04:33.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 -- 172.21.15.73:0/3248907433 learned_addr learned my addr 172.21.15.73:0/3248907433 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b659c640 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 -- 172.21.15.73:0/3248907433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 msgr2=0x559c28af4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 -- 172.21.15.73:0/3248907433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28af4b60 msgr2=0x559c28af6f50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28af4b60 0x559c28af6f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b6d9d640 1 -- 172.21.15.73:0/3248907433 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c289ff9b0 con 0x559c28ae10b0 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.606+0000 7fa6b659c640 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.607+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0035000 tx=0x7fa6b0009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aed18347bec049ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.607+0000 7fa6a4ff9640 1 -- 172.21.15.73:0/3248907433 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0048070 con 0x559c28ae10b0 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.607+0000 7fa6a4ff9640 1 -- 172.21.15.73:0/3248907433 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6b002e840 con 0x559c28ae10b0 2024-06-06T07:04:33.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.607+0000 7fa6a4ff9640 1 -- 172.21.15.73:0/3248907433 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0033040 con 0x559c28ae10b0 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.607+0000 7fa6a77fe640 1 -- 172.21.15.73:0/4113054192 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa6ac004ba0 con 0x7fa6b8103550 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3248907433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 msgr2=0x559c28ae1490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0035000 tx=0x7fa6b0009a20 comp rx=0 tx=0).stop 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3248907433 shutdown_connections 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28af4b60 0x559c28af6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.608+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3248907433 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28ae1490 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3248907433 >> 172.21.15.73:0/3248907433 conn(0x559c28add680 msgr2=0x559c28adda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6a77fe640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 0x7fa6a00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6a77fe640 1 -- 172.21.15.73:0/4113054192 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa6ac0cf6f0 con 0x7fa6b8103550 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6b659c640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 0x7fa6a00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3248907433 shutdown_connections 2024-06-06T07:04:33.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.609+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3248907433 wait complete. 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 -- start start 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b659c640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 0x7fa6a00a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009c60 tx=0x7fa6a8005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 Processor -- start 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 0x559c28af25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559c28af76d0 con 0x559c28ae10b0 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559c289ceae0 con 0x559c28ae19d0 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559c28af7850 con 0x559c28af4b60 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6bfbe7740 10 client.0 _ll_register_callbacks cb 0x559c28a079b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56396/0 (socket says 172.21.15.73:56396) 2024-06-06T07:04:33.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b659c640 1 -- 172.21.15.73:0/2057078042 learned_addr learned my addr 172.21.15.73:0/2057078042 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.610+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 0x559c28af25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 msgr2=0x559c28af25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 0x559c28af25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/2057078042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 msgr2=0x559c28af1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/2057078042 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c28759210 con 0x559c28ae19d0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b659c640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 0x559c28af25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af20a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0cca90 tx=0x7fa6ac09b280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b346bc0b04f1f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0b4060 con 0x559c28ae19d0 2024-06-06T07:04:33.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa6ac0051d0 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6b002e840 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac01c070 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6bfbe7740 10 client.25535 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c28825620 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6b002e840 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.611+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa6ac0860c0 con 0x559c28ae19d0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 10 client.25535 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 10 client.25535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 20 client.25535 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b04aa0 0x559c28b06e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 4 client.25535 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c28a58f10 con 0x559c28b04aa0 2024-06-06T07:04:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6b659c640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b04aa0 0x559c28b06e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa6b659c640 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b04aa0 0x559c28b06e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ce30f6d2a88f471 server_cookie=35c9e2bedb6402af in_seq=0 out_seq=0 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.612+0000 7fa69dffb640 10 client.25535 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa69dffb640 1 -- 172.21.15.73:0/2057078042 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c28a58f10 con 0x559c28b04aa0 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa69dffb640 10 client.25535 handle_command_reply: tid=0 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 client.25535 shutdown 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 msgr2=0x559c28af20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af20a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0cca90 tx=0x7fa6ac09b280 comp rx=0 tx=0).stop 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 shutdown_connections 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b04aa0 0x559c28b06e90 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28af4b60 0x559c28af25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28ae19d0 0x559c28af20a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/2057078042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 >> 172.21.15.73:0/2057078042 conn(0x559c28add680 msgr2=0x559c28ae3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa69dffb640 0 client.25535 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 shutdown_connections 2024-06-06T07:04:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/2057078042 wait complete. 2024-06-06T07:04:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.613+0000 7fa6bfbe7740 20 client.25535 trim_cache size 0 max 16384 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 Processor -- start 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 -- start start 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae19d0 0x559c28af0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28af0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81acaf0 con 0x559c28ae10b0 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81ae690 con 0x559c28af4b60 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81ad1c0 con 0x559c28ae19d0 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56412/0 (socket says 172.21.15.73:56412) 2024-06-06T07:04:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b659c640 1 -- 172.21.15.73:0/1895946546 learned_addr learned my addr 172.21.15.73:0/1895946546 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28af0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae19d0 0x559c28af0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b659c640 1 -- 172.21.15.73:0/1895946546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae19d0 msgr2=0x559c28af0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.630+0000 7fa6b659c640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae19d0 0x559c28af0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa6b659c640 1 -- 172.21.15.73:0/1895946546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 msgr2=0x559c28af0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa6b659c640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28af0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa6b659c640 1 -- 172.21.15.73:0/1895946546 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c28add3e0 con 0x559c28ae10b0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa6b6d9d640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28af0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa6b659c640 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8013cc0 tx=0x7fa6a8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2fd823736ed7375 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa69ffff640 1 -- 172.21.15.73:0/1895946546 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8004220 con 0x559c28ae10b0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa69ffff640 1 -- 172.21.15.73:0/1895946546 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6a801ddf0 con 0x559c28ae10b0 2024-06-06T07:04:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.631+0000 7fa69ffff640 1 -- 172.21.15.73:0/1895946546 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a8029470 con 0x559c28ae10b0 2024-06-06T07:04:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/1895946546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 msgr2=0x559c28af2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8013cc0 tx=0x7fa6a8016000 comp rx=0 tx=0).stop 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/1895946546 shutdown_connections 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28af0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae19d0 0x559c28af0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/1895946546 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae10b0 0x559c28af2400 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/1895946546 >> 172.21.15.73:0/1895946546 conn(0x559c28add680 msgr2=0x559c28aece80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/1895946546 shutdown_connections 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/1895946546 wait complete. 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 -- start start 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 Processor -- start 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28aef7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.632+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28afd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81adfc0 con 0x559c28ae19d0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81ad890 con 0x559c28af4b60 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa6b81aab10 con 0x559c28ae10b0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 10 client.0 _ll_register_callbacks cb 0x7fa6ac0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28aef7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28afd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56426/0 (socket says 172.21.15.73:56426) 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/3099316785 learned_addr learned my addr 172.21.15.73:0/3099316785 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b6d9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28afd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56358/0 (socket says 172.21.15.73:56358) 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 msgr2=0x559c28aef7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28aef7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/3099316785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 msgr2=0x559c28afd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28afd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 -- 172.21.15.73:0/3099316785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c28aede70 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6b5d9b640 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004fd0 tx=0x7fa6ac0cf2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd911e85ae948670 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa69dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ac0a44d0 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa6ac0a4040 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a801ddf0 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fa6ac0cf3d0 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 10 client.16110 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.633+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x559c28b6ea30 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6a801ddf0 con 0x559c28ae19d0 2024-06-06T07:04:33.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fa6ac081020 con 0x559c28ae19d0 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 10 client.16110 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 10 client.16110 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 20 client.16110 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b6f210 0x559c28b71630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 4 client.16110 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c28a52c20 con 0x559c28b6f210 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6b659c640 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b6f210 0x559c28b71630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa69dffb640 10 client.16110 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:33.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.634+0000 7fa6b659c640 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b6f210 0x559c28b71630 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=509305428631e5f8 server_cookie=4e659e882980c253 in_seq=0 out_seq=0 2024-06-06T07:04:33.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:33 smithi170 ceph-mon[36108]: pgmap v99: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa69dffb640 1 -- 172.21.15.73:0/3099316785 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559c28a52c20 con 0x559c28b6f210 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa69dffb640 10 client.16110 handle_command_reply: tid=0 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 client.16110 shutdown 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 msgr2=0x559c28afcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac004fd0 tx=0x7fa6ac0cf2e0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 shutdown_connections 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559c28b6f210 0x559c28b71630 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559c28af4b60 0x559c28afd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559c28ae19d0 0x559c28afcb10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.651 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa69dffb640 0 client.16110 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 --2- 172.21.15.73:0/3099316785 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559c28ae10b0 0x559c28aef7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.649+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 >> 172.21.15.73:0/3099316785 conn(0x559c28add680 msgr2=0x559c28af0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 shutdown_connections 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bfbe7740 1 -- 172.21.15.73:0/3099316785 wait complete. 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bfbe7740 20 client.16110 trim_cache size 0 max 16384 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:04:33.652 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:04:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 msgr2=0x7fa6a00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 0x7fa6a00a5260 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa6a8009c60 tx=0x7fa6a8005f90 comp rx=0 tx=0).stop 2024-06-06T07:04:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 msgr2=0x7fa6b8100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac00e9f0 tx=0x7fa6ac00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 shutdown_connections 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6b810bf70 0x7fa6b8101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa6b8103550 0x7fa6b8100dc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa6a00a2da0 0x7fa6a00a5260 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 --2- 172.21.15.73:0/4113054192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa6b8102c30 0x7fa6b8100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 >> 172.21.15.73:0/4113054192 conn(0x7fa6b8077a20 msgr2=0x7fa6b80fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 shutdown_connections 2024-06-06T07:04:33.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:33.650+0000 7fa6bcdf9640 1 -- 172.21.15.73:0/4113054192 wait complete. 2024-06-06T07:04:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:33 smithi073 ceph-mon[32303]: pgmap v99: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:33 smithi136 ceph-mon[35901]: pgmap v99: 129 pgs: 129 active+clean; 45 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-06T07:04:34.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 49413 2.81 MB/sec warmup 101 sec latency 16.485 ms 2024-06-06T07:04:35.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 49724 2.81 MB/sec warmup 102 sec latency 98.784 ms 2024-06-06T07:04:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:35 smithi073 ceph-mon[32303]: pgmap v100: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:04:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:35 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:04:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:35 smithi136 ceph-mon[35901]: pgmap v100: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:04:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:35 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:04:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:35 smithi170 ceph-mon[36108]: pgmap v100: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.7 MiB/s wr, 225 op/s 2024-06-06T07:04:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:35 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:04:36.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 50069 2.80 MB/sec warmup 103 sec latency 15.171 ms 2024-06-06T07:04:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:04:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:04:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:04:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:04:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:04:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:04:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:04:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:04:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:04:37.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 50374 2.78 MB/sec warmup 104 sec latency 15.494 ms 2024-06-06T07:04:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:37 smithi073 ceph-mon[32303]: pgmap v101: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:04:37.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:37 smithi136 ceph-mon[35901]: pgmap v101: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:04:37.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:37 smithi170 ceph-mon[36108]: pgmap v101: 129 pgs: 129 active+clean; 45 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:04:38.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 50877 2.77 MB/sec warmup 105 sec latency 11.478 ms 2024-06-06T07:04:39.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 51501 2.77 MB/sec warmup 106 sec latency 20.421 ms 2024-06-06T07:04:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:39 smithi073 ceph-mon[32303]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-06-06T07:04:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:39 smithi136 ceph-mon[35901]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-06-06T07:04:39.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:39 smithi170 ceph-mon[36108]: pgmap v102: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-06-06T07:04:40.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 52090 2.76 MB/sec warmup 107 sec latency 19.694 ms 2024-06-06T07:04:40.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:40 smithi170 ceph-mon[36108]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:41.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:40 smithi073 ceph-mon[32303]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:41.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:40 smithi136 ceph-mon[35901]: pgmap v103: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:41.435 INFO:tasks.workunit.client.0.smithi073.stdout: 1 52676 2.79 MB/sec warmup 108 sec latency 16.480 ms 2024-06-06T07:04:42.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 53189 2.81 MB/sec warmup 109 sec latency 11.928 ms 2024-06-06T07:04:43.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:42 smithi170 ceph-mon[36108]: pgmap v104: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:42 smithi073 ceph-mon[32303]: pgmap v104: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:42 smithi136 ceph-mon[35901]: pgmap v104: 129 pgs: 129 active+clean; 49 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 156 op/s 2024-06-06T07:04:43.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 53556 2.80 MB/sec warmup 110 sec latency 15.087 ms 2024-06-06T07:04:44.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 53846 2.77 MB/sec warmup 111 sec latency 15.570 ms 2024-06-06T07:04:45.075 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:44 smithi073 ceph-mon[32303]: pgmap v105: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:04:45.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:44 smithi170 ceph-mon[36108]: pgmap v105: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:04:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:44 smithi136 ceph-mon[35901]: pgmap v105: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.7 MiB/s wr, 231 op/s 2024-06-06T07:04:45.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 54293 2.77 MB/sec warmup 112 sec latency 59.370 ms 2024-06-06T07:04:46.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 54858 2.77 MB/sec warmup 113 sec latency 15.406 ms 2024-06-06T07:04:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 -- 172.21.15.73:0/876204147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 msgr2=0x7f495c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 --2- 172.21.15.73:0/876204147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c101070 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f494c009900 tx=0x7f494c02a410 comp rx=0 tx=0).stop 2024-06-06T07:04:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 -- 172.21.15.73:0/876204147 shutdown_connections 2024-06-06T07:04:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 --2- 172.21.15.73:0/876204147 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c1015b0 0x7f495c10d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 --2- 172.21.15.73:0/876204147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c101070 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 --2- 172.21.15.73:0/876204147 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c105840 0x7f495c105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 -- 172.21.15.73:0/876204147 >> 172.21.15.73:0/876204147 conn(0x7f495c0766a0 msgr2=0x7f495c076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 -- 172.21.15.73:0/876204147 shutdown_connections 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.761+0000 7f49639b7640 1 -- 172.21.15.73:0/876204147 wait complete. 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 Processor -- start 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 -- start start 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c1015b0 0x7f495c0fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 0x7f495c0ff170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f495c110b20 con 0x7f495c100bf0 2024-06-06T07:04:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1109a0 con 0x7f495c105840 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f49639b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f495c110ca0 con 0x7f495c1015b0 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47484/0 (socket says 172.21.15.73:47484) 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 -- 172.21.15.73:0/2200167109 learned_addr learned my addr 172.21.15.73:0/2200167109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4960f52640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c1015b0 0x7f495c0fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961f54640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 0x7f495c0ff170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 -- 172.21.15.73:0/2200167109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c1015b0 msgr2=0x7f495c0fec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c1015b0 0x7f495c0fec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 -- 172.21.15.73:0/2200167109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 msgr2=0x7f495c0ff170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 0x7f495c0ff170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961753640 1 -- 172.21.15.73:0/2200167109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f495c1a6570 con 0x7f495c100bf0 2024-06-06T07:04:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.762+0000 7f4961f54640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 0x7f495c0ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f4961753640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f495800e9f0 tx=0x7f495800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f49527fc640 1 -- 172.21.15.73:0/2200167109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800cd80 con 0x7f495c100bf0 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f49527fc640 1 -- 172.21.15.73:0/2200167109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49580054d0 con 0x7f495c100bf0 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f495c1a6860 con 0x7f495c100bf0 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f49527fc640 1 -- 172.21.15.73:0/2200167109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958010490 con 0x7f495c100bf0 2024-06-06T07:04:46.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.763+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f495c1a6d80 con 0x7f495c100bf0 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.765+0000 7f49527fc640 1 -- 172.21.15.73:0/2200167109 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4958005020 con 0x7f495c100bf0 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 Processor -- start 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 -- start start 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898bebc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 0x55898bec9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55898be204f0 con 0x55898bec7350 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55898bdbe870 con 0x55898bebc170 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49667a4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55898be1c4b0 con 0x55898bebca90 2024-06-06T07:04:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4961753640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4960f52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 0x55898bec9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4961753640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:37394/0 (socket says 172.21.15.73:37394) 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4960f52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 0x55898bec9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47494/0 (socket says 172.21.15.73:47494) 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4961753640 1 -- 172.21.15.73:0/3773138548 learned_addr learned my addr 172.21.15.73:0/3773138548 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f4961f54640 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898bebc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.766+0000 7f49527fc640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 0x7f49480a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961f54640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 0x7f49480a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 -- 172.21.15.73:0/3773138548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 msgr2=0x55898bebc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898bebc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 -- 172.21.15.73:0/3773138548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 msgr2=0x55898bec9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 0x55898bec9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 -- 172.21.15.73:0/3773138548 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898bdda9b0 con 0x55898bebca90 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f49527fc640 1 -- 172.21.15.73:0/2200167109 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f49580cfc40 con 0x7f495c100bf0 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961753640 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49580ccef0 tx=0x7f4958021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54cfefd36bd5eb3b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f4961f54640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 0x7f49480a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4954009fd0 tx=0x7f4954004ab0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f49477fe640 1 -- 172.21.15.73:0/3773138548 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958005310 con 0x55898bebca90 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f49477fe640 1 -- 172.21.15.73:0/3773138548 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4958005680 con 0x55898bebca90 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.767+0000 7f49477fe640 1 -- 172.21.15.73:0/3773138548 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958016df0 con 0x55898bebca90 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- 172.21.15.73:0/3773138548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 msgr2=0x55898bec6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f49580ccef0 tx=0x7f4958021000 comp rx=0 tx=0).stop 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- 172.21.15.73:0/3773138548 shutdown_connections 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bec7350 0x55898bec9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec6e10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 --2- 172.21.15.73:0/3773138548 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898bebc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- 172.21.15.73:0/3773138548 >> 172.21.15.73:0/3773138548 conn(0x55898beb8680 msgr2=0x55898beb8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- 172.21.15.73:0/3773138548 shutdown_connections 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- 172.21.15.73:0/3773138548 wait complete. 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.768+0000 7f49667a4740 1 -- start start 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 Processor -- start 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebca90 0x55898bec4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898bec4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55898bec9f20 con 0x55898bebc170 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55898bda9ae0 con 0x55898bebca90 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55898beca0a0 con 0x55898bec7350 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47506/0 (socket says 172.21.15.73:47506) 2024-06-06T07:04:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 -- 172.21.15.73:0/2487547758 learned_addr learned my addr 172.21.15.73:0/2487547758 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961f54640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898bec4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4960f52640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebca90 0x55898bec4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f49667a4740 10 client.0 _ll_register_callbacks cb 0x55898bde29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 -- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 msgr2=0x55898bec4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898bec4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 -- 172.21.15.73:0/2487547758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebca90 msgr2=0x55898bec4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebca90 0x55898bec4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961753640 1 -- 172.21.15.73:0/2487547758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898bb34210 con 0x55898bebc170 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.769+0000 7f4961f54640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898bec4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4961753640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4958098720 tx=0x7f49580ccf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204234f7b43e9a6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580cd400 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f495804f590 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4958005680 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f4958005310 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f49667a4740 10 client.16122 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55898bc00620 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4958005680 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f49580a45c0 con 0x55898bebc170 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f49667a4740 10 client.16122 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.770+0000 7f49667a4740 10 client.16122 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f49667a4740 20 client.16122 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bed70e0 0x55898bed94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f49667a4740 4 client.16122 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55898be33f10 con 0x55898bed70e0 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f4960f52640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bed70e0 0x55898bed94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f4944ff9640 10 client.16122 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.771+0000 7f4960f52640 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bed70e0 0x55898bed94d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b5d7a8f1ec8f52 server_cookie=a8ad10795178f22 in_seq=0 out_seq=0 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f4944ff9640 1 -- 172.21.15.73:0/2487547758 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55898be33f10 con 0x55898bed70e0 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f4944ff9640 10 client.16122 handle_command_reply: tid=0 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 client.16122 shutdown 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 msgr2=0x55898bec4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4958098720 tx=0x7f49580ccf60 comp rx=0 tx=0).stop 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 shutdown_connections 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bed70e0 0x55898bed94d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898bec4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebca90 0x55898bec4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 --2- 172.21.15.73:0/2487547758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bec4310 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 >> 172.21.15.73:0/2487547758 conn(0x55898beb8680 msgr2=0x55898bebe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f4944ff9640 0 client.16122 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 shutdown_connections 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.772+0000 7f49667a4740 1 -- 172.21.15.73:0/2487547758 wait complete. 2024-06-06T07:04:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.773+0000 7f49667a4740 20 client.16122 trim_cache size 0 max 16384 2024-06-06T07:04:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.790+0000 7f49667a4740 1 Processor -- start 2024-06-06T07:04:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.790+0000 7f49667a4740 1 -- start start 2024-06-06T07:04:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bec7350 0x55898bedeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1b0f20 con 0x55898bebc170 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1b2ac0 con 0x55898bec7350 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f49667a4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1b15f0 con 0x55898bebca90 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47510/0 (socket says 172.21.15.73:47510) 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 -- 172.21.15.73:0/4015548085 learned_addr learned my addr 172.21.15.73:0/4015548085 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4960f52640 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961f54640 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bec7350 0x55898bedeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 -- 172.21.15.73:0/4015548085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 msgr2=0x55898bec31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 -- 172.21.15.73:0/4015548085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bec7350 msgr2=0x55898bedeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bec7350 0x55898bedeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 -- 172.21.15.73:0/4015548085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55898beca010 con 0x55898bebc170 2024-06-06T07:04:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.791+0000 7f4961753640 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f49580981b0 tx=0x7f4958005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b834f17e5a3fa9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.792+0000 7f4946ffd640 1 -- 172.21.15.73:0/4015548085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580a8070 con 0x55898bebc170 2024-06-06T07:04:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.792+0000 7f4946ffd640 1 -- 172.21.15.73:0/4015548085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49580cd400 con 0x55898bebc170 2024-06-06T07:04:46.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.792+0000 7f4946ffd640 1 -- 172.21.15.73:0/4015548085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49580a0100 con 0x55898bebc170 2024-06-06T07:04:46.797 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- 172.21.15.73:0/4015548085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 msgr2=0x55898bebeb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f49580981b0 tx=0x7f4958005ab0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- 172.21.15.73:0/4015548085 shutdown_connections 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bec7350 0x55898bedeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bebca90 0x55898bec31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 --2- 172.21.15.73:0/4015548085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebc170 0x55898bebeb30 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- 172.21.15.73:0/4015548085 >> 172.21.15.73:0/4015548085 conn(0x55898beb8680 msgr2=0x55898bec2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- 172.21.15.73:0/4015548085 shutdown_connections 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- 172.21.15.73:0/4015548085 wait complete. 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 -- start start 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.793+0000 7f49667a4740 1 Processor -- start 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898beca2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.798 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898becad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1b23f0 con 0x55898bebca90 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1b1cc0 con 0x55898bebc170 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f495c1aef00 con 0x55898bec7350 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f49667a4740 10 client.0 _ll_register_callbacks cb 0x7f4958005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47526/0 (socket says 172.21.15.73:47526) 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 -- 172.21.15.73:0/1097860429 learned_addr learned my addr 172.21.15.73:0/1097860429 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4961f54640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898becad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4961753640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898beca2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 -- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 msgr2=0x55898becad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898becad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 -- 172.21.15.73:0/1097860429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 msgr2=0x55898beca2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898beca2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 -- 172.21.15.73:0/1097860429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55898bf482d0 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4960f52640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f494c005c40 tx=0x7f494c0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73139288c7a634e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.794+0000 7f4944ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c035040 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f494c005040 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49580cd400 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f494c054050 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 10 client.16128 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55898bf495f0 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f49580cd400 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f494c004cc0 con 0x55898bebca90 2024-06-06T07:04:46.800 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 10 client.16128 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 20 client.16128 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bf49e00 0x55898bf4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 4 client.16128 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55898be2dc20 con 0x55898bf49e00 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.795+0000 7f4961753640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bf49e00 0x55898bf4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.796+0000 7f4961753640 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bf49e00 0x55898bf4c220 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8485472601f15676 server_cookie=a8cbcd3c14f1c1ad in_seq=0 out_seq=0 2024-06-06T07:04:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.796+0000 7f4944ff9640 10 client.16128 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f4944ff9640 1 -- 172.21.15.73:0/1097860429 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55898be2dc20 con 0x55898bf49e00 2024-06-06T07:04:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f4944ff9640 10 client.16128 handle_command_reply: tid=0 2024-06-06T07:04:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 client.16128 shutdown 2024-06-06T07:04:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 msgr2=0x55898beca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f494c005c40 tx=0x7f494c0095d0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 shutdown_connections 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55898bf49e00 0x55898bf4c220 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55898bec7350 0x55898becad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55898bebca90 0x55898beca7f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 --2- 172.21.15.73:0/1097860429 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55898bebc170 0x55898beca2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f4944ff9640 0 client.16128 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 >> 172.21.15.73:0/1097860429 conn(0x55898beb8680 msgr2=0x55898bec3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 shutdown_connections 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 1 -- 172.21.15.73:0/1097860429 wait complete. 2024-06-06T07:04:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.820+0000 7f49667a4740 20 client.16128 trim_cache size 0 max 16384 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 msgr2=0x7f49480a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 0x7f49480a52e0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4954009fd0 tx=0x7f4954004ab0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 msgr2=0x7f495c0fe6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f495800e9f0 tx=0x7f495800edf0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 shutdown_connections 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f495c105840 0x7f495c0ff170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49480a2e20 0x7f49480a52e0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f495c1015b0 0x7f495c0fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 --2- 172.21.15.73:0/2200167109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c100bf0 0x7f495c0fe6f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 >> 172.21.15.73:0/2200167109 conn(0x7f495c0766a0 msgr2=0x7f495c06a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 shutdown_connections 2024-06-06T07:04:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:46.821+0000 7f49639b7640 1 -- 172.21.15.73:0/2200167109 wait complete. 2024-06-06T07:04:47.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:46 smithi170 ceph-mon[36108]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-06T07:04:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:46 smithi073 ceph-mon[32303]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-06T07:04:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:46 smithi136 ceph-mon[35901]: pgmap v106: 129 pgs: 129 active+clean; 50 MiB data, 632 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-06T07:04:47.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 55507 2.75 MB/sec warmup 114 sec latency 16.832 ms 2024-06-06T07:04:48.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 56012 2.77 MB/sec warmup 115 sec latency 16.020 ms 2024-06-06T07:04:49.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 56566 2.78 MB/sec warmup 116 sec latency 15.441 ms 2024-06-06T07:04:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:49 smithi073 ceph-mon[32303]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-06-06T07:04:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:49 smithi136 ceph-mon[35901]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-06-06T07:04:49.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:49 smithi170 ceph-mon[36108]: pgmap v107: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 230 op/s 2024-06-06T07:04:50.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 57020 2.79 MB/sec warmup 117 sec latency 15.788 ms 2024-06-06T07:04:51.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 57286 2.77 MB/sec warmup 118 sec latency 15.727 ms 2024-06-06T07:04:51.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:51 smithi073 ceph-mon[32303]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:51 smithi136 ceph-mon[35901]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:51 smithi170 ceph-mon[36108]: pgmap v108: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:52.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 57724 2.75 MB/sec warmup 119 sec latency 14.962 ms 2024-06-06T07:04:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:53 smithi073 ceph-mon[32303]: pgmap v109: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:53 smithi136 ceph-mon[35901]: pgmap v109: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:53.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:53 smithi170 ceph-mon[36108]: pgmap v109: 129 pgs: 129 active+clean; 46 MiB data, 695 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.1 MiB/s wr, 151 op/s 2024-06-06T07:04:54.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 58921 0.64 MB/sec execute 1 sec latency 20.214 ms 2024-06-06T07:04:55.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 59394 1.52 MB/sec execute 2 sec latency 15.135 ms 2024-06-06T07:04:55.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:55 smithi073 ceph-mon[32303]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 227 op/s 2024-06-06T07:04:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:55 smithi136 ceph-mon[35901]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 227 op/s 2024-06-06T07:04:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:55 smithi170 ceph-mon[36108]: pgmap v110: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 227 op/s 2024-06-06T07:04:56.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 60092 3.11 MB/sec execute 3 sec latency 10.603 ms 2024-06-06T07:04:57.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 60557 3.43 MB/sec execute 4 sec latency 12.401 ms 2024-06-06T07:04:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:57 smithi073 ceph-mon[32303]: pgmap v111: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:57 smithi136 ceph-mon[35901]: pgmap v111: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:57.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:57 smithi170 ceph-mon[36108]: pgmap v111: 129 pgs: 129 active+clean; 51 MiB data, 700 MiB used, 1.0 TiB / 1.0 TiB avail; 966 KiB/s rd, 1.2 MiB/s wr, 152 op/s 2024-06-06T07:04:58.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 60826 2.81 MB/sec execute 5 sec latency 11.947 ms 2024-06-06T07:04:59.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 61275 2.48 MB/sec execute 6 sec latency 15.541 ms 2024-06-06T07:04:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:04:59 smithi073 ceph-mon[32303]: pgmap v112: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2024-06-06T07:04:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:04:59 smithi136 ceph-mon[35901]: pgmap v112: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2024-06-06T07:04:59.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:04:59 smithi170 ceph-mon[36108]: pgmap v112: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 231 op/s 2024-06-06T07:04:59.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 -- 172.21.15.73:0/2599279769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c101290 msgr2=0x7ff33c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 --2- 172.21.15.73:0/2599279769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c101290 0x7ff33c10d750 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff334009cd0 tx=0x7ff33402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 -- 172.21.15.73:0/2599279769 shutdown_connections 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 --2- 172.21.15.73:0/2599279769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c101290 0x7ff33c10d750 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 --2- 172.21.15.73:0/2599279769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 --2- 172.21.15.73:0/2599279769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c106890 0x7ff33c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.917+0000 7ff34342c640 1 -- 172.21.15.73:0/2599279769 >> 172.21.15.73:0/2599279769 conn(0x7ff33c077a20 msgr2=0x7ff33c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- 172.21.15.73:0/2599279769 shutdown_connections 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- 172.21.15.73:0/2599279769 wait complete. 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 Processor -- start 2024-06-06T07:04:59.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- start start 2024-06-06T07:04:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c101290 0x7ff33c199470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c110b10 con 0x7ff33c106890 2024-06-06T07:04:59.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c110990 con 0x7ff33c1008d0 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.918+0000 7ff34342c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c110c90 con 0x7ff33c101290 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:50586/0 (socket says 172.21.15.73:50586) 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55892/0 (socket says 172.21.15.73:55892) 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3411c8640 1 -- 172.21.15.73:0/3706421272 learned_addr learned my addr 172.21.15.73:0/3706421272 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3409c7640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c101290 0x7ff33c199470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 -- 172.21.15.73:0/3706421272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c101290 msgr2=0x7ff33c199470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c101290 0x7ff33c199470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 -- 172.21.15.73:0/3706421272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 msgr2=0x7ff33c198f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 -- 172.21.15.73:0/3706421272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33c1a4330 con 0x7ff33c106890 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3411c8640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3419c9640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff33402ae00 tx=0x7ff334031dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3327fc640 1 -- 172.21.15.73:0/3706421272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334004040 con 0x7ff33c106890 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33c1a45c0 con 0x7ff33c106890 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3327fc640 1 -- 172.21.15.73:0/3706421272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff334004690 con 0x7ff33c106890 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff3327fc640 1 -- 172.21.15.73:0/3706421272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff334030d30 con 0x7ff33c106890 2024-06-06T07:04:59.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.919+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff33c1a4b10 con 0x7ff33c106890 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.921+0000 7ff3327fc640 1 -- 172.21.15.73:0/3706421272 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff3340305a0 con 0x7ff33c106890 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 Processor -- start 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 -- start start 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff3327fc640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 0x7ff3280a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170240b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f1701934f0 con 0x55f17022f050 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f170131870 con 0x55f17023e760 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff346210740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f17018f4b0 con 0x55f17022f970 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.922+0000 7ff3409c7640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 0x7ff3280a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3327fc640 1 -- 172.21.15.73:0/3706421272 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff33402e030 con 0x7ff33c106890 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55906/0 (socket says 172.21.15.73:55906) 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 -- 172.21.15.73:0/2184505613 learned_addr learned my addr 172.21.15.73:0/2184505613 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3409c7640 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3419c9640 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170240b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 -- 172.21.15.73:0/2184505613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 msgr2=0x55f17023e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 -- 172.21.15.73:0/2184505613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 msgr2=0x55f170240b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170240b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 -- 172.21.15.73:0/2184505613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17014d9b0 con 0x55f17022f050 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3409c7640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 0x7ff3280a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff33c1053a0 tx=0x7ff32c009ab0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3411c8640 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff3380115f0 tx=0x7ff338011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49e15172a7d074fa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3277fe640 1 -- 172.21.15.73:0/2184505613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33801e070 con 0x55f17022f050 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3277fe640 1 -- 172.21.15.73:0/2184505613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33800fa90 con 0x55f17022f050 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.923+0000 7ff3277fe640 1 -- 172.21.15.73:0/2184505613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338019ae0 con 0x55f17022f050 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- 172.21.15.73:0/2184505613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 msgr2=0x55f17022f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff3380115f0 tx=0x7ff338011ab0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- 172.21.15.73:0/2184505613 shutdown_connections 2024-06-06T07:04:59.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170240b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 --2- 172.21.15.73:0/2184505613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17022f430 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- 172.21.15.73:0/2184505613 >> 172.21.15.73:0/2184505613 conn(0x55f17022b680 msgr2=0x55f17022ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- 172.21.15.73:0/2184505613 shutdown_connections 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- 172.21.15.73:0/2184505613 wait complete. 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 -- start start 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.924+0000 7ff346210740 1 Processor -- start 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 0x55f17023bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f17023c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f170241390 con 0x55f17022f050 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f17011cae0 con 0x55f17022f970 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f170241510 con 0x55f17023e760 2024-06-06T07:04:59.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff346210740 10 client.0 _ll_register_callbacks cb 0x55f1701559b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55912/0 (socket says 172.21.15.73:55912) 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 -- 172.21.15.73:0/4228930157 learned_addr learned my addr 172.21.15.73:0/4228930157 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3419c9640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f17023c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3409c7640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 0x55f17023bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 -- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 msgr2=0x55f17023c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f17023c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 -- 172.21.15.73:0/4228930157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 msgr2=0x55f17023bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 0x55f17023bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 -- 172.21.15.73:0/4228930157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f16fea7210 con 0x55f17022f050 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3419c9640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f17023c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3409c7640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 0x55f17023bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:59.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff3411c8640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff33801c040 tx=0x7ff33800f890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4d16ac14e2e20a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.925+0000 7ff324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33801e040 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff338005050 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33800fa90 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff33802f050 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 10 client.16140 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f16ff73620 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33800fa90 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff33802a030 con 0x55f17022f050 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 10 client.16140 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 10 client.16140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 20 client.16140 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:59.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f17024e920 0x55f170250d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 4 client.16140 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1701a6f10 con 0x55f17024e920 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.926+0000 7ff3409c7640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f17024e920 0x55f170250d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.927+0000 7ff324ff9640 10 client.16140 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.927+0000 7ff3409c7640 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f17024e920 0x55f170250d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ed7acedc7d72eae server_cookie=ca64cf879759b3c3 in_seq=0 out_seq=0 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.927+0000 7ff324ff9640 1 -- 172.21.15.73:0/4228930157 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f1701a6f10 con 0x55f17024e920 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.927+0000 7ff324ff9640 10 client.16140 handle_command_reply: tid=0 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 client.16140 shutdown 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 msgr2=0x55f17023b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff33801c040 tx=0x7ff33800f890 comp rx=0 tx=0).stop 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 shutdown_connections 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f17024e920 0x55f170250d10 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f17023c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f970 0x55f17023bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 --2- 172.21.15.73:0/4228930157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f17023b710 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 >> 172.21.15.73:0/4228930157 conn(0x55f17022b680 msgr2=0x55f1702314f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff324ff9640 0 client.16140 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 shutdown_connections 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 1 -- 172.21.15.73:0/4228930157 wait complete. 2024-06-06T07:04:59.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.928+0000 7ff346210740 20 client.16140 trim_cache size 0 max 16384 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 Processor -- start 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 -- start start 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170256510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ac9f0 con 0x55f17022f050 2024-06-06T07:04:59.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ae590 con 0x55f17023e760 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.946+0000 7ff346210740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ad0c0 con 0x55f17022f970 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55926/0 (socket says 172.21.15.73:55926) 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 -- 172.21.15.73:0/1044101446 learned_addr learned my addr 172.21.15.73:0/1044101446 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3409c7640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3419c9640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170256510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 -- 172.21.15.73:0/1044101446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 msgr2=0x55f17023a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 -- 172.21.15.73:0/1044101446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 msgr2=0x55f170256510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170256510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 -- 172.21.15.73:0/1044101446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f17022f7a0 con 0x55f17022f050 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3419c9640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170256510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:04:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff338009c90 tx=0x7ff338005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab53cc4b67e260f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff326ffd640 1 -- 172.21.15.73:0/1044101446 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338015020 con 0x55f17022f050 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff326ffd640 1 -- 172.21.15.73:0/1044101446 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33800f900 con 0x55f17022f050 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.947+0000 7ff326ffd640 1 -- 172.21.15.73:0/1044101446 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff33801e070 con 0x55f17022f050 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 -- 172.21.15.73:0/1044101446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 msgr2=0x55f170239fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff338009c90 tx=0x7ff338005970 comp rx=0 tx=0).stop 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 -- 172.21.15.73:0/1044101446 shutdown_connections 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17023e760 0x55f170256510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17022f970 0x55f17023a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 --2- 172.21.15.73:0/1044101446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f050 0x55f170239fe0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 -- 172.21.15.73:0/1044101446 >> 172.21.15.73:0/1044101446 conn(0x55f17022b680 msgr2=0x55f1702369a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 -- 172.21.15.73:0/1044101446 shutdown_connections 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.948+0000 7ff346210740 1 -- 172.21.15.73:0/1044101446 wait complete. 2024-06-06T07:04:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 -- start start 2024-06-06T07:04:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 Processor -- start 2024-06-06T07:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 0x55f170247050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f170241960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1adec0 con 0x55f17022f970 2024-06-06T07:04:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1ad790 con 0x55f17022f050 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff33c1aa9d0 con 0x55f17023e760 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff346210740 10 client.0 _ll_register_callbacks cb 0x7ff33801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff3411c8640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:50612/0 (socket says 172.21.15.73:50612) 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff3411c8640 1 -- 172.21.15.73:0/1998567637 learned_addr learned my addr 172.21.15.73:0/1998567637 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff3409c7640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 0x55f170247050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.949+0000 7ff3419c9640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f170241960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 -- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 msgr2=0x55f170241960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f170241960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 -- 172.21.15.73:0/1998567637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 msgr2=0x55f170247050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 0x55f170247050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 -- 172.21.15.73:0/1998567637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f170242160 con 0x55f17022f050 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3409c7640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 0x55f170247050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:04:59.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff3411c8640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff338019f70 tx=0x7ff338004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce49662b3082ed9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff324ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff338015030 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff33800f900 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7ff3380199a0 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7ff338045050 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff346210740 10 client.25571 fetch_fsmap learned FSMap version 34 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.950+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55f1702b8040 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff33800f900 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7ff33802a040 con 0x55f17022f050 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 10 client.25571 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 10 client.25571 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 20 client.25571 populate_metadata read hostname 'smithi073' 2024-06-06T07:04:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f1702b8730 0x55f1702bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:04:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 4 client.25571 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:04:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f1701a0c20 con 0x55f1702b8730 2024-06-06T07:04:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff3409c7640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f1702b8730 0x55f1702bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:04:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff324ff9640 10 client.25571 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.951+0000 7ff3409c7640 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f1702b8730 0x55f1702bab50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=336f6b776233ef1a server_cookie=3143dd30f7ceb8ff in_seq=0 out_seq=0 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff324ff9640 1 -- 172.21.15.73:0/1998567637 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f1701a0c20 con 0x55f1702b8730 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff324ff9640 10 client.25571 handle_command_reply: tid=0 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 client.25571 shutdown 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 msgr2=0x55f170246b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff338019f70 tx=0x7ff338004a50 comp rx=0 tx=0).stop 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 shutdown_connections 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f1702b8730 0x55f1702bab50 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f17023e760 0x55f170241960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f17022f970 0x55f170247050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 --2- 172.21.15.73:0/1998567637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f17022f050 0x55f170246b10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff324ff9640 0 client.25571 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.970+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 >> 172.21.15.73:0/1998567637 conn(0x55f17022b680 msgr2=0x55f170239fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 shutdown_connections 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff346210740 1 -- 172.21.15.73:0/1998567637 wait complete. 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff346210740 20 client.25571 trim_cache size 0 max 16384 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:04:59.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 msgr2=0x7ff3280a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 0x7ff3280a5260 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff33c1053a0 tx=0x7ff32c009ab0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 msgr2=0x7ff33c1999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff33402ae00 tx=0x7ff334031dd0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 shutdown_connections 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff33c106890 0x7ff33c1999f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3280a2da0 0x7ff3280a5260 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff33c101290 0x7ff33c199470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 --2- 172.21.15.73:0/3706421272 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff33c1008d0 0x7ff33c198f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:04:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 >> 172.21.15.73:0/3706421272 conn(0x7ff33c077a20 msgr2=0x7ff33c103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:04:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 shutdown_connections 2024-06-06T07:04:59.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:04:59.971+0000 7ff34342c640 1 -- 172.21.15.73:0/3706421272 wait complete. 2024-06-06T07:05:00.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 61799 2.73 MB/sec execute 7 sec latency 16.588 ms 2024-06-06T07:05:01.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 62414 2.47 MB/sec execute 8 sec latency 25.720 ms 2024-06-06T07:05:01.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:01 smithi073 ceph-mon[32303]: pgmap v113: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:01 smithi136 ceph-mon[35901]: pgmap v113: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:01 smithi170 ceph-mon[36108]: pgmap v113: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:02.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 62921 2.45 MB/sec execute 9 sec latency 15.317 ms 2024-06-06T07:05:03.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 63628 2.84 MB/sec execute 10 sec latency 14.983 ms 2024-06-06T07:05:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:03 smithi073 ceph-mon[32303]: pgmap v114: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:03 smithi136 ceph-mon[35901]: pgmap v114: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:03.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:03 smithi170 ceph-mon[36108]: pgmap v114: 129 pgs: 129 active+clean; 51 MiB data, 697 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-06T07:05:04.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 64063 2.94 MB/sec execute 11 sec latency 17.104 ms 2024-06-06T07:05:05.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 64320 2.76 MB/sec execute 12 sec latency 50.579 ms 2024-06-06T07:05:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:05 smithi073 ceph-mon[32303]: pgmap v115: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-06-06T07:05:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:05 smithi136 ceph-mon[35901]: pgmap v115: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-06-06T07:05:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:05 smithi170 ceph-mon[36108]: pgmap v115: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 245 op/s 2024-06-06T07:05:06.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 64731 2.60 MB/sec execute 13 sec latency 15.234 ms 2024-06-06T07:05:07.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 65166 2.53 MB/sec execute 14 sec latency 15.558 ms 2024-06-06T07:05:07.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:07 smithi073 ceph-mon[32303]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-06T07:05:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:07 smithi136 ceph-mon[35901]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-06T07:05:07.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:07 smithi170 ceph-mon[36108]: pgmap v116: 129 pgs: 129 active+clean; 53 MiB data, 758 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.1 MiB/s wr, 170 op/s 2024-06-06T07:05:08.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 65827 2.58 MB/sec execute 15 sec latency 24.585 ms 2024-06-06T07:05:09.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 66356 2.56 MB/sec execute 16 sec latency 15.421 ms 2024-06-06T07:05:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:09 smithi073 ceph-mon[32303]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-06-06T07:05:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:09 smithi136 ceph-mon[35901]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-06-06T07:05:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:09 smithi170 ceph-mon[36108]: pgmap v117: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.8 MiB/s wr, 249 op/s 2024-06-06T07:05:10.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 67072 2.73 MB/sec execute 17 sec latency 86.989 ms 2024-06-06T07:05:11.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 67438 2.81 MB/sec execute 18 sec latency 16.816 ms 2024-06-06T07:05:11.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:11 smithi073 ceph-mon[32303]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:11 smithi136 ceph-mon[35901]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:11.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:11 smithi170 ceph-mon[36108]: pgmap v118: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:12.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 67796 2.75 MB/sec execute 19 sec latency 15.692 ms 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- 172.21.15.73:0/1213029940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70101290 msgr2=0x7fdb7010d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 --2- 172.21.15.73:0/1213029940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70101290 0x7fdb7010d750 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009cd0 tx=0x7fdb6802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- 172.21.15.73:0/1213029940 shutdown_connections 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 --2- 172.21.15.73:0/1213029940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70101290 0x7fdb7010d750 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 --2- 172.21.15.73:0/1213029940 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb701008d0 0x7fdb70100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 --2- 172.21.15.73:0/1213029940 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb70106890 0x7fdb70106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- 172.21.15.73:0/1213029940 >> 172.21.15.73:0/1213029940 conn(0x7fdb70077a20 msgr2=0x7fdb70077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- 172.21.15.73:0/1213029940 shutdown_connections 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- 172.21.15.73:0/1213029940 wait complete. 2024-06-06T07:05:13.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 Processor -- start 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.068+0000 7fdb76be3640 1 -- start start 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb701008d0 0x7fdb70197010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb70101290 0x7fdb70197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70110b50 con 0x7fdb70106890 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701109d0 con 0x7fdb701008d0 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70110cd0 con 0x7fdb70101290 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40110/0 (socket says 172.21.15.73:40110) 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 -- 172.21.15.73:0/2436773712 learned_addr learned my addr 172.21.15.73:0/2436773712 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb70101290 0x7fdb70197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb7497f640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb701008d0 0x7fdb70197010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 -- 172.21.15.73:0/2436773712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb70101290 msgr2=0x7fdb70197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb70101290 0x7fdb70197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 -- 172.21.15.73:0/2436773712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb701008d0 msgr2=0x7fdb70197010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb701008d0 0x7fdb70197010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 -- 172.21.15.73:0/2436773712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb7019bfb0 con 0x7fdb70106890 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb75180640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdb68033040 tx=0x7fdb68009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb6dffb640 1 -- 172.21.15.73:0/2436773712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68004040 con 0x7fdb70106890 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb6dffb640 1 -- 172.21.15.73:0/2436773712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb68004690 con 0x7fdb70106890 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb6dffb640 1 -- 172.21.15.73:0/2436773712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb68030da0 con 0x7fdb70106890 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.069+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb7019c240 con 0x7fdb70106890 2024-06-06T07:05:13.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.070+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb701a3b30 con 0x7fdb70106890 2024-06-06T07:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb6dffb640 1 -- 172.21.15.73:0/2436773712 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb680041e0 con 0x7fdb70106890 2024-06-06T07:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 Processor -- start 2024-06-06T07:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 -- start start 2024-06-06T07:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1f250 0x562a27d22b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d235b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a27c7d4f0 con 0x562a27d1e8a0 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a27c1b870 con 0x562a27d1f250 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.072+0000 7fdb799cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a27c794b0 con 0x562a27d23130 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40126/0 (socket says 172.21.15.73:40126) 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 -- 172.21.15.73:0/924984155 learned_addr learned my addr 172.21.15.73:0/924984155 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb7497f640 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1f250 0x562a27d22b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d235b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6dffb640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 0x7fdb600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 -- 172.21.15.73:0/924984155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 msgr2=0x562a27d235b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d235b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 -- 172.21.15.73:0/924984155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1f250 msgr2=0x562a27d22b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1f250 0x562a27d22b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 -- 172.21.15.73:0/924984155 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a27c379b0 con 0x562a27d1e8a0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb75180640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 0x7fdb600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x562a27d23af0 tx=0x7fdb5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=714e2bcf410594f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb6dffb640 1 -- 172.21.15.73:0/2436773712 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb6802e030 con 0x7fdb70106890 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb5affd640 1 -- 172.21.15.73:0/924984155 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c013e50 con 0x562a27d1e8a0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb5affd640 1 -- 172.21.15.73:0/924984155 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5c013120 con 0x562a27d1e8a0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.073+0000 7fdb5affd640 1 -- 172.21.15.73:0/924984155 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c00fea0 con 0x562a27d1e8a0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- 172.21.15.73:0/924984155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 msgr2=0x562a27d1ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb75180640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 0x7fdb600a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb680edc30 tx=0x7fdb68003f10 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x562a27d23af0 tx=0x7fdb5c009780 comp rx=0 tx=0).stop 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- 172.21.15.73:0/924984155 shutdown_connections 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d235b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1f250 0x562a27d22b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 --2- 172.21.15.73:0/924984155 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1e8a0 0x562a27d1ec80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- 172.21.15.73:0/924984155 >> 172.21.15.73:0/924984155 conn(0x562a27d15170 msgr2=0x562a27d15550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- 172.21.15.73:0/924984155 shutdown_connections 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- 172.21.15.73:0/924984155 wait complete. 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.074+0000 7fdb799cd740 1 -- start start 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 Processor -- start 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d27df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d28330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d28870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a27d2b2c0 con 0x562a27d23130 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a27c06ae0 con 0x562a27d1e8a0 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a27d2b440 con 0x562a27d1f250 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb799cd740 10 client.0 _ll_register_callbacks cb 0x562a27c3f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d28870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb7497f640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d28330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d28870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40138/0 (socket says 172.21.15.73:40138) 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb6ffff640 1 -- 172.21.15.73:0/1901813771 learned_addr learned my addr 172.21.15.73:0/1901813771 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.075+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d27df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 -- 172.21.15.73:0/1901813771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 msgr2=0x562a27d27df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d27df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 -- 172.21.15.73:0/1901813771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 msgr2=0x562a27d28870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d28870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 -- 172.21.15.73:0/1901813771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a27991210 con 0x562a27d1f250 2024-06-06T07:05:13.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d28330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012700 tx=0x7fdb64012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e532f3a3537abb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64004030 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb640041d0 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb5c013120 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb640043d0 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb799cd740 10 client.25696 fetch_fsmap learned FSMap version 34 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562a27a5d620 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.076+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb5c013120 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdb640039c0 con 0x562a27d1f250 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 10 client.25696 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 10 client.25696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 20 client.25696 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27d389f0 0x562a27d3ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 4 client.25696 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:05:13.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a27c90f10 con 0x562a27d389f0 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27d389f0 0x562a27d3ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27d389f0 0x562a27d3ae70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1eea1f2cbdd9112 server_cookie=5f1baed1a87766dc in_seq=0 out_seq=0 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.077+0000 7fdb4ffff640 10 client.25696 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb4ffff640 1 -- 172.21.15.73:0/1901813771 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a27c90f10 con 0x562a27d389f0 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb4ffff640 10 client.25696 handle_command_reply: tid=0 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 client.25696 shutdown 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 msgr2=0x562a27d28330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d28330 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdb64012700 tx=0x7fdb64012bc0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 shutdown_connections 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27d389f0 0x562a27d3ae70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d28870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d28330 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1901813771 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d27df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb4ffff640 0 client.25696 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:13.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 >> 172.21.15.73:0/1901813771 conn(0x562a27d15170 msgr2=0x562a27d24500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 shutdown_connections 2024-06-06T07:05:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 1 -- 172.21.15.73:0/1901813771 wait complete. 2024-06-06T07:05:13.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.078+0000 7fdb799cd740 20 client.25696 trim_cache size 0 max 16384 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 Processor -- start 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 -- start start 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d14660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d29620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701acac0 con 0x562a27d1f250 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701ae660 con 0x562a27d1e8a0 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb799cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701ad190 con 0x562a27d23130 2024-06-06T07:05:13.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb7497f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb7497f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40148/0 (socket says 172.21.15.73:40148) 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb7497f640 1 -- 172.21.15.73:0/1889243581 learned_addr learned my addr 172.21.15.73:0/1889243581 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d14660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.096+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d29620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 -- 172.21.15.73:0/1889243581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 msgr2=0x562a27d29620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d29620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 -- 172.21.15.73:0/1889243581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 msgr2=0x562a27d14660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d14660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 -- 172.21.15.73:0/1889243581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a27d25dd0 con 0x562a27d1f250 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb7497f640 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb64000c00 tx=0x7fdb64005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c462f7156b531a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb5a7fc640 1 -- 172.21.15.73:0/1889243581 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb64009dc0 con 0x562a27d1f250 2024-06-06T07:05:13.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb5a7fc640 1 -- 172.21.15.73:0/1889243581 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb6400fdd0 con 0x562a27d1f250 2024-06-06T07:05:13.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.097+0000 7fdb5a7fc640 1 -- 172.21.15.73:0/1889243581 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb6401c880 con 0x562a27d1f250 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- 172.21.15.73:0/1889243581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 msgr2=0x562a27d207c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fdb64000c00 tx=0x7fdb64005850 comp rx=0 tx=0).stop 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- 172.21.15.73:0/1889243581 shutdown_connections 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d23130 0x562a27d29620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d1f250 0x562a27d207c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 --2- 172.21.15.73:0/1889243581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d14660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- 172.21.15.73:0/1889243581 >> 172.21.15.73:0/1889243581 conn(0x562a27d15170 msgr2=0x562a27d13f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- 172.21.15.73:0/1889243581 shutdown_connections 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- 172.21.15.73:0/1889243581 wait complete. 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 -- start start 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.098+0000 7fdb799cd740 1 Processor -- start 2024-06-06T07:05:13.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d30ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d3fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701adf90 con 0x562a27d23130 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701ad860 con 0x562a27d1e8a0 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb701aaaa0 con 0x562a27d1f250 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 10 client.0 _ll_register_callbacks cb 0x7fdb64009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40152/0 (socket says 172.21.15.73:40152) 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 -- 172.21.15.73:0/938807063 learned_addr learned my addr 172.21.15.73:0/938807063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb7497f640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d3fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d30ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 -- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 msgr2=0x562a27d3fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d3fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 -- 172.21.15.73:0/938807063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 msgr2=0x562a27d30ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d30ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 -- 172.21.15.73:0/938807063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a27d40910 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb6ffff640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0056a0 tx=0x7fdb5c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6d3a4e3a69ca8d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c035070 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fdb5c030e80 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb6400fdd0 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.099+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fdb5c00fc70 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 10 client.16164 fetch_fsmap learned FSMap version 34 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x562a27da34d0 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb6400fdd0 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7fdb5c039020 con 0x562a27d23130 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 10 client.16164 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:05:13.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 10 client.16164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 20 client.16164 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27da38f0 0x562a27da5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 4 client.16164 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a27c8ac20 con 0x562a27da38f0 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb7497f640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27da38f0 0x562a27da5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb4ffff640 10 client.16164 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:13.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.100+0000 7fdb7497f640 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27da38f0 0x562a27da5ce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=426fa8c5d7a52455 server_cookie=3956ded48723e50a in_seq=0 out_seq=0 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.121+0000 7fdb4ffff640 1 -- 172.21.15.73:0/938807063 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a27c8ac20 con 0x562a27da38f0 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.121+0000 7fdb4ffff640 10 client.16164 handle_command_reply: tid=0 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.121+0000 7fdb799cd740 1 client.16164 shutdown 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 msgr2=0x562a27d40350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0056a0 tx=0x7fdb5c009780 comp rx=0 tx=0).stop 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 shutdown_connections 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a27da38f0 0x562a27da5ce0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a27d23130 0x562a27d40350 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb4ffff640 0 client.16164 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a27d1f250 0x562a27d3fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 --2- 172.21.15.73:0/938807063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a27d1e8a0 0x562a27d30ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 >> 172.21.15.73:0/938807063 conn(0x562a27d15170 msgr2=0x562a27d147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 shutdown_connections 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 1 -- 172.21.15.73:0/938807063 wait complete. 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.122+0000 7fdb799cd740 20 client.16164 trim_cache size 0 max 16384 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:05:13.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 msgr2=0x7fdb600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 0x7fdb600a5260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fdb680edc30 tx=0x7fdb68003f10 comp rx=0 tx=0).stop 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 msgr2=0x7fdb7019b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fdb68033040 tx=0x7fdb68009360 comp rx=0 tx=0).stop 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 shutdown_connections 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb70106890 0x7fdb7019b880 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb600a2da0 0x7fdb600a5260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb70101290 0x7fdb70197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 --2- 172.21.15.73:0/2436773712 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb701008d0 0x7fdb70197010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 >> 172.21.15.73:0/2436773712 conn(0x7fdb70077a20 msgr2=0x7fdb701033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 shutdown_connections 2024-06-06T07:05:13.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:13.123+0000 7fdb76be3640 1 -- 172.21.15.73:0/2436773712 wait complete. 2024-06-06T07:05:13.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 68111 2.63 MB/sec execute 20 sec latency 14.746 ms 2024-06-06T07:05:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:13 smithi073 ceph-mon[32303]: pgmap v119: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:13 smithi136 ceph-mon[35901]: pgmap v119: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:13 smithi170 ceph-mon[36108]: pgmap v119: 129 pgs: 129 active+clean; 55 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 170 op/s 2024-06-06T07:05:14.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 68599 2.59 MB/sec execute 21 sec latency 15.111 ms 2024-06-06T07:05:15.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 69213 2.62 MB/sec execute 22 sec latency 18.036 ms 2024-06-06T07:05:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:15 smithi073 ceph-mon[32303]: pgmap v120: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-06-06T07:05:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:15 smithi136 ceph-mon[35901]: pgmap v120: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-06-06T07:05:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:15 smithi170 ceph-mon[36108]: pgmap v120: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 243 op/s 2024-06-06T07:05:16.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 69810 2.58 MB/sec execute 23 sec latency 16.964 ms 2024-06-06T07:05:17.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 70410 2.70 MB/sec execute 24 sec latency 15.922 ms 2024-06-06T07:05:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:17 smithi073 ceph-mon[32303]: pgmap v121: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:17 smithi136 ceph-mon[35901]: pgmap v121: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:17.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:17 smithi170 ceph-mon[36108]: pgmap v121: 129 pgs: 129 active+clean; 59 MiB data, 798 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:18.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 70917 2.79 MB/sec execute 25 sec latency 11.199 ms 2024-06-06T07:05:19.436 INFO:tasks.workunit.client.0.smithi073.stdout: 1 71287 2.73 MB/sec execute 26 sec latency 14.166 ms 2024-06-06T07:05:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:19 smithi073 ceph-mon[32303]: pgmap v122: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-06-06T07:05:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:19 smithi136 ceph-mon[35901]: pgmap v122: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-06-06T07:05:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:19 smithi170 ceph-mon[36108]: pgmap v122: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 231 op/s 2024-06-06T07:05:20.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 71560 2.65 MB/sec execute 27 sec latency 15.384 ms 2024-06-06T07:05:21.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 72055 2.62 MB/sec execute 28 sec latency 15.317 ms 2024-06-06T07:05:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:21 smithi073 ceph-mon[32303]: pgmap v123: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:21 smithi136 ceph-mon[35901]: pgmap v123: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:21 smithi170 ceph-mon[36108]: pgmap v123: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:22.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 72610 2.64 MB/sec execute 29 sec latency 16.120 ms 2024-06-06T07:05:23.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 73301 2.61 MB/sec execute 30 sec latency 19.519 ms 2024-06-06T07:05:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:23 smithi073 ceph-mon[32303]: pgmap v124: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:23 smithi136 ceph-mon[35901]: pgmap v124: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:23 smithi170 ceph-mon[36108]: pgmap v124: 129 pgs: 129 active+clean; 56 MiB data, 833 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-06T07:05:24.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 73894 2.69 MB/sec execute 31 sec latency 14.589 ms 2024-06-06T07:05:25.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 74330 2.68 MB/sec execute 32 sec latency 23.292 ms 2024-06-06T07:05:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:25 smithi073 ceph-mon[32303]: pgmap v125: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-06-06T07:05:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:25 smithi136 ceph-mon[35901]: pgmap v125: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-06-06T07:05:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:25 smithi170 ceph-mon[36108]: pgmap v125: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 226 op/s 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 -- 172.21.15.73:0/1699466501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 msgr2=0x7f0264100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 --2- 172.21.15.73:0/1699466501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264100d50 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f0258009930 tx=0x7f025802a420 comp rx=0 tx=0).stop 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 -- 172.21.15.73:0/1699466501 shutdown_connections 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 --2- 172.21.15.73:0/1699466501 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 0x7f026410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 --2- 172.21.15.73:0/1699466501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264100d50 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 --2- 172.21.15.73:0/1699466501 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f0264106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 -- 172.21.15.73:0/1699466501 >> 172.21.15.73:0/1699466501 conn(0x7f0264077a20 msgr2=0x7f0264077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 -- 172.21.15.73:0/1699466501 shutdown_connections 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.216+0000 7f0268b29640 1 -- 172.21.15.73:0/1699466501 wait complete. 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 Processor -- start 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 -- start start 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 0x7f02641986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f026419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0264110b50 con 0x7f02641008d0 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02641109d0 con 0x7f0264101290 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0268b29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0264110cd0 con 0x7f0264106890 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f026259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f026259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32898/0 (socket says 172.21.15.73:32898) 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f026259c640 1 -- 172.21.15.73:0/1532639602 learned_addr learned my addr 172.21.15.73:0/1532639602 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0262d9d640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f026419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0261d9b640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 0x7f02641986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0262d9d640 1 -- 172.21.15.73:0/1532639602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 msgr2=0x7f02641986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0262d9d640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 0x7f02641986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0262d9d640 1 -- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 msgr2=0x7f0264198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.217+0000 7f0262d9d640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f0262d9d640 1 -- 172.21.15.73:0/1532639602 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f026419d130 con 0x7f0264106890 2024-06-06T07:05:26.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f026259c640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f0262d9d640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f026419ca00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f025c00da10 tx=0x7f025c00ded0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f02537fe640 1 -- 172.21.15.73:0/1532639602 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c00bd70 con 0x7f0264106890 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f026419d420 con 0x7f0264106890 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f02537fe640 1 -- 172.21.15.73:0/1532639602 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c003e20 con 0x7f0264106890 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f02537fe640 1 -- 172.21.15.73:0/1532639602 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c0106c0 con 0x7f0264106890 2024-06-06T07:05:26.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.218+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02641a4cb0 con 0x7f0264106890 2024-06-06T07:05:26.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.220+0000 7f02537fe640 1 -- 172.21.15.73:0/1532639602 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f025c003970 con 0x7f0264106890 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 Processor -- start 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 -- start start 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f02537fe640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 0x7f024c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d8386550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f0261d9b640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 0x7f024c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 0x55c1d8390e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f02537fe640 1 -- 172.21.15.73:0/1532639602 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f025c0cfb70 con 0x7f0264106890 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c1d82ea4f0 con 0x55c1d8391350 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8288870 con 0x55c1d8386a90 2024-06-06T07:05:26.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f026b837740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c1d82e64b0 con 0x55c1d8386170 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f0262d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f0262d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32902/0 (socket says 172.21.15.73:32902) 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.221+0000 7f0262d9d640 1 -- 172.21.15.73:0/2454596867 learned_addr learned my addr 172.21.15.73:0/2454596867 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f026259c640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d8386550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0261d9b640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 0x55c1d8390e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 -- 172.21.15.73:0/2454596867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 msgr2=0x55c1d8386550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d8386550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 -- 172.21.15.73:0/2454596867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 msgr2=0x55c1d8390e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 0x55c1d8390e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 -- 172.21.15.73:0/2454596867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d82a49b0 con 0x55c1d8391350 2024-06-06T07:05:26.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f026259c640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d8386550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0261d9b640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 0x55c1d8390e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0261d9b640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 0x7f024c0a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0258009900 tx=0x7f0258005fd0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0262d9d640 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f025c098140 tx=0x7f025c09b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9555196bb69e01f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0250ff9640 1 -- 172.21.15.73:0/2454596867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c09bce0 con 0x55c1d8391350 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0250ff9640 1 -- 172.21.15.73:0/2454596867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c0a0090 con 0x55c1d8391350 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.222+0000 7f0250ff9640 1 -- 172.21.15.73:0/2454596867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c098350 con 0x55c1d8391350 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- 172.21.15.73:0/2454596867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 msgr2=0x55c1d8393740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f025c098140 tx=0x7f025c09b640 comp rx=0 tx=0).stop 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- 172.21.15.73:0/2454596867 shutdown_connections 2024-06-06T07:05:26.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d8393740 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386a90 0x55c1d8390e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 --2- 172.21.15.73:0/2454596867 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d8386550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- 172.21.15.73:0/2454596867 >> 172.21.15.73:0/2454596867 conn(0x55c1d8382680 msgr2=0x55c1d8382a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- 172.21.15.73:0/2454596867 shutdown_connections 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- 172.21.15.73:0/2454596867 wait complete. 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 -- start start 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.223+0000 7f026b837740 1 Processor -- start 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d838aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d838ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8393ec0 con 0x55c1d8386170 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8273ae0 con 0x55c1d8391350 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c1d8394040 con 0x55c1d8386a90 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 10 client.0 _ll_register_callbacks cb 0x55c1d82ac9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32904/0 (socket says 172.21.15.73:32904) 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 -- 172.21.15.73:0/4230499009 learned_addr learned my addr 172.21.15.73:0/4230499009 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0261d9b640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d838aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0262d9d640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d838ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 -- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 msgr2=0x55c1d838aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d838aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 -- 172.21.15.73:0/4230499009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 msgr2=0x55c1d838ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d838ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 -- 172.21.15.73:0/4230499009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d7ffe210 con 0x55c1d8386170 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026259c640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0254005f60 tx=0x7f025400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f490782b1f448a6d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02540120d0 con 0x55c1d8386170 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f025c0a0090 con 0x55c1d8386170 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f0254005020 con 0x55c1d8386170 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f0254012270 con 0x55c1d8386170 2024-06-06T07:05:26.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 10 client.16176 fetch_fsmap learned FSMap version 34 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.224+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1d80ca620 con 0x55c1d8386170 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c0a0090 con 0x55c1d8386170 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f025401c470 con 0x55c1d8386170 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 10 client.16176 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 20 client.16176 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d83a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 4 client.16176 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1d82fdf10 con 0x55c1d83a1600 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.225+0000 7f0261d9b640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d83a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f0261d9b640 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d83a39f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b6c95095da94bce server_cookie=7e4c43b688b401b2 in_seq=0 out_seq=0 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f0249ffb640 10 client.16176 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f0249ffb640 1 -- 172.21.15.73:0/4230499009 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c1d82fdf10 con 0x55c1d83a1600 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f0249ffb640 10 client.16176 handle_command_reply: tid=0 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 client.16176 shutdown 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 msgr2=0x55c1d838a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0254005f60 tx=0x7f025400f790 comp rx=0 tx=0).stop 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 shutdown_connections 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d83a39f0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d838ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f0249ffb640 0 client.16176 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:26.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d838aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 --2- 172.21.15.73:0/4230499009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386170 0x55c1d838a4e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 >> 172.21.15.73:0/4230499009 conn(0x55c1d8382680 msgr2=0x55c1d83883b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 shutdown_connections 2024-06-06T07:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.226+0000 7f026b837740 1 -- 172.21.15.73:0/4230499009 wait complete. 2024-06-06T07:05:26.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.227+0000 7f026b837740 20 client.16176 trim_cache size 0 max 16384 2024-06-06T07:05:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.241+0000 7f026b837740 1 Processor -- start 2024-06-06T07:05:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.241+0000 7f026b837740 1 -- start start 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386170 0x55c1d838ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d83a92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02641acc10 con 0x55c1d8391350 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02641ae7b0 con 0x55c1d8386170 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026b837740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02641ad2e0 con 0x55c1d8386a90 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32910/0 (socket says 172.21.15.73:32910) 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 -- 172.21.15.73:0/3642929415 learned_addr learned my addr 172.21.15.73:0/3642929415 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f026259c640 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386170 0x55c1d838ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 -- 172.21.15.73:0/3642929415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 msgr2=0x55c1d83a92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d83a92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 -- 172.21.15.73:0/3642929415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386170 msgr2=0x55c1d838ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386170 0x55c1d838ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 -- 172.21.15.73:0/3642929415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1d8393ab0 con 0x55c1d8391350 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f0262d9d640 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f025c0cd510 tx=0x7f025c0980b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fb3e9083a7d19e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f024bfff640 1 -- 172.21.15.73:0/3642929415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c014070 con 0x55c1d8391350 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f024bfff640 1 -- 172.21.15.73:0/3642929415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c09bce0 con 0x55c1d8391350 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.242+0000 7f024bfff640 1 -- 172.21.15.73:0/3642929415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f025c098350 con 0x55c1d8391350 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- 172.21.15.73:0/3642929415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 msgr2=0x55c1d83a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f025c0cd510 tx=0x7f025c0980b0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- 172.21.15.73:0/3642929415 shutdown_connections 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8391350 0x55c1d83a97f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386a90 0x55c1d83a92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- 172.21.15.73:0/3642929415 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8386170 0x55c1d838ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- 172.21.15.73:0/3642929415 >> 172.21.15.73:0/3642929415 conn(0x55c1d8382680 msgr2=0x55c1d8382a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- 172.21.15.73:0/3642929415 shutdown_connections 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- 172.21.15.73:0/3642929415 wait complete. 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- start start 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 Processor -- start 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d839da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d83968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02641ae0e0 con 0x55c1d8386a90 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02641ad9b0 con 0x55c1d8391350 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02641aabf0 con 0x55c1d8386170 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f026b837740 10 client.0 _ll_register_callbacks cb 0x7f0254009590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f0261d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f0261d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:32926/0 (socket says 172.21.15.73:32926) 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f0261d9b640 1 -- 172.21.15.73:0/44579792 learned_addr learned my addr 172.21.15.73:0/44579792 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.243+0000 7f0262d9d640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d83968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026259c640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d839da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 -- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 msgr2=0x55c1d839da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d839da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 -- 172.21.15.73:0/44579792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 msgr2=0x55c1d83968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d83968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 -- 172.21.15.73:0/44579792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1d83970a0 con 0x55c1d8386a90 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0261d9b640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0258046010 tx=0x7f025802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec96bc2185db4b00 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0258035040 con 0x55c1d8386a90 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f025c09bce0 con 0x55c1d8386a90 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f0258004040 con 0x55c1d8386a90 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7f025806a050 con 0x55c1d8386a90 2024-06-06T07:05:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 10 client.16182 fetch_fsmap learned FSMap version 34 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c1d840bf00 con 0x55c1d8386a90 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f025c09bce0 con 0x55c1d8386a90 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7f02580430b0 con 0x55c1d8386a90 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 10 client.16182 fetch_fsmap finished waiting for FSMap version 34 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.244+0000 7f026b837740 20 client.16182 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d840e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f026b837740 4 client.16182 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1d82f7c20 con 0x55c1d83a1600 2024-06-06T07:05:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f026259c640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d840e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f0249ffb640 10 client.16182 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:26.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.245+0000 7f026259c640 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d840e6a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf0b917dc4ceed4a server_cookie=e52ef897c8cd1f83 in_seq=0 out_seq=0 2024-06-06T07:05:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f0249ffb640 1 -- 172.21.15.73:0/44579792 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c1d82f7c20 con 0x55c1d83a1600 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f0249ffb640 10 client.16182 handle_command_reply: tid=0 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 client.16182 shutdown 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 msgr2=0x55c1d839dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0258046010 tx=0x7f025802a420 comp rx=0 tx=0).stop 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 shutdown_connections 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c1d83a1600 0x55c1d840e6a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c1d8391350 0x55c1d83968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c1d8386a90 0x55c1d839dfb0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 --2- 172.21.15.73:0/44579792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c1d8386170 0x55c1d839da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 >> 172.21.15.73:0/44579792 conn(0x55c1d8382680 msgr2=0x55c1d8389cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f0249ffb640 0 client.16182 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 shutdown_connections 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.322+0000 7f026b837740 1 -- 172.21.15.73:0/44579792 wait complete. 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f026b837740 20 client.16182 trim_cache size 0 max 16384 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:05:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 msgr2=0x7f024c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 0x7f024c0a5260 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0258009900 tx=0x7f0258005fd0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 msgr2=0x7f026419ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f026419ca00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f025c00da10 tx=0x7f025c00ded0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 shutdown_connections 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f024c0a2da0 0x7f024c0a5260 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0264106890 0x7f026419ca00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0264101290 0x7f02641986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 --2- 172.21.15.73:0/1532639602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02641008d0 0x7f0264198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.323+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 >> 172.21.15.73:0/1532639602 conn(0x7f0264077a20 msgr2=0x7f02640693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.324+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 shutdown_connections 2024-06-06T07:05:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:26.324+0000 7f0268b29640 1 -- 172.21.15.73:0/1532639602 wait complete. 2024-06-06T07:05:26.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 74781 2.74 MB/sec execute 33 sec latency 110.396 ms 2024-06-06T07:05:27.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 75055 2.66 MB/sec execute 34 sec latency 15.531 ms 2024-06-06T07:05:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:27 smithi073 ceph-mon[32303]: pgmap v126: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-06T07:05:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:27 smithi136 ceph-mon[35901]: pgmap v126: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-06T07:05:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:27 smithi170 ceph-mon[36108]: pgmap v126: 129 pgs: 129 active+clean; 56 MiB data, 839 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 153 op/s 2024-06-06T07:05:28.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 75485 2.61 MB/sec execute 35 sec latency 16.260 ms 2024-06-06T07:05:29.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 76039 2.66 MB/sec execute 36 sec latency 20.250 ms 2024-06-06T07:05:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:29 smithi073 ceph-mon[32303]: pgmap v127: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:29 smithi136 ceph-mon[35901]: pgmap v127: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:29.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:29 smithi170 ceph-mon[36108]: pgmap v127: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:30.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 76692 2.60 MB/sec execute 37 sec latency 21.974 ms 2024-06-06T07:05:31.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 77166 2.60 MB/sec execute 38 sec latency 13.901 ms 2024-06-06T07:05:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:31 smithi073 ceph-mon[32303]: pgmap v128: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:31 smithi136 ceph-mon[35901]: pgmap v128: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:31 smithi170 ceph-mon[36108]: pgmap v128: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:32.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 77855 2.69 MB/sec execute 39 sec latency 15.012 ms 2024-06-06T07:05:33.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 78308 2.73 MB/sec execute 40 sec latency 15.594 ms 2024-06-06T07:05:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:33 smithi073 ceph-mon[32303]: pgmap v129: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:33 smithi136 ceph-mon[35901]: pgmap v129: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:33.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:33 smithi170 ceph-mon[36108]: pgmap v129: 129 pgs: 129 active+clean; 57 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-06T07:05:34.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 78562 2.68 MB/sec execute 41 sec latency 11.898 ms 2024-06-06T07:05:35.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 79001 2.63 MB/sec execute 42 sec latency 11.480 ms 2024-06-06T07:05:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:35 smithi073 ceph-mon[32303]: pgmap v130: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-06-06T07:05:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:35 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:35 smithi136 ceph-mon[35901]: pgmap v130: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-06-06T07:05:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:35 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:35 smithi170 ceph-mon[36108]: pgmap v130: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 223 op/s 2024-06-06T07:05:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:35 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:36.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 79511 2.67 MB/sec execute 43 sec latency 14.929 ms 2024-06-06T07:05:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:05:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:05:36.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:05:36.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:05:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:05:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:05:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:05:36.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:05:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:05:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:05:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:05:36.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:05:37.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 80125 2.62 MB/sec execute 44 sec latency 23.862 ms 2024-06-06T07:05:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:37 smithi073 ceph-mon[32303]: pgmap v131: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-06T07:05:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:37 smithi136 ceph-mon[35901]: pgmap v131: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-06T07:05:37.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:37 smithi170 ceph-mon[36108]: pgmap v131: 129 pgs: 129 active+clean; 58 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-06T07:05:38.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 80644 2.61 MB/sec execute 45 sec latency 11.695 ms 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 -- 172.21.15.73:0/3375917852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100c10 msgr2=0x7f7678104c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 --2- 172.21.15.73:0/3375917852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100c10 0x7f7678104c50 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f766c00aa10 tx=0x7f766c02b720 comp rx=0 tx=0).stop 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 -- 172.21.15.73:0/3375917852 shutdown_connections 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 --2- 172.21.15.73:0/3375917852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7678105330 0x7f767810cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 --2- 172.21.15.73:0/3375917852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100c10 0x7f7678104c50 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 --2- 172.21.15.73:0/3375917852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678100260 0x7f7678100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 -- 172.21.15.73:0/3375917852 >> 172.21.15.73:0/3375917852 conn(0x7f7678076e30 msgr2=0x7f76780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 -- 172.21.15.73:0/3375917852 shutdown_connections 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.423+0000 7f767d272640 1 -- 172.21.15.73:0/3375917852 wait complete. 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 Processor -- start 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 -- start start 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678105330 0x7f76781986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f767819ca20 0x7f767819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f767810fa10 con 0x7f7678100260 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f767810f890 con 0x7f767819ca20 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f767d272640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f767810fb90 con 0x7f7678105330 2024-06-06T07:05:39.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42890/0 (socket says 172.21.15.73:42890) 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f7676ffd640 1 -- 172.21.15.73:0/1473066389 learned_addr learned my addr 172.21.15.73:0/1473066389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f76767fc640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678105330 0x7f76781986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f76777fe640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f767819ca20 0x7f767819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f7676ffd640 1 -- 172.21.15.73:0/1473066389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678105330 msgr2=0x7f76781986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.424+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678105330 0x7f76781986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f7676ffd640 1 -- 172.21.15.73:0/1473066389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f767819ca20 msgr2=0x7f767819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f767819ca20 0x7f767819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f7676ffd640 1 -- 172.21.15.73:0/1473066389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767819d510 con 0x7f7678100260 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9f0 tx=0x7f766800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f765ffff640 1 -- 172.21.15.73:0/1473066389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766800cd80 con 0x7f7678100260 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f765ffff640 1 -- 172.21.15.73:0/1473066389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76680054d0 con 0x7f7678100260 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767819d800 con 0x7f7678100260 2024-06-06T07:05:39.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f765ffff640 1 -- 172.21.15.73:0/1473066389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668010490 con 0x7f7678100260 2024-06-06T07:05:39.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.425+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a4f70 con 0x7f7678100260 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.427+0000 7f765ffff640 1 -- 172.21.15.73:0/1473066389 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7668005020 con 0x7f7678100260 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f765ffff640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 0x7f76600a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 Processor -- start 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 -- start start 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f76767fc640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 0x7f76600a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f765ffff640 1 -- 172.21.15.73:0/1473066389 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f76680cfb60 con 0x7f7678100260 2024-06-06T07:05:39.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b34803b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803ba90 0x55b348045e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b347f9f4f0 con 0x55b348046350 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b347f3d870 con 0x55b34803b170 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f767ff7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b347f9b4b0 con 0x55b34803ba90 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f76777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f76767fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803ba90 0x55b348045e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b34803b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f76777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42900/0 (socket says 172.21.15.73:42900) 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.428+0000 7f76777fe640 1 -- 172.21.15.73:0/2557254246 learned_addr learned my addr 172.21.15.73:0/2557254246 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76767fc640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 0x7f76600a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c030510 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 -- 172.21.15.73:0/2557254246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803ba90 msgr2=0x55b348045e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803ba90 0x55b348045e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 -- 172.21.15.73:0/2557254246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 msgr2=0x55b34803b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b34803b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 -- 172.21.15.73:0/2557254246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b347f599b0 con 0x55b348046350 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f7676ffd640 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b34803b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f76777fe640 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7670009f70 tx=0x7f7670010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb9d8a3fb94c38ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f765d7fa640 1 -- 172.21.15.73:0/2557254246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670016070 con 0x55b348046350 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f765d7fa640 1 -- 172.21.15.73:0/2557254246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7670004d20 con 0x55b348046350 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.429+0000 7f765d7fa640 1 -- 172.21.15.73:0/2557254246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670012790 con 0x55b348046350 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- 172.21.15.73:0/2557254246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 msgr2=0x55b348048740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7670009f70 tx=0x7f7670010800 comp rx=0 tx=0).stop 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- 172.21.15.73:0/2557254246 shutdown_connections 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b348048740 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803ba90 0x55b348045e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 --2- 172.21.15.73:0/2557254246 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b34803b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- 172.21.15.73:0/2557254246 >> 172.21.15.73:0/2557254246 conn(0x55b348037680 msgr2=0x55b348037a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- 172.21.15.73:0/2557254246 shutdown_connections 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- 172.21.15.73:0/2557254246 wait complete. 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 -- start start 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.430+0000 7f767ff7f740 1 Processor -- start 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b348043770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 0x55b348043cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b348048f80 con 0x55b34803ba90 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b347f28ae0 con 0x55b34803b170 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b348049100 con 0x55b348046350 2024-06-06T07:05:39.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 10 client.0 _ll_register_callbacks cb 0x55b347f619b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b348043770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:32962/0 (socket says 172.21.15.73:32962) 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f7676ffd640 1 -- 172.21.15.73:0/303912281 learned_addr learned my addr 172.21.15.73:0/303912281 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76777fe640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 0x55b348043cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 -- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 msgr2=0x55b348043cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 0x55b348043cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 -- 172.21.15.73:0/303912281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 msgr2=0x55b348043230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 -- 172.21.15.73:0/303912281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b347cb3210 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76777fe640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 0x55b348043cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f7676ffd640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f76767fc640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b348043770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c00a370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a7307de427ec1e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f765aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c043070 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f766c005510 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.431+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7670004d20 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f766c036040 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 10 client.16194 fetch_fsmap learned FSMap version 35 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b347d7f620 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7670004d20 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f766c03b040 con 0x55b34803ba90 2024-06-06T07:05:39.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 10 client.16194 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 10 client.16194 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 20 client.16194 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480564d0 0x55b3480588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 4 client.16194 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b347fb2f10 con 0x55b3480564d0 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.432+0000 7f7676ffd640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480564d0 0x55b3480588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f765aff5640 10 client.16194 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f7676ffd640 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480564d0 0x55b3480588c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b80dbf3a3cf3776 server_cookie=d941522839efe88c in_seq=0 out_seq=0 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f765aff5640 1 -- 172.21.15.73:0/303912281 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b347fb2f10 con 0x55b3480564d0 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f765aff5640 10 client.16194 handle_command_reply: tid=0 2024-06-06T07:05:39.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f767ff7f740 1 client.16194 shutdown 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 msgr2=0x55b348043770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.433+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b348043770 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f766c006010 tx=0x7f766c00a370 comp rx=0 tx=0).stop 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 shutdown_connections 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480564d0 0x55b3480588c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b348046350 0x55b348043cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b348043770 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 --2- 172.21.15.73:0/303912281 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803b170 0x55b348043230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 >> 172.21.15.73:0/303912281 conn(0x55b348037680 msgr2=0x55b34803d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f765aff5640 0 client.16194 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 shutdown_connections 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 1 -- 172.21.15.73:0/303912281 wait complete. 2024-06-06T07:05:39.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.434+0000 7f767ff7f740 20 client.16194 trim_cache size 0 max 16384 2024-06-06T07:05:39.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 81369 2.70 MB/sec execute 46 sec latency 15.142 ms 2024-06-06T07:05:39.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 Processor -- start 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 -- start start 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34803db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803ba90 0x55b34805e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aedc0 con 0x55b348046350 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b0930 con 0x55b34803ba90 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f767ff7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76781af490 con 0x55b34803b170 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42914/0 (socket says 172.21.15.73:42914) 2024-06-06T07:05:39.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f7676ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34803db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76767fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803ba90 0x55b34805e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 -- 172.21.15.73:0/437194552 learned_addr learned my addr 172.21.15.73:0/437194552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 -- 172.21.15.73:0/437194552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 msgr2=0x55b34803db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34803db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 -- 172.21.15.73:0/437194552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803ba90 msgr2=0x55b34805e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803ba90 0x55b34805e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.451+0000 7f76777fe640 1 -- 172.21.15.73:0/437194552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b348048ab0 con 0x55b348046350 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.452+0000 7f76777fe640 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55b348038810 tx=0x7f7670005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=642a58071f4ab603 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.452+0000 7f765cff9640 1 -- 172.21.15.73:0/437194552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767002a070 con 0x55b348046350 2024-06-06T07:05:39.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.452+0000 7f765cff9640 1 -- 172.21.15.73:0/437194552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7670004c80 con 0x55b348046350 2024-06-06T07:05:39.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.452+0000 7f765cff9640 1 -- 172.21.15.73:0/437194552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7670016070 con 0x55b348046350 2024-06-06T07:05:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- 172.21.15.73:0/437194552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 msgr2=0x55b34805e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55b348038810 tx=0x7f7670005c70 comp rx=0 tx=0).stop 2024-06-06T07:05:39.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- 172.21.15.73:0/437194552 shutdown_connections 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b348046350 0x55b34805e6f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b34803ba90 0x55b34805e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- 172.21.15.73:0/437194552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34803db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- 172.21.15.73:0/437194552 >> 172.21.15.73:0/437194552 conn(0x55b348037680 msgr2=0x55b34803e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- 172.21.15.73:0/437194552 shutdown_connections 2024-06-06T07:05:39.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- 172.21.15.73:0/437194552 wait complete. 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- start start 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 Processor -- start 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34804e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b348046350 0x55b3480bc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f76781b0260 con 0x55b34803ba90 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f76781afb60 con 0x55b348046350 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f76781acda0 con 0x55b34803b170 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f767ff7f740 10 client.0 _ll_register_callbacks cb 0x7f766c030e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f76767fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f76767fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42920/0 (socket says 172.21.15.73:42920) 2024-06-06T07:05:39.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f76767fc640 1 -- 172.21.15.73:0/1701325186 learned_addr learned my addr 172.21.15.73:0/1701325186 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.453+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34804e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76777fe640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b348046350 0x55b3480bc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 -- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 msgr2=0x55b34804e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34804e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 -- 172.21.15.73:0/1701325186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b348046350 msgr2=0x55b3480bc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:39.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b348046350 0x55b3480bc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 -- 172.21.15.73:0/1701325186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3480bc6a0 con 0x55b34803ba90 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f76767fc640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55b348042a90 tx=0x7f766c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99fdf06375a3b19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f765aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766c030950 con 0x55b34803ba90 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f766c0046c0 con 0x55b34803ba90 2024-06-06T07:05:39.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7670004c80 con 0x55b34803ba90 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f766c069050 con 0x55b34803ba90 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f767ff7f740 10 client.16200 fetch_fsmap learned FSMap version 35 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.454+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55b348040040 con 0x55b34803ba90 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7670004c80 con 0x55b34803ba90 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f766c03b040 con 0x55b34803ba90 2024-06-06T07:05:39.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 10 client.16200 fetch_fsmap finished waiting for FSMap version 35 2024-06-06T07:05:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 20 client.16200 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480c5590 0x55b3480c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 4 client.16200 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:05:39.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b347facc20 con 0x55b3480c5590 2024-06-06T07:05:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480c5590 0x55b3480c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f765aff5640 10 client.16200 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:39.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.455+0000 7f7676ffd640 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480c5590 0x55b3480c7980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c2917ed2b21dee1 server_cookie=95ed7bd3dd05ee5 in_seq=0 out_seq=0 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.477+0000 7f765aff5640 1 -- 172.21.15.73:0/1701325186 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b347facc20 con 0x55b3480c5590 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.477+0000 7f765aff5640 10 client.16200 handle_command_reply: tid=0 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.477+0000 7f767ff7f740 1 client.16200 shutdown 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.477+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 msgr2=0x55b34804e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.477+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x55b348042a90 tx=0x7f766c003820 comp rx=0 tx=0).stop 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 shutdown_connections 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3480c5590 0x55b3480c7980 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b348046350 0x55b3480bc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f765aff5640 0 client.16200 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b34803ba90 0x55b34804e8c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 --2- 172.21.15.73:0/1701325186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b34803b170 0x55b34804e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 >> 172.21.15.73:0/1701325186 conn(0x55b348037680 msgr2=0x55b3480431f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 shutdown_connections 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 1 -- 172.21.15.73:0/1701325186 wait complete. 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767ff7f740 20 client.16200 trim_cache size 0 max 16384 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:05:39.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:05:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:05:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 msgr2=0x7f76600a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 0x7f76600a5260 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f766c000c00 tx=0x7f766c030510 comp rx=0 tx=0).stop 2024-06-06T07:05:39.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 msgr2=0x7f76781981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f766800e9f0 tx=0x7f766800edf0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 shutdown_connections 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f767819ca20 0x7f767819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f76600a2da0 0x7f76600a5260 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7678105330 0x7f76781986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 --2- 172.21.15.73:0/1473066389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7678100260 0x7f76781981b0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.478+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 >> 172.21.15.73:0/1473066389 conn(0x7f7678076e30 msgr2=0x7f76780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.479+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 shutdown_connections 2024-06-06T07:05:39.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:39.479+0000 7f767d272640 1 -- 172.21.15.73:0/1473066389 wait complete. 2024-06-06T07:05:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:39 smithi073 ceph-mon[32303]: pgmap v132: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:05:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:39 smithi136 ceph-mon[35901]: pgmap v132: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:05:39.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:39 smithi170 ceph-mon[36108]: pgmap v132: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 224 op/s 2024-06-06T07:05:40.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 81752 2.71 MB/sec execute 47 sec latency 24.072 ms 2024-06-06T07:05:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:40 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:40.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:40 smithi136 ceph-mon[35901]: pgmap v133: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:40.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:40 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:40.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:40 smithi170 ceph-mon[36108]: pgmap v133: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:41.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:40 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:05:41.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:40 smithi073 ceph-mon[32303]: pgmap v133: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:41.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 82049 2.68 MB/sec execute 48 sec latency 12.312 ms 2024-06-06T07:05:42.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 82426 2.64 MB/sec execute 49 sec latency 12.734 ms 2024-06-06T07:05:43.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:42 smithi170 ceph-mon[36108]: pgmap v134: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:42 smithi073 ceph-mon[32303]: pgmap v134: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:42 smithi136 ceph-mon[35901]: pgmap v134: 129 pgs: 129 active+clean; 58 MiB data, 829 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.1 MiB/s wr, 148 op/s 2024-06-06T07:05:43.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 82901 2.62 MB/sec execute 50 sec latency 15.388 ms 2024-06-06T07:05:44.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 83529 2.63 MB/sec execute 51 sec latency 15.438 ms 2024-06-06T07:05:45.075 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:44 smithi073 ceph-mon[32303]: pgmap v135: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:45.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:44 smithi170 ceph-mon[36108]: pgmap v135: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:44 smithi136 ceph-mon[35901]: pgmap v135: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.8 MiB/s wr, 229 op/s 2024-06-06T07:05:45.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 84103 2.62 MB/sec execute 52 sec latency 14.864 ms 2024-06-06T07:05:46.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 84892 2.70 MB/sec execute 53 sec latency 10.564 ms 2024-06-06T07:05:47.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:46 smithi170 ceph-mon[36108]: pgmap v136: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:46 smithi073 ceph-mon[32303]: pgmap v136: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:46 smithi136 ceph-mon[35901]: pgmap v136: 129 pgs: 129 active+clean; 63 MiB data, 860 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:47.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 85289 2.71 MB/sec execute 54 sec latency 14.928 ms 2024-06-06T07:05:48.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 85588 2.69 MB/sec execute 55 sec latency 15.362 ms 2024-06-06T07:05:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:48 smithi170 ceph-mon[36108]: pgmap v137: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:49.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:48 smithi073 ceph-mon[32303]: pgmap v137: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:49.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:48 smithi136 ceph-mon[35901]: pgmap v137: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:49.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 85967 2.65 MB/sec execute 56 sec latency 17.209 ms 2024-06-06T07:05:50.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 86463 2.63 MB/sec execute 57 sec latency 12.627 ms 2024-06-06T07:05:51.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:50 smithi170 ceph-mon[36108]: pgmap v138: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:51.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:50 smithi073 ceph-mon[32303]: pgmap v138: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:51.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:50 smithi136 ceph-mon[35901]: pgmap v138: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:51.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 87085 2.65 MB/sec execute 58 sec latency 29.422 ms 2024-06-06T07:05:52.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 87618 2.63 MB/sec execute 59 sec latency 60.740 ms 2024-06-06T07:05:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.574+0000 7fb533aee640 1 -- 172.21.15.73:0/4126451868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 msgr2=0x7fb52c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.574+0000 7fb533aee640 1 --2- 172.21.15.73:0/4126451868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c100d50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009930 tx=0x7fb51c02a420 comp rx=0 tx=0).stop 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 -- 172.21.15.73:0/4126451868 shutdown_connections 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 --2- 172.21.15.73:0/4126451868 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c101290 0x7fb52c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 --2- 172.21.15.73:0/4126451868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c100d50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 --2- 172.21.15.73:0/4126451868 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c106890 0x7fb52c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 -- 172.21.15.73:0/4126451868 >> 172.21.15.73:0/4126451868 conn(0x7fb52c077a20 msgr2=0x7fb52c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 -- 172.21.15.73:0/4126451868 shutdown_connections 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 -- 172.21.15.73:0/4126451868 wait complete. 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 Processor -- start 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.575+0000 7fb533aee640 1 -- start start 2024-06-06T07:05:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c101290 0x7fb52c1994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c106890 0x7fb52c1a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c110b50 con 0x7fb52c1008d0 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1109d0 con 0x7fb52c101290 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c110cd0 con 0x7fb52c106890 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37908/0 (socket says 172.21.15.73:37908) 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 -- 172.21.15.73:0/4242613270 learned_addr learned my addr 172.21.15.73:0/4242613270 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53208b640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c106890 0x7fb52c1a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb531089640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c101290 0x7fb52c1994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 -- 172.21.15.73:0/4242613270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c106890 msgr2=0x7fb52c1a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c106890 0x7fb52c1a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 -- 172.21.15.73:0/4242613270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c101290 msgr2=0x7fb52c1994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c101290 0x7fb52c1994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 -- 172.21.15.73:0/4242613270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52c1a4530 con 0x7fb52c1008d0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb53188a640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e9f0 tx=0x7fb52800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb522ffd640 1 -- 172.21.15.73:0/4242613270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52800cd80 con 0x7fb52c1008d0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb522ffd640 1 -- 172.21.15.73:0/4242613270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb5280108e0 con 0x7fb52c1008d0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52c1a4700 con 0x7fb52c1008d0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb522ffd640 1 -- 172.21.15.73:0/4242613270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb528018920 con 0x7fb52c1008d0 2024-06-06T07:05:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.576+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb52c1a4c20 con 0x7fb52c1008d0 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.579+0000 7fb522ffd640 1 -- 172.21.15.73:0/4242613270 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb528010a90 con 0x7fb52c1008d0 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.579+0000 7fb5367fb740 1 Processor -- start 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.579+0000 7fb5367fb740 1 -- start start 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695170 0x56192c695550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56192c5f94f0 con 0x56192c695a90 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56192c597870 con 0x56192c6a0350 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb5367fb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56192c5f54b0 con 0x56192c695170 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37916/0 (socket says 172.21.15.73:37916) 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53188a640 1 -- 172.21.15.73:0/3218329443 learned_addr learned my addr 172.21.15.73:0/3218329443 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53208b640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695170 0x56192c695550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 -- 172.21.15.73:0/3218329443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695170 msgr2=0x56192c695550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb522ffd640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 0x7fb5180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695170 0x56192c695550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 -- 172.21.15.73:0/3218329443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 msgr2=0x56192c69fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 -- 172.21.15.73:0/3218329443 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192c5b39b0 con 0x56192c6a0350 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53188a640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb53208b640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 0x7fb5180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb522ffd640 1 -- 172.21.15.73:0/4242613270 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb5280cfb40 con 0x7fb52c1008d0 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.580+0000 7fb531089640 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a2740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009900 tx=0x7fb51c032b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65f4b76332965c71 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb517fff640 1 -- 172.21.15.73:0/3218329443 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c03f070 con 0x56192c6a0350 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb517fff640 1 -- 172.21.15.73:0/3218329443 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb51c032e30 con 0x56192c6a0350 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb517fff640 1 -- 172.21.15.73:0/3218329443 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb51c035070 con 0x56192c6a0350 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 -- 172.21.15.73:0/3218329443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 msgr2=0x56192c6a2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a2740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009900 tx=0x7fb51c032b80 comp rx=0 tx=0).stop 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb53208b640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 0x7fb5180a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb524009fd0 tx=0x7fb524004a40 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 -- 172.21.15.73:0/3218329443 shutdown_connections 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a2740 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695a90 0x56192c69fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 --2- 172.21.15.73:0/3218329443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695170 0x56192c695550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.581+0000 7fb5367fb740 1 -- 172.21.15.73:0/3218329443 >> 172.21.15.73:0/3218329443 conn(0x56192c691680 msgr2=0x56192c691a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 -- 172.21.15.73:0/3218329443 shutdown_connections 2024-06-06T07:05:52.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 -- 172.21.15.73:0/3218329443 wait complete. 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 -- start start 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 Processor -- start 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 0x56192c69d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 0x56192c69dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.582+0000 7fb5367fb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56192c6a2f20 con 0x56192c695170 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56192c582ae0 con 0x56192c695a90 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56192c6a30a0 con 0x56192c6a0350 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 10 client.0 _ll_register_callbacks cb 0x56192c5bb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37928/0 (socket says 172.21.15.73:37928) 2024-06-06T07:05:52.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 -- 172.21.15.73:0/2949991834 learned_addr learned my addr 172.21.15.73:0/2949991834 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb531089640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 0x56192c69d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53208b640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 0x56192c69dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 -- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 msgr2=0x56192c69dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 0x56192c69dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 -- 172.21.15.73:0/2949991834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 msgr2=0x56192c69d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 0x56192c69d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 -- 172.21.15.73:0/2949991834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192c30d210 con 0x56192c695170 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb531089640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 0x56192c69d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53208b640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 0x56192c69dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb53188a640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5280ccf90 tx=0x7fb528009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8204579c4e187800 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5157fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52809b7f0 con 0x56192c695170 2024-06-06T07:05:52.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb528010630 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb51c032e30 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb52809b610 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 10 client.16212 fetch_fsmap learned FSMap version 36 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.583+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56192c3d9620 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb51c032e30 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb528097a00 con 0x56192c695170 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 10 client.16212 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 10 client.16212 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 20 client.16212 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c6b0460 0x56192c6b2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 4 client.16212 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56192c60cf10 con 0x56192c6b0460 2024-06-06T07:05:52.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb531089640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c6b0460 0x56192c6b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb5157fa640 10 client.16212 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.584+0000 7fb531089640 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c6b0460 0x56192c6b2870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484ae6fe39dc102c server_cookie=39e345ae085aecf9 in_seq=0 out_seq=0 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5157fa640 1 -- 172.21.15.73:0/2949991834 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56192c60cf10 con 0x56192c6b0460 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5157fa640 10 client.16212 handle_command_reply: tid=0 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 client.16212 shutdown 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 msgr2=0x56192c69d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fb5280ccf90 tx=0x7fb528009310 comp rx=0 tx=0).stop 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 shutdown_connections 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c6b0460 0x56192c6b2870 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c6a0350 0x56192c69dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695a90 0x56192c69d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 --2- 172.21.15.73:0/2949991834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c69d2e0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5157fa640 0 client.16212 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 >> 172.21.15.73:0/2949991834 conn(0x56192c691680 msgr2=0x56192c697400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 shutdown_connections 2024-06-06T07:05:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.585+0000 7fb5367fb740 1 -- 172.21.15.73:0/2949991834 wait complete. 2024-06-06T07:05:52.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.586+0000 7fb5367fb740 20 client.16212 trim_cache size 0 max 16384 2024-06-06T07:05:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.601+0000 7fb5367fb740 1 Processor -- start 2024-06-06T07:05:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 -- start start 2024-06-06T07:05:52.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695170 0x56192c697b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6b81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1aece0 con 0x56192c6a0350 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b0820 con 0x56192c695170 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb5367fb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1af3b0 con 0x56192c695a90 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37930/0 (socket says 172.21.15.73:37930) 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 -- 172.21.15.73:0/4060093650 learned_addr learned my addr 172.21.15.73:0/4060093650 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb531089640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6b81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53188a640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695170 0x56192c697b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 -- 172.21.15.73:0/4060093650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 msgr2=0x56192c6b81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6b81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 -- 172.21.15.73:0/4060093650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695170 msgr2=0x56192c697b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695170 0x56192c697b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 -- 172.21.15.73:0/4060093650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56192c698ec0 con 0x56192c6a0350 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb531089640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.602+0000 7fb53208b640 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb52400fcf0 tx=0x7fb524013030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa1d01f58465ed26 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5177fe640 1 -- 172.21.15.73:0/4060093650 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52402e070 con 0x56192c6a0350 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5177fe640 1 -- 172.21.15.73:0/4060093650 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52401e0d0 con 0x56192c6a0350 2024-06-06T07:05:52.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5177fe640 1 -- 172.21.15.73:0/4060093650 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52401c680 con 0x56192c6a0350 2024-06-06T07:05:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 -- 172.21.15.73:0/4060093650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 msgr2=0x56192c6b8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb52400fcf0 tx=0x7fb524013030 comp rx=0 tx=0).stop 2024-06-06T07:05:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 -- 172.21.15.73:0/4060093650 shutdown_connections 2024-06-06T07:05:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c6a0350 0x56192c6b8710 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6b81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 --2- 172.21.15.73:0/4060093650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c695170 0x56192c697b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 -- 172.21.15.73:0/4060093650 >> 172.21.15.73:0/4060093650 conn(0x56192c691680 msgr2=0x56192c69bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 -- 172.21.15.73:0/4060093650 shutdown_connections 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.603+0000 7fb5367fb740 1 -- 172.21.15.73:0/4060093650 wait complete. 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 -- start start 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 Processor -- start 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6a38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1b0150 con 0x56192c695170 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1afa80 con 0x56192c6a0350 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb52c1accc0 con 0x56192c695a90 2024-06-06T07:05:52.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb5367fb740 10 client.0 _ll_register_callbacks cb 0x7fb52800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb53188a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37932/0 (socket says 172.21.15.73:37932) 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb53188a640 1 -- 172.21.15.73:0/512389446 learned_addr learned my addr 172.21.15.73:0/512389446 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb531089640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6a38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.604+0000 7fb53208b640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 -- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 msgr2=0x56192c6a38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6a38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 -- 172.21.15.73:0/512389446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 msgr2=0x56192c6a3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 -- 172.21.15.73:0/512389446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56192c719480 con 0x56192c695170 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb53188a640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb528099f60 tx=0x7fb52809bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22373134a222bd12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:05:52.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52809b7f0 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fb528010640 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb52401e0d0 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb5280a4970 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 10 client.16218 fetch_fsmap learned FSMap version 36 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x56192c71a790 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb52401e0d0 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb52809ab70 con 0x56192c695170 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 10 client.16218 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 10 client.16218 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 20 client.16218 populate_metadata read hostname 'smithi073' 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c71ad10 0x56192c71d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 4 client.16218 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:05:52.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.605+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56192c606c20 con 0x56192c71ad10 2024-06-06T07:05:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.606+0000 7fb531089640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c71ad10 0x56192c71d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:05:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.606+0000 7fb5157fa640 10 client.16218 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:52.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.606+0000 7fb531089640 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c71ad10 0x56192c71d100 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21f21114aeefbfe server_cookie=cd8fb4daf519bba6 in_seq=0 out_seq=0 2024-06-06T07:05:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.627+0000 7fb5157fa640 1 -- 172.21.15.73:0/512389446 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56192c606c20 con 0x56192c71ad10 2024-06-06T07:05:52.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.627+0000 7fb5157fa640 10 client.16218 handle_command_reply: tid=0 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.627+0000 7fb5367fb740 1 client.16218 shutdown 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.627+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 msgr2=0x56192c6a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.627+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb528099f60 tx=0x7fb52809bb40 comp rx=0 tx=0).stop 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 shutdown_connections 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56192c71ad10 0x56192c71d100 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56192c6a0350 0x56192c6a3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56192c695a90 0x56192c6a38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5157fa640 0 client.16218 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 --2- 172.21.15.73:0/512389446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56192c695170 0x56192c6a3390 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 >> 172.21.15.73:0/512389446 conn(0x56192c691680 msgr2=0x56192c69c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 shutdown_connections 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 1 -- 172.21.15.73:0/512389446 wait complete. 2024-06-06T07:05:52.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb5367fb740 20 client.16218 trim_cache size 0 max 16384 2024-06-06T07:05:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:05:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:05:52.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 msgr2=0x7fb5180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 0x7fb5180a5260 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fb524009fd0 tx=0x7fb524004a40 comp rx=0 tx=0).stop 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 msgr2=0x7fb52c198f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fb52800e9f0 tx=0x7fb52800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 shutdown_connections 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb5180a2da0 0x7fb5180a5260 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb52c106890 0x7fb52c1a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb52c101290 0x7fb52c1994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 --2- 172.21.15.73:0/4242613270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb52c1008d0 0x7fb52c198f70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:05:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 >> 172.21.15.73:0/4242613270 conn(0x7fb52c077a20 msgr2=0x7fb52c103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:05:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 shutdown_connections 2024-06-06T07:05:52.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:05:52.628+0000 7fb533aee640 1 -- 172.21.15.73:0/4242613270 wait complete. 2024-06-06T07:05:53.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:52 smithi170 ceph-mon[36108]: pgmap v139: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:52 smithi073 ceph-mon[32303]: pgmap v139: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:52 smithi136 ceph-mon[35901]: pgmap v139: 129 pgs: 129 active+clean; 64 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 159 op/s 2024-06-06T07:05:53.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 88364 2.69 MB/sec execute 60 sec latency 23.791 ms 2024-06-06T07:05:54.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 88751 2.71 MB/sec execute 61 sec latency 14.886 ms 2024-06-06T07:05:55.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:54 smithi170 ceph-mon[36108]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-06-06T07:05:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:54 smithi073 ceph-mon[32303]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-06-06T07:05:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:54 smithi136 ceph-mon[35901]: pgmap v140: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1.9 MiB/s wr, 236 op/s 2024-06-06T07:05:55.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 89104 2.69 MB/sec execute 62 sec latency 14.192 ms 2024-06-06T07:05:56.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 89420 2.66 MB/sec execute 63 sec latency 15.277 ms 2024-06-06T07:05:57.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:56 smithi170 ceph-mon[36108]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:56 smithi073 ceph-mon[32303]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:56 smithi136 ceph-mon[35901]: pgmap v141: 129 pgs: 129 active+clean; 61 MiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:05:57.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 89910 2.64 MB/sec execute 64 sec latency 15.615 ms 2024-06-06T07:05:58.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 90538 2.65 MB/sec execute 65 sec latency 15.761 ms 2024-06-06T07:05:59.098 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:05:58 smithi170 ceph-mon[36108]: pgmap v142: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:05:58 smithi073 ceph-mon[32303]: pgmap v142: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:05:58 smithi136 ceph-mon[35901]: pgmap v142: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 235 op/s 2024-06-06T07:05:59.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 91119 2.64 MB/sec execute 66 sec latency 15.920 ms 2024-06-06T07:06:00.437 INFO:tasks.workunit.client.0.smithi073.stdout: 1 91725 2.68 MB/sec execute 67 sec latency 17.367 ms 2024-06-06T07:06:01.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:00 smithi170 ceph-mon[36108]: pgmap v143: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:00 smithi073 ceph-mon[32303]: pgmap v143: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:00 smithi136 ceph-mon[35901]: pgmap v143: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:01.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 92236 2.71 MB/sec execute 68 sec latency 17.165 ms 2024-06-06T07:06:02.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 92609 2.70 MB/sec execute 69 sec latency 15.163 ms 2024-06-06T07:06:03.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:02 smithi170 ceph-mon[36108]: pgmap v144: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:02 smithi073 ceph-mon[32303]: pgmap v144: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:02 smithi136 ceph-mon[35901]: pgmap v144: 129 pgs: 129 active+clean; 63 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-06T07:06:03.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 92898 2.66 MB/sec execute 70 sec latency 11.949 ms 2024-06-06T07:06:04.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 93434 2.65 MB/sec execute 71 sec latency 10.736 ms 2024-06-06T07:06:05.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:04 smithi170 ceph-mon[36108]: pgmap v145: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-06T07:06:05.103 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:04 smithi136 ceph-mon[35901]: pgmap v145: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-06T07:06:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:04 smithi073 ceph-mon[32303]: pgmap v145: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 242 op/s 2024-06-06T07:06:05.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 94058 2.66 MB/sec execute 72 sec latency 15.509 ms 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 -- 172.21.15.73:0/3477691192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443810bf70 msgr2=0x7f443810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 --2- 172.21.15.73:0/3477691192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443810bf70 0x7f443810e410 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f4430009ca0 tx=0x7f443002a9e0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 -- 172.21.15.73:0/3477691192 shutdown_connections 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 --2- 172.21.15.73:0/3477691192 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f443810bf70 0x7f443810e410 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 --2- 172.21.15.73:0/3477691192 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4438103550 0x7f443810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.736+0000 7f443f488640 1 --2- 172.21.15.73:0/3477691192 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 0x7f4438103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- 172.21.15.73:0/3477691192 >> 172.21.15.73:0/3477691192 conn(0x7f4438077a20 msgr2=0x7f4438077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- 172.21.15.73:0/3477691192 shutdown_connections 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- 172.21.15.73:0/3477691192 wait complete. 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 Processor -- start 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- start start 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 0x7f4438100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443810bf70 0x7f4438101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111c20 con 0x7f4438103550 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111aa0 con 0x7f4438102c30 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.737+0000 7f443f488640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4438111da0 con 0x7f443810bf70 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54480/0 (socket says 172.21.15.73:54480) 2024-06-06T07:06:05.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 -- 172.21.15.73:0/1151704095 learned_addr learned my addr 172.21.15.73:0/1151704095 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443d224640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 0x7f4438100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443da25640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443810bf70 0x7f4438101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 -- 172.21.15.73:0/1151704095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443810bf70 msgr2=0x7f4438101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443810bf70 0x7f4438101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 -- 172.21.15.73:0/1151704095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 msgr2=0x7f4438100880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 0x7f4438100880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 -- 172.21.15.73:0/1151704095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44381a65c0 con 0x7f4438103550 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443ca23640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f442e7fc640 1 -- 172.21.15.73:0/1151704095 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442800cd80 con 0x7f4438103550 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f442e7fc640 1 -- 172.21.15.73:0/1151704095 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f44280054d0 con 0x7f4438103550 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44381a68b0 con 0x7f4438103550 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f442e7fc640 1 -- 172.21.15.73:0/1151704095 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428010490 con 0x7f4438103550 2024-06-06T07:06:05.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.738+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44381a6db0 con 0x7f4438103550 2024-06-06T07:06:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f442e7fc640 1 -- 172.21.15.73:0/1151704095 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4428005020 con 0x7f4438103550 2024-06-06T07:06:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 Processor -- start 2024-06-06T07:06:05.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 -- start start 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b43e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ece8a9d4f0 con 0x55ece8b44350 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ece8a3b870 con 0x55ece8b39a90 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f4442192740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ece8a994b0 con 0x55ece8b39170 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f443da25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f443ca23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f443da25640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56848/0 (socket says 172.21.15.73:56848) 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f443da25640 1 -- 172.21.15.73:0/1708485542 learned_addr learned my addr 172.21.15.73:0/1708485542 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.741+0000 7f443d224640 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b43e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 -- 172.21.15.73:0/1708485542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 msgr2=0x55ece8b43e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b43e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 -- 172.21.15.73:0/1708485542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 msgr2=0x55ece8b46740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b46740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f442e7fc640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 0x7f44240a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 -- 172.21.15.73:0/1708485542 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece8a579b0 con 0x55ece8b39170 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f442e7fc640 1 -- 172.21.15.73:0/1151704095 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f44280cfad0 con 0x7f4438103550 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443d224640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 0x7f44240a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443da25640 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4430009420 tx=0x7f4430005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ca1ad77a280fc1f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f44237fe640 1 -- 172.21.15.73:0/1708485542 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430049070 con 0x55ece8b39170 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.742+0000 7f443d224640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 0x7f44240a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55ece8b3a710 tx=0x7f4434005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.743+0000 7f44237fe640 1 -- 172.21.15.73:0/1708485542 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443002e840 con 0x55ece8b39170 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.743+0000 7f44237fe640 1 -- 172.21.15.73:0/1708485542 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430033040 con 0x55ece8b39170 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- 172.21.15.73:0/1708485542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 msgr2=0x55ece8b39550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4430009420 tx=0x7f4430005ef0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- 172.21.15.73:0/1708485542 shutdown_connections 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b43e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- 172.21.15.73:0/1708485542 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b39550 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- 172.21.15.73:0/1708485542 >> 172.21.15.73:0/1708485542 conn(0x55ece8b35680 msgr2=0x55ece8b35a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- 172.21.15.73:0/1708485542 shutdown_connections 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- 172.21.15.73:0/1708485542 wait complete. 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- start start 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 Processor -- start 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 0x55ece8b41820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.751 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.744+0000 7f4442192740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ece8b46f20 con 0x55ece8b44350 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4442192740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ece8a26ae0 con 0x55ece8b39170 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4442192740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ece8b470a0 con 0x55ece8b39a90 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4442192740 10 client.0 _ll_register_callbacks cb 0x55ece8a5f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443ca23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 0x55ece8b41820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443d224640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443ca23640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 0x55ece8b41820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56854/0 (socket says 172.21.15.73:56854) 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443d224640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53386/0 (socket says 172.21.15.73:53386) 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443ca23640 1 -- 172.21.15.73:0/298991401 learned_addr learned my addr 172.21.15.73:0/298991401 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54510/0 (socket says 172.21.15.73:54510) 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 -- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 msgr2=0x55ece8b41820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 0x55ece8b41820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 -- 172.21.15.73:0/298991401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 msgr2=0x55ece8b412e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.752 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 -- 172.21.15.73:0/298991401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece87b1210 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443d224640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f443da25640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f443002af00 tx=0x7f443002ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9a5754c94f86a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430054060 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443002e840 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.745+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f443002e390 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f443005b050 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 10 client.16230 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ece887d620 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443002e840 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4430043070 con 0x55ece8b44350 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 10 client.16230 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:05.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 10 client.16230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 20 client.16230 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8b54340 0x55ece8b56730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 4 client.16230 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ece8ab0f10 con 0x55ece8b54340 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.746+0000 7f443d224640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8b54340 0x55ece8b56730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.747+0000 7f4420ff9640 10 client.16230 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.747+0000 7f443d224640 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8b54340 0x55ece8b56730 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e74aef46341dbb9e server_cookie=b3c9928e1e45cca7 in_seq=0 out_seq=0 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.747+0000 7f4420ff9640 1 -- 172.21.15.73:0/298991401 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ece8ab0f10 con 0x55ece8b54340 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.747+0000 7f4420ff9640 10 client.16230 handle_command_reply: tid=0 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.747+0000 7f4442192740 1 client.16230 shutdown 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 msgr2=0x55ece8b41d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f443002af00 tx=0x7f443002ff10 comp rx=0 tx=0).stop 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 shutdown_connections 2024-06-06T07:06:05.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8b54340 0x55ece8b56730 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b41d60 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39a90 0x55ece8b41820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4420ff9640 0 client.16230 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 --2- 172.21.15.73:0/298991401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b412e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 >> 172.21.15.73:0/298991401 conn(0x55ece8b35680 msgr2=0x55ece8b3b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 shutdown_connections 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 1 -- 172.21.15.73:0/298991401 wait complete. 2024-06-06T07:06:05.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.748+0000 7f4442192740 20 client.16230 trim_cache size 0 max 16384 2024-06-06T07:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 Processor -- start 2024-06-06T07:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 -- start start 2024-06-06T07:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b5bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b0ec0 con 0x55ece8b44350 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b2a00 con 0x55ece8b39a90 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f4442192740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b1590 con 0x55ece8b39170 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443d224640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54520/0 (socket says 172.21.15.73:54520) 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 -- 172.21.15.73:0/1281859031 learned_addr learned my addr 172.21.15.73:0/1281859031 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443ca23640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b5bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 -- 172.21.15.73:0/1281859031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 msgr2=0x55ece8b419b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b419b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 -- 172.21.15.73:0/1281859031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 msgr2=0x55ece8b5bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b5bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 -- 172.21.15.73:0/1281859031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ece8b398c0 con 0x55ece8b44350 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443ca23640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b5bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443d224640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.765+0000 7f443da25640 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4430031010 tx=0x7f443002eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48823c601a8be4b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4422ffd640 1 -- 172.21.15.73:0/1281859031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430054040 con 0x55ece8b44350 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4422ffd640 1 -- 172.21.15.73:0/1281859031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443002e840 con 0x55ece8b44350 2024-06-06T07:06:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4422ffd640 1 -- 172.21.15.73:0/1281859031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4430049070 con 0x55ece8b44350 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 -- 172.21.15.73:0/1281859031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 msgr2=0x55ece8b5c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f4430031010 tx=0x7f443002eac0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 -- 172.21.15.73:0/1281859031 shutdown_connections 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b44350 0x55ece8b5c2a0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39a90 0x55ece8b5bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 --2- 172.21.15.73:0/1281859031 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b39170 0x55ece8b419b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 -- 172.21.15.73:0/1281859031 >> 172.21.15.73:0/1281859031 conn(0x55ece8b35680 msgr2=0x55ece8b412a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 -- 172.21.15.73:0/1281859031 shutdown_connections 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.766+0000 7f4442192740 1 -- 172.21.15.73:0/1281859031 wait complete. 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 -- start start 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 Processor -- start 2024-06-06T07:06:05.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b4c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b44350 0x55ece8b48200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b2330 con 0x55ece8b39a90 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f44381b1c60 con 0x55ece8b39170 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f44381aeea0 con 0x55ece8b44350 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f4442192740 10 client.0 _ll_register_callbacks cb 0x7f4430047070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f443ca23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f443ca23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54524/0 (socket says 172.21.15.73:54524) 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f443ca23640 1 -- 172.21.15.73:0/750205105 learned_addr learned my addr 172.21.15.73:0/750205105 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f443d224640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b4c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.767+0000 7f443da25640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b44350 0x55ece8b48200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 -- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b44350 msgr2=0x55ece8b48200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b44350 0x55ece8b48200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 -- 172.21.15.73:0/750205105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 msgr2=0x55ece8b4c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:05.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b4c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 -- 172.21.15.73:0/750205105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ece8b488f0 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f443ca23640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4428009250 tx=0x7f4428097fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70a7eea27fe2cc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4428005310 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f44280a4420 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f443002e840 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f44280b9050 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 10 client.16236 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ece8bbeca0 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f443002e840 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f44280a4b00 con 0x55ece8b39a90 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 10 client.16236 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:05.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 10 client.16236 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 20 client.16236 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8bbf400 0x55ece8bc1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 4 client.16236 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.768+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ece8aaac20 con 0x55ece8bbf400 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.769+0000 7f443d224640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8bbf400 0x55ece8bc1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.769+0000 7f4420ff9640 10 client.16236 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:05.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.769+0000 7f443d224640 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8bbf400 0x55ece8bc1820 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e953c3ac4f5b5530 server_cookie=e28e80a0e9070a6b in_seq=0 out_seq=0 2024-06-06T07:06:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4420ff9640 1 -- 172.21.15.73:0/750205105 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ece8aaac20 con 0x55ece8bbf400 2024-06-06T07:06:05.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4420ff9640 10 client.16236 handle_command_reply: tid=0 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 client.16236 shutdown 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 msgr2=0x55ece8b47cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f4428009250 tx=0x7f4428097fa0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 shutdown_connections 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ece8bbf400 0x55ece8bc1820 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ece8b44350 0x55ece8b48200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ece8b39a90 0x55ece8b47cc0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 --2- 172.21.15.73:0/750205105 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ece8b39170 0x55ece8b4c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 >> 172.21.15.73:0/750205105 conn(0x55ece8b35680 msgr2=0x55ece8b3b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4420ff9640 0 client.16236 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 shutdown_connections 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.787+0000 7f4442192740 1 -- 172.21.15.73:0/750205105 wait complete. 2024-06-06T07:06:05.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f4442192740 20 client.16236 trim_cache size 0 max 16384 2024-06-06T07:06:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:06:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:06:05.790 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:06:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 msgr2=0x7f44240a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 0x7f44240a5260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x55ece8b3a710 tx=0x7f4434005f90 comp rx=0 tx=0).stop 2024-06-06T07:06:05.791 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 msgr2=0x7f4438100dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f442800e9f0 tx=0x7f442800edf0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 shutdown_connections 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f44240a2da0 0x7f44240a5260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f443810bf70 0x7f4438101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4438103550 0x7f4438100dc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 --2- 172.21.15.73:0/1151704095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4438102c30 0x7f4438100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 >> 172.21.15.73:0/1151704095 conn(0x7f4438077a20 msgr2=0x7f44380fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 shutdown_connections 2024-06-06T07:06:05.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:05.788+0000 7f443f488640 1 -- 172.21.15.73:0/1151704095 wait complete. 2024-06-06T07:06:06.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 94634 2.65 MB/sec execute 73 sec latency 19.563 ms 2024-06-06T07:06:07.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:06 smithi170 ceph-mon[36108]: pgmap v146: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-06T07:06:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:06 smithi073 ceph-mon[32303]: pgmap v146: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-06T07:06:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:06 smithi136 ceph-mon[35901]: pgmap v146: 129 pgs: 129 active+clean; 67 MiB data, 995 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-06T07:06:07.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 95170 2.68 MB/sec execute 74 sec latency 74.596 ms 2024-06-06T07:06:08.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 95639 2.68 MB/sec execute 75 sec latency 15.717 ms 2024-06-06T07:06:09.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:08 smithi170 ceph-mon[36108]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-06T07:06:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:08 smithi073 ceph-mon[32303]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-06T07:06:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:08 smithi136 ceph-mon[35901]: pgmap v147: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 240 op/s 2024-06-06T07:06:09.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 96102 2.70 MB/sec execute 76 sec latency 13.990 ms 2024-06-06T07:06:10.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 96394 2.67 MB/sec execute 77 sec latency 14.899 ms 2024-06-06T07:06:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:10 smithi073 ceph-mon[32303]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:10 smithi136 ceph-mon[35901]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:11.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:10 smithi170 ceph-mon[36108]: pgmap v148: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:11.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 96859 2.66 MB/sec execute 78 sec latency 19.463 ms 2024-06-06T07:06:12.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 97384 2.66 MB/sec execute 79 sec latency 16.634 ms 2024-06-06T07:06:13.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:12 smithi073 ceph-mon[32303]: pgmap v149: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:13.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:12 smithi136 ceph-mon[35901]: pgmap v149: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:13.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:12 smithi170 ceph-mon[36108]: pgmap v149: 129 pgs: 129 active+clean; 68 MiB data, 1007 MiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-06T07:06:13.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 98000 2.64 MB/sec execute 80 sec latency 25.056 ms 2024-06-06T07:06:14.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 98435 2.63 MB/sec execute 81 sec latency 15.913 ms 2024-06-06T07:06:15.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:14 smithi073 ceph-mon[32303]: pgmap v150: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-06-06T07:06:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:14 smithi136 ceph-mon[35901]: pgmap v150: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-06-06T07:06:15.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:14 smithi170 ceph-mon[36108]: pgmap v150: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 238 op/s 2024-06-06T07:06:15.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 99128 2.68 MB/sec execute 82 sec latency 15.590 ms 2024-06-06T07:06:16.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 99584 2.70 MB/sec execute 83 sec latency 14.852 ms 2024-06-06T07:06:17.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:16 smithi073 ceph-mon[32303]: pgmap v151: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-06T07:06:17.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:16 smithi136 ceph-mon[35901]: pgmap v151: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-06T07:06:17.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:16 smithi170 ceph-mon[36108]: pgmap v151: 129 pgs: 129 active+clean; 68 MiB data, 1023 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-06T07:06:17.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 99843 2.67 MB/sec execute 84 sec latency 13.172 ms 2024-06-06T07:06:18.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 100299 2.65 MB/sec execute 85 sec latency 15.129 ms 2024-06-06T07:06:18.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/3967244362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 msgr2=0x7f6c14106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/3967244362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c14106c70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009930 tx=0x7f6c1002a420 comp rx=0 tx=0).stop 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/3967244362 shutdown_connections 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/3967244362 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c14101290 0x7f6c1410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/3967244362 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c141008d0 0x7f6c14100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/3967244362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c14106c70 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.883+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/3967244362 >> 172.21.15.73:0/3967244362 conn(0x7f6c14077a20 msgr2=0x7f6c14077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/3967244362 shutdown_connections 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/3967244362 wait complete. 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 Processor -- start 2024-06-06T07:06:18.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- start start 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c141008d0 0x7f6c140fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c14101290 0x7f6c140fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14110b50 con 0x7f6c14106890 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141109d0 con 0x7f6c141008d0 2024-06-06T07:06:18.887 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.884+0000 7f6c1bcbd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c14110cd0 con 0x7f6c14101290 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55102/0 (socket says 172.21.15.73:55102) 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 -- 172.21.15.73:0/661801136 learned_addr learned my addr 172.21.15.73:0/661801136 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c19258640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c14101290 0x7f6c140fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c19a59640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c141008d0 0x7f6c140fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 -- 172.21.15.73:0/661801136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c14101290 msgr2=0x7f6c140fdda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c14101290 0x7f6c140fdda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 -- 172.21.15.73:0/661801136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c141008d0 msgr2=0x7f6c140fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c141008d0 0x7f6c140fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 -- 172.21.15.73:0/661801136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c141a6620 con 0x7f6c14106890 2024-06-06T07:06:18.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1a25a640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00eb10 tx=0x7f6c0c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c0affd640 1 -- 172.21.15.73:0/661801136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c004b20 con 0x7f6c14106890 2024-06-06T07:06:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c141a6910 con 0x7f6c14106890 2024-06-06T07:06:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c0affd640 1 -- 172.21.15.73:0/661801136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c0c004cc0 con 0x7f6c14106890 2024-06-06T07:06:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c0affd640 1 -- 172.21.15.73:0/661801136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0c010690 con 0x7f6c14106890 2024-06-06T07:06:18.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.885+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c141a6e10 con 0x7f6c14106890 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.887+0000 7f6c0affd640 1 -- 172.21.15.73:0/661801136 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6c0c005020 con 0x7f6c14106890 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 Processor -- start 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 -- start start 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddd8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632edde3350 0x5632edde5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632edd3c4f0 con 0x5632eddd8a90 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5632edcda870 con 0x5632edde3350 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5632edd384b0 con 0x5632eddd8170 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c19258640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddd8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.888+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55112/0 (socket says 172.21.15.73:55112) 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19a59640 1 -- 172.21.15.73:0/2994134614 learned_addr learned my addr 172.21.15.73:0/2994134614 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c0affd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 0x7f6c000a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632edde3350 0x5632edde5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c1a25a640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 0x7f6c000a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 -- 172.21.15.73:0/2994134614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632edde3350 msgr2=0x5632edde5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632edde3350 0x5632edde5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 -- 172.21.15.73:0/2994134614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 msgr2=0x5632edde2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 -- 172.21.15.73:0/2994134614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632edcf69b0 con 0x5632eddd8170 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c0affd640 1 -- 172.21.15.73:0/661801136 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6c0c0cfd10 con 0x7f6c14106890 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c19258640 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddd8550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5632eddd5810 tx=0x7f6c04005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=691a94e4fbe1ed40 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6c1a25a640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 0x7f6c000a5260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c09bb80 tx=0x7f6c0c0bc040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6bfffff640 1 -- 172.21.15.73:0/2994134614 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c04012ca0 con 0x5632eddd8170 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6bfffff640 1 -- 172.21.15.73:0/2994134614 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c04012e40 con 0x5632eddd8170 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.889+0000 7f6bfffff640 1 -- 172.21.15.73:0/2994134614 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040049a0 con 0x5632eddd8170 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2994134614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 msgr2=0x5632eddd8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddd8550 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x5632eddd5810 tx=0x7f6c04005c90 comp rx=0 tx=0).stop 2024-06-06T07:06:18.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2994134614 shutdown_connections 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632edde3350 0x5632edde5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2994134614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddd8550 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2994134614 >> 172.21.15.73:0/2994134614 conn(0x5632eddd4680 msgr2=0x5632eddd4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2994134614 shutdown_connections 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2994134614 wait complete. 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 -- start start 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.890+0000 7f6c1e9cf740 1 Processor -- start 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632edde02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632edde3350 0x5632edde0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5632edde5f80 con 0x5632eddd8a90 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5632edcc5ae0 con 0x5632eddd8170 2024-06-06T07:06:18.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5632edde6100 con 0x5632edde3350 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c1e9cf740 10 client.0 _ll_register_callbacks cb 0x5632edcfe9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55120/0 (socket says 172.21.15.73:55120) 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 -- 172.21.15.73:0/800349151 learned_addr learned my addr 172.21.15.73:0/800349151 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19258640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632edde02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19258640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632edde3350 0x5632edde0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 -- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632edde3350 msgr2=0x5632edde0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632edde3350 0x5632edde0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 -- 172.21.15.73:0/800349151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 msgr2=0x5632edde02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632edde02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 -- 172.21.15.73:0/800349151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632eda50210 con 0x5632eddd8a90 2024-06-06T07:06:18.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6c19a59640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6c10033000 tx=0x7f6c10031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682d92f16f5fc16a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.891+0000 7f6bfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10035040 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6c10004030 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5632edd4ff10 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c1004d050 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 10 client.16248 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632edb1c620 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5632edd4ff10 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f6c10004890 con 0x5632eddd8a90 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 10 client.16248 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 20 client.16248 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:18.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632eddf34c0 0x5632eddf58d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 4 client.16248 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:06:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5632edd49c20 con 0x5632eddf34c0 2024-06-06T07:06:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.892+0000 7f6c19258640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632eddf34c0 0x5632eddf58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.893+0000 7f6bfd7fa640 10 client.16248 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:18.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.893+0000 7f6c19258640 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632eddf34c0 0x5632eddf58d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e5984b7d5074a3 server_cookie=ab148ed9f90d95b1 in_seq=0 out_seq=0 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/800349151 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5632edd49c20 con 0x5632eddf34c0 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6bfd7fa640 10 client.16248 handle_command_reply: tid=0 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 client.16248 shutdown 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 msgr2=0x5632edde0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f6c10033000 tx=0x7f6c10031ef0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 shutdown_connections 2024-06-06T07:06:18.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632eddf34c0 0x5632eddf58d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632edde3350 0x5632edde0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632eddd8a90 0x5632edde0820 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/800349151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632edde02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 >> 172.21.15.73:0/800349151 conn(0x5632eddd4680 msgr2=0x5632eddda400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.894+0000 7f6bfd7fa640 0 client.16248 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.895+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 shutdown_connections 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.895+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/800349151 wait complete. 2024-06-06T07:06:18.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.895+0000 7f6c1e9cf740 20 client.16248 trim_cache size 0 max 16384 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 Processor -- start 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 -- start start 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddfb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141b0ff0 con 0x5632edde3350 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141b2b30 con 0x5632eddd8a90 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141b16c0 con 0x5632eddd8170 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19258640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:60582/0 (socket says 172.21.15.73:60582) 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19258640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddfb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 -- 172.21.15.73:0/1751858544 learned_addr learned my addr 172.21.15.73:0/1751858544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 -- 172.21.15.73:0/1751858544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 msgr2=0x5632eddfb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddfb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 -- 172.21.15.73:0/1751858544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 msgr2=0x5632edddd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.912+0000 7f6c19a59640 1 -- 172.21.15.73:0/1751858544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5632eddf5f50 con 0x5632eddd8a90 2024-06-06T07:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6c19a59640 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009850 tx=0x7f6c1002a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab4622f3662b71c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6bff7fe640 1 -- 172.21.15.73:0/1751858544 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10056070 con 0x5632eddd8a90 2024-06-06T07:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6bff7fe640 1 -- 172.21.15.73:0/1751858544 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c10041070 con 0x5632eddd8a90 2024-06-06T07:06:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6bff7fe640 1 -- 172.21.15.73:0/1751858544 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c10035030 con 0x5632eddd8a90 2024-06-06T07:06:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/1751858544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 msgr2=0x5632eddfb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.913+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009850 tx=0x7f6c1002a420 comp rx=0 tx=0).stop 2024-06-06T07:06:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/1751858544 shutdown_connections 2024-06-06T07:06:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8a90 0x5632eddfb720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/1751858544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8170 0x5632eddfb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/1751858544 >> 172.21.15.73:0/1751858544 conn(0x5632eddd4680 msgr2=0x5632edde0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/1751858544 shutdown_connections 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/1751858544 wait complete. 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- start start 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 Processor -- start 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632eddeb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141b2460 con 0x5632edde3350 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141b1d90 con 0x5632eddd8170 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.914+0000 7f6c1e9cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6c141aefd0 con 0x5632eddd8a90 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c1e9cf740 10 client.0 _ll_register_callbacks cb 0x7f6c10031b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632eddeb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19a59640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57990/0 (socket says 172.21.15.73:57990) 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19a59640 1 -- 172.21.15.73:0/2149131389 learned_addr learned my addr 172.21.15.73:0/2149131389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 -- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 msgr2=0x5632edddcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 -- 172.21.15.73:0/2149131389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 msgr2=0x5632eddeb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632eddeb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 -- 172.21.15.73:0/2149131389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5632edddd850 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19a59640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c19258640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd310 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5632eddf6150 tx=0x7f6c04017040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd98b59ff351a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6bfd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c040108f0 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f6c04010a90 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c10041070 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6c04010cb0 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.915+0000 7f6c1e9cf740 10 client.16254 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5632ede5df10 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c10041070 con 0x5632edde3350 2024-06-06T07:06:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f6c04038070 con 0x5632edde3350 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 10 client.16254 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 20 client.16254 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632ede5e3e0 0x5632ede607d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 4 client.16254 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5632edd43f70 con 0x5632ede5e3e0 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c19a59640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632ede5e3e0 0x5632ede607d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6bfd7fa640 10 client.16254 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.916+0000 7f6c19a59640 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632ede5e3e0 0x5632ede607d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1df9804956fa691 server_cookie=b6612bb9f1799a08 in_seq=0 out_seq=0 2024-06-06T07:06:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6bfd7fa640 1 -- 172.21.15.73:0/2149131389 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5632edd43f70 con 0x5632ede5e3e0 2024-06-06T07:06:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6bfd7fa640 10 client.16254 handle_command_reply: tid=0 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 client.16254 shutdown 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 msgr2=0x5632edddd310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd310 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x5632eddf6150 tx=0x7f6c04017040 comp rx=0 tx=0).stop 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 shutdown_connections 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5632ede5e3e0 0x5632ede607d0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5632edde3350 0x5632edddd310 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5632eddd8a90 0x5632edddcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6bfd7fa640 0 client.16254 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 --2- 172.21.15.73:0/2149131389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5632eddd8170 0x5632eddeb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.936+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 >> 172.21.15.73:0/2149131389 conn(0x5632eddd4680 msgr2=0x5632eddda3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 shutdown_connections 2024-06-06T07:06:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1e9cf740 1 -- 172.21.15.73:0/2149131389 wait complete. 2024-06-06T07:06:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1e9cf740 20 client.16254 trim_cache size 0 max 16384 2024-06-06T07:06:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:06:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:06:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 msgr2=0x7f6c000a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 0x7f6c000a5260 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c09bb80 tx=0x7f6c0c0bc040 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 msgr2=0x7f6c140fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c00eb10 tx=0x7f6c0c00c2d0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 shutdown_connections 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c14106890 0x7f6c140fe2e0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6c000a2da0 0x7f6c000a5260 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6c14101290 0x7f6c140fdda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 --2- 172.21.15.73:0/661801136 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6c141008d0 0x7f6c140fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:18.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 >> 172.21.15.73:0/661801136 conn(0x7f6c14077a20 msgr2=0x7f6c140693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 shutdown_connections 2024-06-06T07:06:18.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:18.937+0000 7f6c1bcbd640 1 -- 172.21.15.73:0/661801136 wait complete. 2024-06-06T07:06:19.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:18 smithi073 ceph-mon[32303]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-06-06T07:06:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:18 smithi136 ceph-mon[35901]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-06-06T07:06:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:18 smithi170 ceph-mon[36108]: pgmap v152: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 222 op/s 2024-06-06T07:06:19.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 100796 2.67 MB/sec execute 86 sec latency 17.410 ms 2024-06-06T07:06:20.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 101339 2.64 MB/sec execute 87 sec latency 31.433 ms 2024-06-06T07:06:21.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:20 smithi073 ceph-mon[32303]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:21.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:20 smithi136 ceph-mon[35901]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:21.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:20 smithi170 ceph-mon[36108]: pgmap v153: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:21.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 101854 2.64 MB/sec execute 88 sec latency 19.156 ms 2024-06-06T07:06:22.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 102625 2.68 MB/sec execute 89 sec latency 15.863 ms 2024-06-06T07:06:23.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:22 smithi073 ceph-mon[32303]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:22 smithi136 ceph-mon[35901]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:22 smithi170 ceph-mon[36108]: pgmap v154: 129 pgs: 129 active+clean; 73 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:06:23.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 103019 2.69 MB/sec execute 90 sec latency 15.054 ms 2024-06-06T07:06:24.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 103316 2.68 MB/sec execute 91 sec latency 14.793 ms 2024-06-06T07:06:25.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:24 smithi073 ceph-mon[32303]: pgmap v155: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-06-06T07:06:25.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:24 smithi136 ceph-mon[35901]: pgmap v155: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-06-06T07:06:25.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:24 smithi170 ceph-mon[36108]: pgmap v155: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 220 op/s 2024-06-06T07:06:25.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 103668 2.65 MB/sec execute 92 sec latency 16.535 ms 2024-06-06T07:06:26.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 104173 2.64 MB/sec execute 93 sec latency 14.637 ms 2024-06-06T07:06:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:26 smithi073 ceph-mon[32303]: pgmap v156: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-06-06T07:06:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:26 smithi136 ceph-mon[35901]: pgmap v156: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-06-06T07:06:27.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:26 smithi170 ceph-mon[36108]: pgmap v156: 129 pgs: 129 active+clean; 70 MiB data, 1016 MiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 1.4 MiB/s wr, 144 op/s 2024-06-06T07:06:27.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 104736 2.65 MB/sec execute 94 sec latency 24.795 ms 2024-06-06T07:06:28.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 105285 2.64 MB/sec execute 95 sec latency 19.554 ms 2024-06-06T07:06:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:28 smithi073 ceph-mon[32303]: pgmap v157: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-06-06T07:06:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:28 smithi136 ceph-mon[35901]: pgmap v157: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-06-06T07:06:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:28 smithi170 ceph-mon[36108]: pgmap v157: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 219 op/s 2024-06-06T07:06:29.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 105882 2.67 MB/sec execute 96 sec latency 21.709 ms 2024-06-06T07:06:30.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 106315 2.66 MB/sec execute 97 sec latency 15.702 ms 2024-06-06T07:06:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:30 smithi073 ceph-mon[32303]: pgmap v158: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:30 smithi136 ceph-mon[35901]: pgmap v158: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:31.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:30 smithi170 ceph-mon[36108]: pgmap v158: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:31.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 106773 2.68 MB/sec execute 98 sec latency 11.129 ms 2024-06-06T07:06:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 -- 172.21.15.73:0/1935315471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 msgr2=0x7fd998107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/1935315471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd998107db0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd994009930 tx=0x7fd99402a420 comp rx=0 tx=0).stop 2024-06-06T07:06:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 -- 172.21.15.73:0/1935315471 shutdown_connections 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/1935315471 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 0x7fd99810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/1935315471 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd998101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/1935315471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd998107db0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 -- 172.21.15.73:0/1935315471 >> 172.21.15.73:0/1935315471 conn(0x7fd9980fd450 msgr2=0x7fd9980ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 -- 172.21.15.73:0/1935315471 shutdown_connections 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.035+0000 7fd99fbf9640 1 -- 172.21.15.73:0/1935315471 wait complete. 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 Processor -- start 2024-06-06T07:06:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 -- start start 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 0x7fd998198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd99819ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd998111c60 con 0x7fd9981079d0 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd998111ae0 con 0x7fd9981019c0 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99fbf9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd998111de0 con 0x7fd998102380 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99d995640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99e196640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd99819ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.039 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99d194640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 0x7fd998198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99d995640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:41694/0 (socket says 172.21.15.73:41694) 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.036+0000 7fd99d995640 1 -- 172.21.15.73:0/593415094 learned_addr learned my addr 172.21.15.73:0/593415094 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 -- 172.21.15.73:0/593415094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 msgr2=0x7fd998198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 0x7fd998198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 -- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 msgr2=0x7fd99819ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd99819ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 -- 172.21.15.73:0/593415094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99819d160 con 0x7fd9981019c0 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99e196640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd99819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99d995640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd994004770 tx=0x7fd9940042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd98effd640 1 -- 172.21.15.73:0/593415094 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99403f070 con 0x7fd9981019c0 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99819d3f0 con 0x7fd9981019c0 2024-06-06T07:06:32.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd98effd640 1 -- 172.21.15.73:0/593415094 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd99403b460 con 0x7fd9981019c0 2024-06-06T07:06:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd98effd640 1 -- 172.21.15.73:0/593415094 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994035070 con 0x7fd9981019c0 2024-06-06T07:06:32.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.037+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9981a4ce0 con 0x7fd9981019c0 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd98effd640 1 -- 172.21.15.73:0/593415094 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd99403b610 con 0x7fd9981019c0 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd9a29e9740 1 Processor -- start 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd9a29e9740 1 -- start start 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b661e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.040+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b664740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556a2b5bb4f0 con 0x556a2b662350 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a2b559870 con 0x556a2b657a90 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556a2b5b74b0 con 0x556a2b657170 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99d194640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b664740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:50710/0 (socket says 172.21.15.73:50710) 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 -- 172.21.15.73:0/1575703881 learned_addr learned my addr 172.21.15.73:0/1575703881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99d995640 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b661e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 -- 172.21.15.73:0/1575703881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 msgr2=0x556a2b661e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd98effd640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 0x7fd9840a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b661e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 -- 172.21.15.73:0/1575703881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 msgr2=0x556a2b664740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b664740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 -- 172.21.15.73:0/1575703881 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2b5759b0 con 0x556a2b657170 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd98effd640 1 -- 172.21.15.73:0/593415094 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fd9940efae0 con 0x7fd9981019c0 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 0x7fd9840a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.041+0000 7fd99e196640 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd990010710 tx=0x7fd990010bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a496734c90967f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd983fff640 1 -- 172.21.15.73:0/1575703881 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990012be0 con 0x556a2b657170 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd983fff640 1 -- 172.21.15.73:0/1575703881 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd990012d80 con 0x556a2b657170 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd983fff640 1 -- 172.21.15.73:0/1575703881 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd990011620 con 0x556a2b657170 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1575703881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 msgr2=0x556a2b657550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd99e196640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 0x7fd9840a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd99001f000 tx=0x7fd990022000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd990010710 tx=0x7fd990010bd0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1575703881 shutdown_connections 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b664740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b661e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1575703881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b657550 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1575703881 >> 172.21.15.73:0/1575703881 conn(0x556a2b653680 msgr2=0x556a2b653a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1575703881 shutdown_connections 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.042+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1575703881 wait complete. 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 -- start start 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 Processor -- start 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b65f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b65fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556a2b664f80 con 0x556a2b657a90 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556a2b544ae0 con 0x556a2b662350 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556a2b665100 con 0x556a2b657170 2024-06-06T07:06:32.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd9a29e9740 10 client.0 _ll_register_callbacks cb 0x556a2b57d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd99d194640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd99d194640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52280/0 (socket says 172.21.15.73:52280) 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd99d194640 1 -- 172.21.15.73:0/281140127 learned_addr learned my addr 172.21.15.73:0/281140127 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd99e196640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b65fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.043+0000 7fd99d995640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b65f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 -- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 msgr2=0x556a2b65f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b65f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 -- 172.21.15.73:0/281140127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 msgr2=0x556a2b65fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b65fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 -- 172.21.15.73:0/281140127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2b2cf210 con 0x556a2b657a90 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd99d194640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd98800bf00 tx=0x7fd988011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1607dcc3643e3343 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd988004020 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fd9880041c0 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd990012d80 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd988024050 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9a29e9740 10 client.16266 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556a2b39b620 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.044+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd990012d80 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd988003850 con 0x556a2b657a90 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 10 client.16266 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 20 client.16266 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6725c0 0x556a2b6749d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 4 client.16266 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:06:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a2b5cef10 con 0x556a2b6725c0 2024-06-06T07:06:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd99d995640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6725c0 0x556a2b6749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd9817fa640 10 client.16266 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:32.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.045+0000 7fd99d995640 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6725c0 0x556a2b6749d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79f1956d5f4dff7b server_cookie=c257e6432840304 in_seq=0 out_seq=0 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9817fa640 1 -- 172.21.15.73:0/281140127 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556a2b5cef10 con 0x556a2b6725c0 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9817fa640 10 client.16266 handle_command_reply: tid=0 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 client.16266 shutdown 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 msgr2=0x556a2b65f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd98800bf00 tx=0x7fd988011810 comp rx=0 tx=0).stop 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 shutdown_connections 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6725c0 0x556a2b6749d0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b65fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657a90 0x556a2b65f7b0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9817fa640 0 client.16266 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/281140127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b65f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 >> 172.21.15.73:0/281140127 conn(0x556a2b653680 msgr2=0x556a2b659400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 shutdown_connections 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 1 -- 172.21.15.73:0/281140127 wait complete. 2024-06-06T07:06:32.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.046+0000 7fd9a29e9740 20 client.16266 trim_cache size 0 max 16384 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 Processor -- start 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 -- start start 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b6749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b659b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981aee80 con 0x556a2b662350 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981b09c0 con 0x556a2b657a90 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981af550 con 0x556a2b657170 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52282/0 (socket says 172.21.15.73:52282) 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 -- 172.21.15.73:0/1048363388 learned_addr learned my addr 172.21.15.73:0/1048363388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99d995640 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b6749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99d194640 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b659b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 -- 172.21.15.73:0/1048363388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 msgr2=0x556a2b6749b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b6749b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 -- 172.21.15.73:0/1048363388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 msgr2=0x556a2b659b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b659b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 -- 172.21.15.73:0/1048363388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a2b66e1e0 con 0x556a2b662350 2024-06-06T07:06:32.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd99e196640 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd9900092f0 tx=0x7fd990026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b362447b262d7c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.064+0000 7fd9837fe640 1 -- 172.21.15.73:0/1048363388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99001eab0 con 0x556a2b662350 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9837fe640 1 -- 172.21.15.73:0/1048363388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd990037070 con 0x556a2b662350 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9837fe640 1 -- 172.21.15.73:0/1048363388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd99001e410 con 0x556a2b662350 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1048363388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 msgr2=0x556a2b652660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd9900092f0 tx=0x7fd990026000 comp rx=0 tx=0).stop 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1048363388 shutdown_connections 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b662350 0x556a2b652660 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b657a90 0x556a2b659b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/1048363388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657170 0x556a2b6749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1048363388 >> 172.21.15.73:0/1048363388 conn(0x556a2b653680 msgr2=0x556a2b651f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1048363388 shutdown_connections 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.065+0000 7fd9a29e9740 1 -- 172.21.15.73:0/1048363388 wait complete. 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 -- start start 2024-06-06T07:06:32.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 Processor -- start 2024-06-06T07:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657a90 0x556a2b665cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b6661f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981b02f0 con 0x556a2b657170 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981afc20 con 0x556a2b662350 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9981ace60 con 0x556a2b657a90 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd9a29e9740 10 client.0 _ll_register_callbacks cb 0x7fd98800bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52288/0 (socket says 172.21.15.73:52288) 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 -- 172.21.15.73:0/500657470 learned_addr learned my addr 172.21.15.73:0/500657470 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d194640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657a90 0x556a2b665cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99e196640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b6661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 -- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657a90 msgr2=0x556a2b665cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657a90 0x556a2b665cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 -- 172.21.15.73:0/500657470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 msgr2=0x556a2b6661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.066+0000 7fd99d995640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b6661f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd99d995640 1 -- 172.21.15.73:0/500657470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a2b666860 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd99d995640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9940bbc90 tx=0x7fd9940f3630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba4542dd496acf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994048040 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fd9940ed100 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd990037070 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd9940bb660 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 10 client.16272 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x556a2b6dd1b0 con 0x556a2b657170 2024-06-06T07:06:32.073 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd990037070 con 0x556a2b657170 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fd99404b060 con 0x556a2b657170 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 10 client.16272 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 20 client.16272 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6dd8d0 0x556a2b6dfcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 4 client.16272 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.067+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a2b5c8c20 con 0x556a2b6dd8d0 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.068+0000 7fd99d194640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6dd8d0 0x556a2b6dfcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.068+0000 7fd99d194640 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6dd8d0 0x556a2b6dfcf0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98e476a91af8385 server_cookie=8fb8b5f9108563f1 in_seq=0 out_seq=0 2024-06-06T07:06:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.068+0000 7fd9817fa640 10 client.16272 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.087+0000 7fd9817fa640 1 -- 172.21.15.73:0/500657470 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556a2b5c8c20 con 0x556a2b6dd8d0 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.087+0000 7fd9817fa640 10 client.16272 handle_command_reply: tid=0 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.087+0000 7fd9a29e9740 1 client.16272 shutdown 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 msgr2=0x556a2b66a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd9940bbc90 tx=0x7fd9940f3630 comp rx=0 tx=0).stop 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 shutdown_connections 2024-06-06T07:06:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556a2b6dd8d0 0x556a2b6dfcf0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556a2b662350 0x556a2b6661f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556a2b657a90 0x556a2b665cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9817fa640 0 client.16272 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 --2- 172.21.15.73:0/500657470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556a2b657170 0x556a2b66a940 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 >> 172.21.15.73:0/500657470 conn(0x556a2b653680 msgr2=0x556a2b66da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 shutdown_connections 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 1 -- 172.21.15.73:0/500657470 wait complete. 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd9a29e9740 20 client.16272 trim_cache size 0 max 16384 2024-06-06T07:06:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:06:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:06:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:06:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 msgr2=0x7fd9840a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 0x7fd9840a5260 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd99001f000 tx=0x7fd990022000 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 msgr2=0x7fd9981981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.088+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd994004770 tx=0x7fd9940042f0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 shutdown_connections 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9981079d0 0x7fd99819ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9840a2da0 0x7fd9840a5260 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd998102380 0x7fd998198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 --2- 172.21.15.73:0/593415094 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9981019c0 0x7fd9981981c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 >> 172.21.15.73:0/593415094 conn(0x7fd9980fd450 msgr2=0x7fd9980fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 shutdown_connections 2024-06-06T07:06:32.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:32.089+0000 7fd99fbf9640 1 -- 172.21.15.73:0/593415094 wait complete. 2024-06-06T07:06:32.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 107037 2.65 MB/sec execute 99 sec latency 66.478 ms 2024-06-06T07:06:33.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:32 smithi073 ceph-mon[32303]: pgmap v159: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:32 smithi136 ceph-mon[35901]: pgmap v159: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:33.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:32 smithi170 ceph-mon[36108]: pgmap v159: 129 pgs: 129 active+clean; 71 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 1020 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-06T07:06:33.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 107507 2.65 MB/sec execute 100 sec latency 17.264 ms 2024-06-06T07:06:34.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 108012 2.65 MB/sec execute 101 sec latency 18.807 ms 2024-06-06T07:06:35.148 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:34 smithi170 ceph-mon[36108]: pgmap v160: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-06-06T07:06:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:34 smithi073 ceph-mon[32303]: pgmap v160: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-06-06T07:06:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:34 smithi136 ceph-mon[35901]: pgmap v160: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 221 op/s 2024-06-06T07:06:35.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 108593 2.63 MB/sec execute 102 sec latency 43.198 ms 2024-06-06T07:06:36.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 109059 2.63 MB/sec execute 103 sec latency 15.766 ms 2024-06-06T07:06:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:36 smithi073 ceph-mon[32303]: pgmap v161: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:06:37.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:36 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:06:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:36 smithi136 ceph-mon[35901]: pgmap v161: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:06:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:36 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:06:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:36 smithi170 ceph-mon[36108]: pgmap v161: 129 pgs: 129 active+clean; 76 MiB data, 1013 MiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:06:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:36 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:06:37.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 109743 2.66 MB/sec execute 104 sec latency 89.247 ms 2024-06-06T07:06:38.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:37 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:06:38.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:37 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:06:38.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:37 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:06:38.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:37 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:06:38.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:37 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:06:38.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:37 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:06:38.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:37 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:06:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:37 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:06:38.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:37 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:06:38.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 110157 2.67 MB/sec execute 105 sec latency 15.092 ms 2024-06-06T07:06:39.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 110426 2.66 MB/sec execute 106 sec latency 12.742 ms 2024-06-06T07:06:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:39 smithi073 ceph-mon[32303]: pgmap v162: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:39.818 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:39 smithi136 ceph-mon[35901]: pgmap v162: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:39 smithi170 ceph-mon[36108]: pgmap v162: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:40.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 110791 2.64 MB/sec execute 107 sec latency 15.473 ms 2024-06-06T07:06:41.438 INFO:tasks.workunit.client.0.smithi073.stdout: 1 111263 2.63 MB/sec execute 108 sec latency 17.925 ms 2024-06-06T07:06:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:41 smithi073 ceph-mon[32303]: pgmap v163: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:41 smithi136 ceph-mon[35901]: pgmap v163: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:41.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:41 smithi170 ceph-mon[36108]: pgmap v163: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:42.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 111820 2.63 MB/sec execute 109 sec latency 25.223 ms 2024-06-06T07:06:43.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 112374 2.63 MB/sec execute 110 sec latency 22.451 ms 2024-06-06T07:06:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:43 smithi073 ceph-mon[32303]: pgmap v164: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:43 smithi136 ceph-mon[35901]: pgmap v164: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:43 smithi170 ceph-mon[36108]: pgmap v164: 129 pgs: 129 active+clean; 77 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 1.4 MiB/s wr, 141 op/s 2024-06-06T07:06:44.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 112974 2.65 MB/sec execute 111 sec latency 15.137 ms 2024-06-06T07:06:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 -- 172.21.15.73:0/873244132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380101990 msgr2=0x7fa380101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 --2- 172.21.15.73:0/873244132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380101990 0x7fa380101e10 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa368009930 tx=0x7fa36802a420 comp rx=0 tx=0).stop 2024-06-06T07:06:45.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 -- 172.21.15.73:0/873244132 shutdown_connections 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 --2- 172.21.15.73:0/873244132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380102350 0x7fa38010e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 --2- 172.21.15.73:0/873244132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380101990 0x7fa380101e10 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 --2- 172.21.15.73:0/873244132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 0x7fa380107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 -- 172.21.15.73:0/873244132 >> 172.21.15.73:0/873244132 conn(0x7fa3800fd460 msgr2=0x7fa3800ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 -- 172.21.15.73:0/873244132 shutdown_connections 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.182+0000 7fa384c2a640 1 -- 172.21.15.73:0/873244132 wait complete. 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 Processor -- start 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 -- start start 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 0x7fa38019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa380111c60 con 0x7fa380102350 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa380111ae0 con 0x7fa380101990 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.183+0000 7fa384c2a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa380111de0 con 0x7fa3801079a0 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa37e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa37e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57512/0 (socket says 172.21.15.73:57512) 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43190/0 (socket says 172.21.15.73:43190) 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa37e59c640 1 -- 172.21.15.73:0/292718386 learned_addr learned my addr 172.21.15.73:0/292718386 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 0x7fa38019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 -- 172.21.15.73:0/292718386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 msgr2=0x7fa38019ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 0x7fa38019ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 -- 172.21.15.73:0/292718386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 msgr2=0x7fa380198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 -- 172.21.15.73:0/292718386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38019d150 con 0x7fa380102350 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa37e59c640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa377fff640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa368031440 tx=0x7fa368004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa3777fe640 1 -- 172.21.15.73:0/292718386 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36803f070 con 0x7fa380102350 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38019d3e0 con 0x7fa380102350 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa3777fe640 1 -- 172.21.15.73:0/292718386 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368003ca0 con 0x7fa380102350 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa3777fe640 1 -- 172.21.15.73:0/292718386 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368035070 con 0x7fa380102350 2024-06-06T07:06:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.184+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3801a4cd0 con 0x7fa380102350 2024-06-06T07:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.187+0000 7fa3777fe640 1 -- 172.21.15.73:0/292718386 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa36803b4f0 con 0x7fa380102350 2024-06-06T07:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.187+0000 7fa387a15740 1 Processor -- start 2024-06-06T07:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 -- start start 2024-06-06T07:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa3777fe640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 0x7fa36c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b5c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 0x55d757b69740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d757ac04f0 con 0x55d757b5ca90 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d757a5e870 con 0x55d757b67350 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa387a15740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d757abc4b0 con 0x55d757b5c170 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 0x7fa36c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43204/0 (socket says 172.21.15.73:43204) 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 -- 172.21.15.73:0/297804666 learned_addr learned my addr 172.21.15.73:0/297804666 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa37e59c640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b5c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa3777fe640 1 -- 172.21.15.73:0/292718386 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa3680efc60 con 0x7fa380102350 2024-06-06T07:06:45.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 0x55d757b69740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 -- 172.21.15.73:0/297804666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 msgr2=0x55d757b5c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b5c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 -- 172.21.15.73:0/297804666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 msgr2=0x55d757b69740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 0x55d757b69740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa377fff640 1 -- 172.21.15.73:0/297804666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d757a7a9b0 con 0x55d757b5ca90 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.188+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 0x55d757b69740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa377fff640 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa36802a940 tx=0x7fa36802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b794552bf8949e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 0x7fa36c0a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55d757b59aa0 tx=0x7fa378009ae0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa374ff9640 1 -- 172.21.15.73:0/297804666 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680bbe80 con 0x55d757b5ca90 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa374ff9640 1 -- 172.21.15.73:0/297804666 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3680c0180 con 0x55d757b5ca90 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa374ff9640 1 -- 172.21.15.73:0/297804666 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3680c4810 con 0x55d757b5ca90 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 -- 172.21.15.73:0/297804666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 msgr2=0x55d757b66e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fa36802a940 tx=0x7fa36802a420 comp rx=0 tx=0).stop 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 -- 172.21.15.73:0/297804666 shutdown_connections 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b67350 0x55d757b69740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5ca90 0x55d757b66e10 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 --2- 172.21.15.73:0/297804666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b5c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.189+0000 7fa387a15740 1 -- 172.21.15.73:0/297804666 >> 172.21.15.73:0/297804666 conn(0x55d757b58680 msgr2=0x55d757b58a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- 172.21.15.73:0/297804666 shutdown_connections 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- 172.21.15.73:0/297804666 wait complete. 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- start start 2024-06-06T07:06:45.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 Processor -- start 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b64200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b64740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d757b6e150 con 0x55d757b67350 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d757a49ae0 con 0x55d757b5ca90 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d757b6e2d0 con 0x55d757b5c170 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa387a15740 10 client.0 _ll_register_callbacks cb 0x55d757a829b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa37ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa37ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43206/0 (socket says 172.21.15.73:43206) 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa37ed9d640 1 -- 172.21.15.73:0/3873344011 learned_addr learned my addr 172.21.15.73:0/3873344011 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa37e59c640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b64200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.190+0000 7fa377fff640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b64740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 -- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 msgr2=0x55d757b64200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b64200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 -- 172.21.15.73:0/3873344011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 msgr2=0x55d757b64740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b64740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 -- 172.21.15.73:0/3873344011 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7577d4210 con 0x55d757b67350 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa378012d20 tx=0x7fa378004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1074b2a07924166d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37802a410 con 0x55d757b67350 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa37802a5b0 con 0x55d757b67350 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3680c0180 con 0x55d757b67350 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa378033050 con 0x55d757b67350 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa387a15740 10 client.16284 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:45.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d7578a0620 con 0x55d757b67350 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3680c0180 con 0x55d757b67350 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.191+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa37800f040 con 0x55d757b67350 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 10 client.16284 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 10 client.16284 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 20 client.16284 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757b7b540 0x55d757b7d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 4 client.16284 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d757ad3f10 con 0x55d757b7b540 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa37e59c640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757b7b540 0x55d757b7d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa37e59c640 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757b7b540 0x55d757b7d930 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15fadb5db8c4db5f server_cookie=a1f02da9b6cda2cb in_seq=0 out_seq=0 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.192+0000 7fa365ffb640 10 client.16284 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:45.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa365ffb640 1 -- 172.21.15.73:0/3873344011 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d757ad3f10 con 0x55d757b7b540 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa365ffb640 10 client.16284 handle_command_reply: tid=0 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 client.16284 shutdown 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 msgr2=0x55d757b64c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa378012d20 tx=0x7fa378004a40 comp rx=0 tx=0).stop 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 shutdown_connections 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757b7b540 0x55d757b7d930 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b64c80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b64740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa365ffb640 0 client.16284 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 --2- 172.21.15.73:0/3873344011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b64200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 >> 172.21.15.73:0/3873344011 conn(0x55d757b58680 msgr2=0x55d757b5e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 shutdown_connections 2024-06-06T07:06:45.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 1 -- 172.21.15.73:0/3873344011 wait complete. 2024-06-06T07:06:45.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.193+0000 7fa387a15740 20 client.16284 trim_cache size 0 max 16384 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 Processor -- start 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 -- start start 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b83220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b67350 0x55d757b83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801acbf0 con 0x55d757b5c170 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ae730 con 0x55d757b5ca90 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa387a15740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ad2c0 con 0x55d757b67350 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43212/0 (socket says 172.21.15.73:43212) 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 -- 172.21.15.73:0/2966611307 learned_addr learned my addr 172.21.15.73:0/2966611307 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:45.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b67350 0x55d757b83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa377fff640 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b83220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 -- 172.21.15.73:0/2966611307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b67350 msgr2=0x55d757b83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b67350 0x55d757b83760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 -- 172.21.15.73:0/2966611307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 msgr2=0x55d757b83220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b83220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 -- 172.21.15.73:0/2966611307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d757b7df90 con 0x55d757b5c170 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa37e59c640 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55d757b7e1c0 tx=0x7fa370005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89d0e8203d2dd228 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.210+0000 7fa367fff640 1 -- 172.21.15.73:0/2966611307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37000f960 con 0x55d757b5c170 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa367fff640 1 -- 172.21.15.73:0/2966611307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa370005020 con 0x55d757b5c170 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa367fff640 1 -- 172.21.15.73:0/2966611307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa370004b30 con 0x55d757b5c170 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- 172.21.15.73:0/2966611307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 msgr2=0x55d757b631e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x55d757b7e1c0 tx=0x7fa370005f90 comp rx=0 tx=0).stop 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- 172.21.15.73:0/2966611307 shutdown_connections 2024-06-06T07:06:45.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b67350 0x55d757b83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b83220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 --2- 172.21.15.73:0/2966611307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b5c170 0x55d757b631e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- 172.21.15.73:0/2966611307 >> 172.21.15.73:0/2966611307 conn(0x55d757b58680 msgr2=0x55d757b62ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- 172.21.15.73:0/2966611307 shutdown_connections 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- 172.21.15.73:0/2966611307 wait complete. 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.211+0000 7fa387a15740 1 -- start start 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 Processor -- start 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b73680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ae060 con 0x55d757b67350 2024-06-06T07:06:45.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801ad990 con 0x55d757b5ca90 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3801aabd0 con 0x55d757b5c170 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa387a15740 10 client.0 _ll_register_callbacks cb 0x7fa3780128f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43216/0 (socket says 172.21.15.73:43216) 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37e59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b73680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 -- 172.21.15.73:0/303879873 learned_addr learned my addr 172.21.15.73:0/303879873 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa377fff640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 -- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 msgr2=0x55d757b73680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b73680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 -- 172.21.15.73:0/303879873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 msgr2=0x55d757b73bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b73bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 -- 172.21.15.73:0/303879873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d757b61300 con 0x55d757b67350 2024-06-06T07:06:45.218 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa377fff640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa37ed9d640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa378020a10 tx=0x7fa378009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b56fac28977ba12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.212+0000 7fa365ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37803b030 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fa378030040 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa370005020 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa378048050 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 10 client.16290 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d757b72e80 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa370005020 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fa37800f070 con 0x55d757b67350 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 10 client.16290 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:45.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 20 client.16290 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757be63d0 0x55d757be87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 4 client.16290 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d757acdc20 con 0x55d757be63d0 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.213+0000 7fa37e59c640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757be63d0 0x55d757be87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.214+0000 7fa365ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.214+0000 7fa37e59c640 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757be63d0 0x55d757be87f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c009308237c9b7 server_cookie=ccf057553a2d2046 in_seq=0 out_seq=0 2024-06-06T07:06:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa365ffb640 1 -- 172.21.15.73:0/303879873 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d757acdc20 con 0x55d757be63d0 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa365ffb640 10 client.16290 handle_command_reply: tid=0 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 client.16290 shutdown 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 msgr2=0x55d757b60ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fa378020a10 tx=0x7fa378009710 comp rx=0 tx=0).stop 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 shutdown_connections 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d757be63d0 0x55d757be87f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d757b67350 0x55d757b60ad0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d757b5ca90 0x55d757b73bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa365ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 --2- 172.21.15.73:0/303879873 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d757b5c170 0x55d757b73680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 >> 172.21.15.73:0/303879873 conn(0x55d757b58680 msgr2=0x55d757b63360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 shutdown_connections 2024-06-06T07:06:45.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.237+0000 7fa387a15740 1 -- 172.21.15.73:0/303879873 wait complete. 2024-06-06T07:06:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa387a15740 20 client.16290 trim_cache size 0 max 16384 2024-06-06T07:06:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:06:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:06:45.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 msgr2=0x7fa36c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 0x7fa36c0a5260 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x55d757b59aa0 tx=0x7fa378009ae0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 msgr2=0x7fa3801986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fa368031440 tx=0x7fa368004270 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 shutdown_connections 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa36c0a2da0 0x7fa36c0a5260 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3801079a0 0x7fa38019ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa380102350 0x7fa3801986a0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 --2- 172.21.15.73:0/292718386 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa380101990 0x7fa380198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 >> 172.21.15.73:0/292718386 conn(0x7fa3800fd460 msgr2=0x7fa3800fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 shutdown_connections 2024-06-06T07:06:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:45.238+0000 7fa384c2a640 1 -- 172.21.15.73:0/292718386 wait complete. 2024-06-06T07:06:45.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 113404 2.65 MB/sec execute 112 sec latency 19.502 ms 2024-06-06T07:06:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:45 smithi073 ceph-mon[32303]: pgmap v165: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:45 smithi136 ceph-mon[35901]: pgmap v165: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:45.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:45 smithi170 ceph-mon[36108]: pgmap v165: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:46.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 113852 2.66 MB/sec execute 113 sec latency 14.753 ms 2024-06-06T07:06:47.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 114135 2.64 MB/sec execute 114 sec latency 14.511 ms 2024-06-06T07:06:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:47 smithi073 ceph-mon[32303]: pgmap v166: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:06:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:47 smithi136 ceph-mon[35901]: pgmap v166: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:06:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:47 smithi170 ceph-mon[36108]: pgmap v166: 129 pgs: 129 active+clean; 77 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:06:48.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 114602 2.63 MB/sec execute 115 sec latency 19.127 ms 2024-06-06T07:06:49.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 115104 2.64 MB/sec execute 116 sec latency 21.146 ms 2024-06-06T07:06:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:49 smithi073 ceph-mon[32303]: pgmap v167: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-06T07:06:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:49 smithi136 ceph-mon[35901]: pgmap v167: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-06T07:06:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:49 smithi170 ceph-mon[36108]: pgmap v167: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 217 op/s 2024-06-06T07:06:50.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 115693 2.62 MB/sec execute 117 sec latency 25.256 ms 2024-06-06T07:06:51.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 116151 2.62 MB/sec execute 118 sec latency 15.010 ms 2024-06-06T07:06:51.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:51 smithi073 ceph-mon[32303]: pgmap v168: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:51 smithi136 ceph-mon[35901]: pgmap v168: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:51.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:51 smithi170 ceph-mon[36108]: pgmap v168: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:52.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 116850 2.65 MB/sec execute 119 sec latency 13.352 ms 2024-06-06T07:06:53.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 117298 2.66 MB/sec execute 120 sec latency 10.752 ms 2024-06-06T07:06:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:53 smithi073 ceph-mon[32303]: pgmap v169: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:53 smithi136 ceph-mon[35901]: pgmap v169: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:53 smithi170 ceph-mon[36108]: pgmap v169: 129 pgs: 129 active+clean; 78 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1016 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-06T07:06:54.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 117549 2.65 MB/sec execute 121 sec latency 13.270 ms 2024-06-06T07:06:55.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 117963 2.63 MB/sec execute 122 sec latency 16.022 ms 2024-06-06T07:06:55.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:55 smithi073 ceph-mon[32303]: pgmap v170: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:06:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:55 smithi136 ceph-mon[35901]: pgmap v170: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:06:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:55 smithi170 ceph-mon[36108]: pgmap v170: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:06:56.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 118493 2.64 MB/sec execute 123 sec latency 17.086 ms 2024-06-06T07:06:57.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 118994 2.62 MB/sec execute 124 sec latency 22.716 ms 2024-06-06T07:06:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:57 smithi073 ceph-mon[32303]: pgmap v171: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-06T07:06:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:57 smithi136 ceph-mon[35901]: pgmap v171: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-06T07:06:57.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:57 smithi170 ceph-mon[36108]: pgmap v171: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-06T07:06:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 -- 172.21.15.73:0/908432333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c1008d0 msgr2=0x7fe16c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 --2- 172.21.15.73:0/908432333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c1008d0 0x7fe16c100d50 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fe160009930 tx=0x7fe16002a420 comp rx=0 tx=0).stop 2024-06-06T07:06:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 -- 172.21.15.73:0/908432333 shutdown_connections 2024-06-06T07:06:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 --2- 172.21.15.73:0/908432333 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c101290 0x7fe16c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 --2- 172.21.15.73:0/908432333 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c1008d0 0x7fe16c100d50 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 --2- 172.21.15.73:0/908432333 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c106890 0x7fe16c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 -- 172.21.15.73:0/908432333 >> 172.21.15.73:0/908432333 conn(0x7fe16c077a20 msgr2=0x7fe16c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 -- 172.21.15.73:0/908432333 shutdown_connections 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 -- 172.21.15.73:0/908432333 wait complete. 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.335+0000 7fe17095c640 1 Processor -- start 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 -- start start 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 0x7fe16c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c106890 0x7fe16c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c110b50 con 0x7fe16c101290 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1109d0 con 0x7fe16c1008d0 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe17095c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c110cd0 con 0x7fe16c106890 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53424/0 (socket says 172.21.15.73:53424) 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 0x7fe16c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe169d9b640 1 -- 172.21.15.73:0/4012364497 learned_addr learned my addr 172.21.15.73:0/4012364497 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c106890 0x7fe16c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 -- 172.21.15.73:0/4012364497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c106890 msgr2=0x7fe16c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c106890 0x7fe16c19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 -- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 msgr2=0x7fe16c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 -- 172.21.15.73:0/4012364497 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16c19d120 con 0x7fe16c1008d0 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe169d9b640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.336+0000 7fe16a59c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 0x7fe16c198180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e9f0 tx=0x7fe15c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.337+0000 7fe15b7fe640 1 -- 172.21.15.73:0/4012364497 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00ccc0 con 0x7fe16c1008d0 2024-06-06T07:06:58.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.337+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16c19d410 con 0x7fe16c1008d0 2024-06-06T07:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.337+0000 7fe15b7fe640 1 -- 172.21.15.73:0/4012364497 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe15c0108e0 con 0x7fe16c1008d0 2024-06-06T07:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.337+0000 7fe15b7fe640 1 -- 172.21.15.73:0/4012364497 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15c018880 con 0x7fe16c1008d0 2024-06-06T07:06:58.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.337+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe16c1a4ca0 con 0x7fe16c1008d0 2024-06-06T07:06:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.339+0000 7fe15b7fe640 1 -- 172.21.15.73:0/4012364497 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe15c010a90 con 0x7fe16c1008d0 2024-06-06T07:06:58.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.339+0000 7fe17365f740 1 Processor -- start 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 -- start start 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x559198667450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667990 0x559198676220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5591985cb4f0 con 0x559198676760 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559198569870 con 0x559198667070 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe17365f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591985c74b0 con 0x559198667990 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe15b7fe640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 0x7fe1540a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe16ad9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x559198667450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53438/0 (socket says 172.21.15.73:53438) 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 -- 172.21.15.73:0/3598105123 learned_addr learned my addr 172.21.15.73:0/3598105123 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 0x7fe1540a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe15b7fe640 1 -- 172.21.15.73:0/4012364497 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fe15c0cfac0 con 0x7fe16c1008d0 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe16a59c640 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667990 0x559198676220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 -- 172.21.15.73:0/3598105123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667990 msgr2=0x559198676220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667990 0x559198676220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 -- 172.21.15.73:0/3598105123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 msgr2=0x559198667450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x559198667450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 -- 172.21.15.73:0/3598105123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591985859b0 con 0x559198676760 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x559198667450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.340+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe160009230 tx=0x7fe160032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=985aafdcf23754b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 0x7fe1540a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe164005fd0 tx=0x7fe164009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe158ff9640 1 -- 172.21.15.73:0/3598105123 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160035040 con 0x559198676760 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe158ff9640 1 -- 172.21.15.73:0/3598105123 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1600317a0 con 0x559198676760 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe158ff9640 1 -- 172.21.15.73:0/3598105123 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1600047a0 con 0x559198676760 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe17365f740 1 -- 172.21.15.73:0/3598105123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 msgr2=0x559198678b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe17365f740 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe160009230 tx=0x7fe160032eb0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe17365f740 1 -- 172.21.15.73:0/3598105123 shutdown_connections 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.341+0000 7fe17365f740 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198678b50 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667990 0x559198676220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 --2- 172.21.15.73:0/3598105123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x559198667450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- 172.21.15.73:0/3598105123 >> 172.21.15.73:0/3598105123 conn(0x559198663680 msgr2=0x559198663a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- 172.21.15.73:0/3598105123 shutdown_connections 2024-06-06T07:06:58.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- 172.21.15.73:0/3598105123 wait complete. 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- start start 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 Processor -- start 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x5591986736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667990 0x559198673c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559198679330 con 0x559198676760 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559198554ae0 con 0x559198667990 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591986794b0 con 0x559198667070 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe17365f740 10 client.0 _ll_register_callbacks cb 0x55919858d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe16ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe16ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53444/0 (socket says 172.21.15.73:53444) 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe16ad9d640 1 -- 172.21.15.73:0/906098151 learned_addr learned my addr 172.21.15.73:0/906098151 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe169d9b640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667990 0x559198673c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.342+0000 7fe16a59c640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x5591986736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 -- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 msgr2=0x5591986736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x5591986736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 -- 172.21.15.73:0/906098151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667990 msgr2=0x559198673c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667990 0x559198673c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 -- 172.21.15.73:0/906098151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591982df210 con 0x559198676760 2024-06-06T07:06:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe164009e80 tx=0x7fe164004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c92529fc2ae9b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164020db0 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fe164004b20 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1600317a0 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe164004d20 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe17365f740 10 client.16302 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5591983ab620 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.343+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1600317a0 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe16401d070 con 0x559198676760 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 10 client.16302 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:58.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 10 client.16302 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 20 client.16302 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559198686990 0x559198688da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 4 client.16302 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591985def10 con 0x559198686990 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe16a59c640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559198686990 0x559198688da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe151ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.344+0000 7fe16a59c640 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559198686990 0x559198688da0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18e4579f0ad80c25 server_cookie=be9ad92455e342c1 in_seq=0 out_seq=0 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe151ffb640 1 -- 172.21.15.73:0/906098151 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5591985def10 con 0x559198686990 2024-06-06T07:06:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe151ffb640 10 client.16302 handle_command_reply: tid=0 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 client.16302 shutdown 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 msgr2=0x559198674160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fe164009e80 tx=0x7fe164004ab0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 shutdown_connections 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559198686990 0x559198688da0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198676760 0x559198674160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667990 0x559198673c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 --2- 172.21.15.73:0/906098151 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x5591986736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe151ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 >> 172.21.15.73:0/906098151 conn(0x559198663680 msgr2=0x5591986694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 shutdown_connections 2024-06-06T07:06:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 1 -- 172.21.15.73:0/906098151 wait complete. 2024-06-06T07:06:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.345+0000 7fe17365f740 20 client.16302 trim_cache size 0 max 16384 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.361+0000 7fe17365f740 1 Processor -- start 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.361+0000 7fe17365f740 1 -- start start 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x55919868e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198676760 0x5591986736a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1acba0 con 0x559198667990 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ae6e0 con 0x559198667070 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe17365f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ad270 con 0x559198676760 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53454/0 (socket says 172.21.15.73:53454) 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 -- 172.21.15.73:0/3802389629 learned_addr learned my addr 172.21.15.73:0/3802389629 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe16a59c640 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x55919868e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198676760 0x5591986736a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 -- 172.21.15.73:0/3802389629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198676760 msgr2=0x5591986736a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198676760 0x5591986736a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 -- 172.21.15.73:0/3802389629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 msgr2=0x55919868e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x55919868e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 -- 172.21.15.73:0/3802389629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559198683700 con 0x559198667990 2024-06-06T07:06:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe160004650 tx=0x7fe1600092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e21d77d075b56 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe153fff640 1 -- 172.21.15.73:0/3802389629 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160035040 con 0x559198667990 2024-06-06T07:06:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.362+0000 7fe153fff640 1 -- 172.21.15.73:0/3802389629 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe160031860 con 0x559198667990 2024-06-06T07:06:58.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe153fff640 1 -- 172.21.15.73:0/3802389629 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160046450 con 0x559198667990 2024-06-06T07:06:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 -- 172.21.15.73:0/3802389629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 msgr2=0x55919868ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe160004650 tx=0x7fe1600092c0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 -- 172.21.15.73:0/3802389629 shutdown_connections 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198676760 0x5591986736a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919868ebf0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 --2- 172.21.15.73:0/3802389629 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198667070 0x55919868e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 -- 172.21.15.73:0/3802389629 >> 172.21.15.73:0/3802389629 conn(0x559198663680 msgr2=0x559198671fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 -- 172.21.15.73:0/3802389629 shutdown_connections 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.363+0000 7fe17365f740 1 -- 172.21.15.73:0/3802389629 wait complete. 2024-06-06T07:06:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 -- start start 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 Processor -- start 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x559198679c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198676760 0x5591986ef670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ae010 con 0x559198667990 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1ad940 con 0x559198676760 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe16c1aab80 con 0x559198667070 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe17365f740 10 client.0 _ll_register_callbacks cb 0x7fe164009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53460/0 (socket says 172.21.15.73:53460) 2024-06-06T07:06:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 -- 172.21.15.73:0/3622279398 learned_addr learned my addr 172.21.15.73:0/3622279398 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe16a59c640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x559198679c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe16ad9d640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198676760 0x5591986ef670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 -- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 msgr2=0x559198679c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x559198679c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 -- 172.21.15.73:0/3622279398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198676760 msgr2=0x5591986ef670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198676760 0x5591986ef670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.364+0000 7fe169d9b640 1 -- 172.21.15.73:0/3622279398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591986efc60 con 0x559198667990 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe169d9b640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe160031440 tx=0x7fe160000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bf89b3c4d654e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe160004030 con 0x559198667990 2024-06-06T07:06:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fe160004980 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe160031860 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe160065050 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 10 client.16308 fetch_fsmap learned FSMap version 36 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5591986f0f70 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe160031860 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fe16005b030 con 0x559198667990 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 10 client.16308 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 10 client.16308 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:06:58.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 20 client.16308 populate_metadata read hostname 'smithi073' 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591986f11f0 0x5591986f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 4 client.16308 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.365+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5591985d8c20 con 0x5591986f11f0 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.366+0000 7fe16a59c640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591986f11f0 0x5591986f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.366+0000 7fe151ffb640 10 client.16308 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:58.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.366+0000 7fe16a59c640 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591986f11f0 0x5591986f35e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2286ccccf5c5c7a server_cookie=d7226ef2cebdd868 in_seq=0 out_seq=0 2024-06-06T07:06:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe151ffb640 1 -- 172.21.15.73:0/3622279398 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5591985d8c20 con 0x5591986f11f0 2024-06-06T07:06:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe151ffb640 10 client.16308 handle_command_reply: tid=0 2024-06-06T07:06:58.389 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 client.16308 shutdown 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 msgr2=0x55919867a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe160031440 tx=0x7fe160000bc0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 shutdown_connections 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591986f11f0 0x5591986f35e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559198676760 0x5591986ef670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559198667990 0x55919867a1c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 --2- 172.21.15.73:0/3622279398 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559198667070 0x559198679c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 >> 172.21.15.73:0/3622279398 conn(0x559198663680 msgr2=0x5591986736a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe151ffb640 0 client.16308 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:06:58.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 shutdown_connections 2024-06-06T07:06:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.387+0000 7fe17365f740 1 -- 172.21.15.73:0/3622279398 wait complete. 2024-06-06T07:06:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17365f740 20 client.16308 trim_cache size 0 max 16384 2024-06-06T07:06:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:06:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:06:58.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:06:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 msgr2=0x7fe1540a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 0x7fe1540a5260 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe164005fd0 tx=0x7fe164009500 comp rx=0 tx=0).stop 2024-06-06T07:06:58.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 msgr2=0x7fe16c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 0x7fe16c198180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe15c00e9f0 tx=0x7fe15c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 shutdown_connections 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1540a2da0 0x7fe1540a5260 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe16c106890 0x7fe16c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c101290 0x7fe16c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 --2- 172.21.15.73:0/4012364497 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe16c1008d0 0x7fe16c198180 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 >> 172.21.15.73:0/4012364497 conn(0x7fe16c077a20 msgr2=0x7fe16c0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 shutdown_connections 2024-06-06T07:06:58.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:06:58.388+0000 7fe17095c640 1 -- 172.21.15.73:0/4012364497 wait complete. 2024-06-06T07:06:58.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 119523 2.62 MB/sec execute 125 sec latency 21.221 ms 2024-06-06T07:06:59.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 120258 2.64 MB/sec execute 126 sec latency 15.757 ms 2024-06-06T07:06:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:06:59 smithi073 ceph-mon[32303]: pgmap v172: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:06:59 smithi136 ceph-mon[35901]: pgmap v172: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:06:59.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:06:59 smithi170 ceph-mon[36108]: pgmap v172: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.8 MiB/s wr, 216 op/s 2024-06-06T07:07:00.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 120617 2.66 MB/sec execute 127 sec latency 15.966 ms 2024-06-06T07:07:01.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 120957 2.65 MB/sec execute 128 sec latency 17.492 ms 2024-06-06T07:07:01.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:01 smithi073 ceph-mon[32303]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:01 smithi136 ceph-mon[35901]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:01 smithi170 ceph-mon[36108]: pgmap v173: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:02.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 121232 2.63 MB/sec execute 129 sec latency 13.218 ms 2024-06-06T07:07:03.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 121693 2.62 MB/sec execute 130 sec latency 27.208 ms 2024-06-06T07:07:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:03 smithi073 ceph-mon[32303]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:03 smithi136 ceph-mon[35901]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:03.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:03 smithi170 ceph-mon[36108]: pgmap v174: 129 pgs: 129 active+clean; 80 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 143 op/s 2024-06-06T07:07:04.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 122209 2.63 MB/sec execute 131 sec latency 18.493 ms 2024-06-06T07:07:05.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 122776 2.61 MB/sec execute 132 sec latency 21.352 ms 2024-06-06T07:07:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:05 smithi073 ceph-mon[32303]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-06-06T07:07:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:05 smithi136 ceph-mon[35901]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-06-06T07:07:05.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:05 smithi170 ceph-mon[36108]: pgmap v175: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 231 op/s 2024-06-06T07:07:06.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 123232 2.61 MB/sec execute 133 sec latency 15.676 ms 2024-06-06T07:07:07.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 123925 2.64 MB/sec execute 134 sec latency 15.298 ms 2024-06-06T07:07:07.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:07 smithi073 ceph-mon[32303]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-06T07:07:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:07 smithi136 ceph-mon[35901]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-06T07:07:07.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:07 smithi170 ceph-mon[36108]: pgmap v176: 129 pgs: 129 active+clean; 82 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-06T07:07:08.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 124329 2.64 MB/sec execute 135 sec latency 15.546 ms 2024-06-06T07:07:09.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 124610 2.63 MB/sec execute 136 sec latency 16.157 ms 2024-06-06T07:07:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:09 smithi073 ceph-mon[32303]: pgmap v177: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-06-06T07:07:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:09 smithi136 ceph-mon[35901]: pgmap v177: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-06-06T07:07:09.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:09 smithi170 ceph-mon[36108]: pgmap v177: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 230 op/s 2024-06-06T07:07:10.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 124988 2.62 MB/sec execute 137 sec latency 17.276 ms 2024-06-06T07:07:11.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 125467 2.61 MB/sec execute 138 sec latency 18.274 ms 2024-06-06T07:07:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 -- 172.21.15.73:0/1649097104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 msgr2=0x7fec20106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1649097104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec20106a80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fec10009930 tx=0x7fec1002a420 comp rx=0 tx=0).stop 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 -- 172.21.15.73:0/1649097104 shutdown_connections 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1649097104 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20101050 0x7fec2010d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1649097104 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20100690 0x7fec20100b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1649097104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec20106a80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 -- 172.21.15.73:0/1649097104 >> 172.21.15.73:0/1649097104 conn(0x7fec200fc1c0 msgr2=0x7fec200fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 -- 172.21.15.73:0/1649097104 shutdown_connections 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.488+0000 7fec1ffff640 1 -- 172.21.15.73:0/1649097104 wait complete. 2024-06-06T07:07:11.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.489+0000 7fec1ffff640 1 Processor -- start 2024-06-06T07:07:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.489+0000 7fec1ffff640 1 -- start start 2024-06-06T07:07:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 0x7fec20198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec20110960 con 0x7fec201066a0 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec201107e0 con 0x7fec20101050 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1ffff640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec20110ae0 con 0x7fec20100690 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 0x7fec20198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:44246/0 (socket says 172.21.15.73:44246) 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 -- 172.21.15.73:0/1273711336 learned_addr learned my addr 172.21.15.73:0/1273711336 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59734/0 (socket says 172.21.15.73:59734) 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 -- 172.21.15.73:0/1273711336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 msgr2=0x7fec20198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 0x7fec20198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 -- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 msgr2=0x7fec2019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 -- 172.21.15.73:0/1273711336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2019cfe0 con 0x7fec20100690 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 0x7fec20198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.490+0000 7fec1effd640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec10004770 tx=0x7fec100042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.491+0000 7fec07fff640 1 -- 172.21.15.73:0/1273711336 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1003f070 con 0x7fec20100690 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.491+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2019d270 con 0x7fec20100690 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.491+0000 7fec07fff640 1 -- 172.21.15.73:0/1273711336 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec1003b460 con 0x7fec20100690 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.491+0000 7fec07fff640 1 -- 172.21.15.73:0/1273711336 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10035070 con 0x7fec20100690 2024-06-06T07:07:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.491+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a4b60 con 0x7fec20100690 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.493+0000 7fec07fff640 1 -- 172.21.15.73:0/1273711336 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fec1003b610 con 0x7fec20100690 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.493+0000 7fec285ed740 1 Processor -- start 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.493+0000 7fec285ed740 1 -- start start 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 0x559ffbe66930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ffbdbd4f0 con 0x559ffbe59c80 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559ffbd5b870 con 0x559ffbe59360 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec285ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559ffbdb94b0 con 0x559ffbe64540 2024-06-06T07:07:11.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec07fff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 0x7fec080a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59748/0 (socket says 172.21.15.73:59748) 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1effd640 1 -- 172.21.15.73:0/2355962269 learned_addr learned my addr 172.21.15.73:0/2355962269 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 0x559ffbe66930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 0x7fec080a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe59740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec07fff640 1 -- 172.21.15.73:0/1273711336 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fec100efae0 con 0x7fec20100690 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 -- 172.21.15.73:0/2355962269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 msgr2=0x559ffbe59740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe59740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 -- 172.21.15.73:0/2355962269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 msgr2=0x559ffbe64000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 -- 172.21.15.73:0/2355962269 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffbd779b0 con 0x559ffbe64540 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1effd640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 0x7fec080a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec18009fd0 tx=0x7fec18012570 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.494+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 0x559ffbe66930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec14005d00 tx=0x7fec1400bf90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70fc74bbd8b7ae7c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec057fa640 1 -- 172.21.15.73:0/2355962269 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14011700 con 0x559ffbe64540 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec057fa640 1 -- 172.21.15.73:0/2355962269 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec14011d50 con 0x559ffbe64540 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec057fa640 1 -- 172.21.15.73:0/2355962269 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1401cd30 con 0x559ffbe64540 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 -- 172.21.15.73:0/2355962269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 msgr2=0x559ffbe66930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 0x559ffbe66930 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fec14005d00 tx=0x7fec1400bf90 comp rx=0 tx=0).stop 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 -- 172.21.15.73:0/2355962269 shutdown_connections 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe64540 0x559ffbe66930 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe59c80 0x559ffbe64000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 --2- 172.21.15.73:0/2355962269 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe59740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 -- 172.21.15.73:0/2355962269 >> 172.21.15.73:0/2355962269 conn(0x559ffbe55870 msgr2=0x559ffbe55c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 -- 172.21.15.73:0/2355962269 shutdown_connections 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.495+0000 7fec285ed740 1 -- 172.21.15.73:0/2355962269 wait complete. 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 -- start start 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 Processor -- start 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe63f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe619c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559ffbe67170 con 0x559ffbe64540 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559ffbd46ae0 con 0x559ffbe59360 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559ffbe672f0 con 0x559ffbe59c80 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec285ed740 10 client.0 _ll_register_callbacks cb 0x559ffbd7f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec1e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec1e7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:44266/0 (socket says 172.21.15.73:44266) 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec1effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe63f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec1e7fc640 1 -- 172.21.15.73:0/3942513473 learned_addr learned my addr 172.21.15.73:0/3942513473 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.496+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 -- 172.21.15.73:0/3942513473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 msgr2=0x559ffbe63f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe63f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 -- 172.21.15.73:0/3942513473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 msgr2=0x559ffbe619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe619c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 -- 172.21.15.73:0/3942513473 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffbad1210 con 0x559ffbe59c80 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1effd640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec1e7fc640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec1401c930 tx=0x7fec1401c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bd9b4509434010c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1400bc50 con 0x559ffbe59c80 2024-06-06T07:07:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fec1401cca0 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec14011d50 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fec14031050 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec285ed740 10 client.25855 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.497+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559ffbb9d620 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec14011d50 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fec140035f0 con 0x559ffbe59c80 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 10 client.25855 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 10 client.25855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 20 client.25855 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbe7ce70 0x559ffbe7f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 4 client.25855 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ffbdd0f10 con 0x559ffbe7ce70 2024-06-06T07:07:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec1effd640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbe7ce70 0x559ffbe7f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec1effd640 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbe7ce70 0x559ffbe7f260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e3db1e5b00006c server_cookie=ece57c79690becc1 in_seq=0 out_seq=0 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.498+0000 7fec02ff5640 10 client.25855 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec02ff5640 1 -- 172.21.15.73:0/3942513473 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559ffbdd0f10 con 0x559ffbe7ce70 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec02ff5640 10 client.25855 handle_command_reply: tid=0 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 client.25855 shutdown 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 msgr2=0x559ffbe61480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec1401c930 tx=0x7fec1401c6b0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 shutdown_connections 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbe7ce70 0x559ffbe7f260 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59c80 0x559ffbe61480 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec02ff5640 0 client.25855 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 --2- 172.21.15.73:0/3942513473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59360 0x559ffbe63f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 >> 172.21.15.73:0/3942513473 conn(0x559ffbe55870 msgr2=0x559ffbe5b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 shutdown_connections 2024-06-06T07:07:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 1 -- 172.21.15.73:0/3942513473 wait complete. 2024-06-06T07:07:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.499+0000 7fec285ed740 20 client.25855 trim_cache size 0 max 16384 2024-06-06T07:07:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 Processor -- start 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 -- start start 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe5bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec201aca30 con 0x559ffbe64540 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec201ae570 con 0x559ffbe59c80 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec285ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec201ad100 con 0x559ffbe59360 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe5bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59780/0 (socket says 172.21.15.73:59780) 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1f7fe640 1 -- 172.21.15.73:0/2876335008 learned_addr learned my addr 172.21.15.73:0/2876335008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1f7fe640 1 -- 172.21.15.73:0/2876335008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 msgr2=0x559ffbe5bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.516+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe5bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec1f7fe640 1 -- 172.21.15.73:0/2876335008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 msgr2=0x559ffbe849f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe849f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec1f7fe640 1 -- 172.21.15.73:0/2876335008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ffbe4fce0 con 0x559ffbe64540 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec1effd640 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe5bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fec18010f40 tx=0x7fec1801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9957155fe348000 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec04ff9640 1 -- 172.21.15.73:0/2876335008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1802f070 con 0x559ffbe64540 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec04ff9640 1 -- 172.21.15.73:0/2876335008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180129c0 con 0x559ffbe64540 2024-06-06T07:07:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.517+0000 7fec04ff9640 1 -- 172.21.15.73:0/2876335008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec18020400 con 0x559ffbe64540 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- 172.21.15.73:0/2876335008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 msgr2=0x559ffbe84f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fec18010f40 tx=0x7fec1801f000 comp rx=0 tx=0).stop 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- 172.21.15.73:0/2876335008 shutdown_connections 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe84f30 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe849f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- 172.21.15.73:0/2876335008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe5bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- 172.21.15.73:0/2876335008 >> 172.21.15.73:0/2876335008 conn(0x559ffbe55870 msgr2=0x559ffbe61440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- 172.21.15.73:0/2876335008 shutdown_connections 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- 172.21.15.73:0/2876335008 wait complete. 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- start start 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 Processor -- start 2024-06-06T07:07:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe791e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe79720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec201adea0 con 0x559ffbe64540 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fec201ad7d0 con 0x559ffbe59c80 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fec201aaa10 con 0x559ffbe59360 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.518+0000 7fec285ed740 10 client.0 _ll_register_callbacks cb 0x7fec100bbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59784/0 (socket says 172.21.15.73:59784) 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1e7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe79720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 -- 172.21.15.73:0/1894550809 learned_addr learned my addr 172.21.15.73:0/1894550809 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1effd640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe791e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 -- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 msgr2=0x559ffbe791e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe791e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 -- 172.21.15.73:0/1894550809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 msgr2=0x559ffbe79720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe79720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 -- 172.21.15.73:0/1894550809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ffbe5dc00 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec1f7fe640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec1801ec80 tx=0x7fec18005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e47c0c5387a1de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1802f070 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec180129c0 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fec1802a410 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fec1802a610 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec285ed740 10 client.16326 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.519+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559ffbee6fc0 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec180129c0 con 0x559ffbe64540 2024-06-06T07:07:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fec1801f070 con 0x559ffbe64540 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 10 client.16326 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 10 client.16326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 20 client.16326 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbee76f0 0x559ffbee9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 4 client.16326 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ffbdcac20 con 0x559ffbee76f0 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec1effd640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbee76f0 0x559ffbee9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec1effd640 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbee76f0 0x559ffbee9b10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a37f7a757e8fb86 server_cookie=8ff4e900f0093c7f in_seq=0 out_seq=0 2024-06-06T07:07:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.520+0000 7fec02ff5640 10 client.16326 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec02ff5640 1 -- 172.21.15.73:0/1894550809 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559ffbdcac20 con 0x559ffbee76f0 2024-06-06T07:07:11.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec02ff5640 10 client.16326 handle_command_reply: tid=0 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 client.16326 shutdown 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 msgr2=0x559ffbe79c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fec1801ec80 tx=0x7fec18005fd0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 shutdown_connections 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559ffbee76f0 0x559ffbee9b10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559ffbe64540 0x559ffbe79c80 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec02ff5640 0 client.16326 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559ffbe59c80 0x559ffbe79720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 --2- 172.21.15.73:0/1894550809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559ffbe59360 0x559ffbe791e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 >> 172.21.15.73:0/1894550809 conn(0x559ffbe55870 msgr2=0x559ffbe54140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 shutdown_connections 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.544+0000 7fec285ed740 1 -- 172.21.15.73:0/1894550809 wait complete. 2024-06-06T07:07:11.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec285ed740 20 client.16326 trim_cache size 0 max 16384 2024-06-06T07:07:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:07:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:07:11.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 msgr2=0x7fec080a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 0x7fec080a51b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fec18009fd0 tx=0x7fec18012570 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 msgr2=0x7fec20197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fec10004770 tx=0x7fec100042f0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 shutdown_connections 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec201066a0 0x7fec2019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fec20101050 0x7fec20198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fec080a2cf0 0x7fec080a51b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 --2- 172.21.15.73:0/1273711336 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fec20100690 0x7fec20197ff0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 >> 172.21.15.73:0/1273711336 conn(0x7fec200fc1c0 msgr2=0x7fec20069210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:11.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 shutdown_connections 2024-06-06T07:07:11.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:11.545+0000 7fec1ffff640 1 -- 172.21.15.73:0/1273711336 wait complete. 2024-06-06T07:07:11.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:11 smithi073 ceph-mon[32303]: pgmap v178: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:11 smithi136 ceph-mon[35901]: pgmap v178: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:11.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:11 smithi170 ceph-mon[36108]: pgmap v178: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:12.439 INFO:tasks.workunit.client.0.smithi073.stdout: 1 125998 2.62 MB/sec execute 139 sec latency 33.233 ms 2024-06-06T07:07:13.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 126559 2.61 MB/sec execute 140 sec latency 21.974 ms 2024-06-06T07:07:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:13 smithi073 ceph-mon[32303]: pgmap v179: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:13 smithi136 ceph-mon[35901]: pgmap v179: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:13 smithi170 ceph-mon[36108]: pgmap v179: 129 pgs: 129 active+clean; 86 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-06T07:07:14.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 127156 2.63 MB/sec execute 141 sec latency 15.885 ms 2024-06-06T07:07:15.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 127579 2.63 MB/sec execute 142 sec latency 17.445 ms 2024-06-06T07:07:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:15 smithi073 ceph-mon[32303]: pgmap v180: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-06-06T07:07:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:15 smithi136 ceph-mon[35901]: pgmap v180: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-06-06T07:07:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:15 smithi170 ceph-mon[36108]: pgmap v180: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 233 op/s 2024-06-06T07:07:16.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 128022 2.64 MB/sec execute 143 sec latency 15.621 ms 2024-06-06T07:07:17.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 128306 2.62 MB/sec execute 144 sec latency 12.890 ms 2024-06-06T07:07:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:17 smithi073 ceph-mon[32303]: pgmap v181: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:17 smithi136 ceph-mon[35901]: pgmap v181: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:17 smithi170 ceph-mon[36108]: pgmap v181: 129 pgs: 129 active+clean; 83 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:18.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 128742 2.61 MB/sec execute 145 sec latency 18.065 ms 2024-06-06T07:07:19.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 129246 2.62 MB/sec execute 146 sec latency 17.947 ms 2024-06-06T07:07:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:19 smithi073 ceph-mon[32303]: pgmap v182: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:19 smithi136 ceph-mon[35901]: pgmap v182: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:19.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:19 smithi170 ceph-mon[36108]: pgmap v182: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:20.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 129799 2.61 MB/sec execute 147 sec latency 23.780 ms 2024-06-06T07:07:21.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 130303 2.60 MB/sec execute 148 sec latency 20.645 ms 2024-06-06T07:07:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:21 smithi073 ceph-mon[32303]: pgmap v183: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:21 smithi136 ceph-mon[35901]: pgmap v183: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:21.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:21 smithi170 ceph-mon[36108]: pgmap v183: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:22.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 131019 2.63 MB/sec execute 149 sec latency 11.892 ms 2024-06-06T07:07:23.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 131441 2.64 MB/sec execute 150 sec latency 17.803 ms 2024-06-06T07:07:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:23 smithi073 ceph-mon[32303]: pgmap v184: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:23 smithi136 ceph-mon[35901]: pgmap v184: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:23 smithi170 ceph-mon[36108]: pgmap v184: 129 pgs: 129 active+clean; 88 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-06T07:07:24.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 131722 2.63 MB/sec execute 151 sec latency 14.760 ms 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 -- 172.21.15.73:0/749789729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c41008d0 msgr2=0x7fc1c4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 --2- 172.21.15.73:0/749789729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c41008d0 0x7fc1c4100d50 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009930 tx=0x7fc1b402a420 comp rx=0 tx=0).stop 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 -- 172.21.15.73:0/749789729 shutdown_connections 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 --2- 172.21.15.73:0/749789729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c4101290 0x7fc1c410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 --2- 172.21.15.73:0/749789729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c41008d0 0x7fc1c4100d50 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 --2- 172.21.15.73:0/749789729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4106890 0x7fc1c4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 -- 172.21.15.73:0/749789729 >> 172.21.15.73:0/749789729 conn(0x7fc1c4077a20 msgr2=0x7fc1c4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 -- 172.21.15.73:0/749789729 shutdown_connections 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.642+0000 7fc1cc314640 1 -- 172.21.15.73:0/749789729 wait complete. 2024-06-06T07:07:24.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 Processor -- start 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 -- start start 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c41008d0 0x7fc1c4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4101290 0x7fc1c4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4110b50 con 0x7fc1c4106890 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41109d0 con 0x7fc1c4101290 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1cc314640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c4110cd0 con 0x7fc1c41008d0 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c41008d0 0x7fc1c4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51950/0 (socket says 172.21.15.73:51950) 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2536386792 learned_addr learned my addr 172.21.15.73:0/2536386792 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4101290 0x7fc1c4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2536386792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c41008d0 msgr2=0x7fc1c4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c41008d0 0x7fc1c4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2536386792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4101290 msgr2=0x7fc1c4199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4101290 0x7fc1c4199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2536386792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c41a4590 con 0x7fc1c4106890 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.643+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00eb10 tx=0x7fc1bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.644+0000 7fc1bb7fe640 1 -- 172.21.15.73:0/2536386792 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc004b20 con 0x7fc1c4106890 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.644+0000 7fc1bb7fe640 1 -- 172.21.15.73:0/2536386792 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1bc004cc0 con 0x7fc1c4106890 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.644+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c41a4760 con 0x7fc1c4106890 2024-06-06T07:07:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.644+0000 7fc1bb7fe640 1 -- 172.21.15.73:0/2536386792 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc010690 con 0x7fc1c4106890 2024-06-06T07:07:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.644+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c41a4c20 con 0x7fc1c4106890 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.646+0000 7fc1bb7fe640 1 -- 172.21.15.73:0/2536386792 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc1bc010830 con 0x7fc1c4106890 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 Processor -- start 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 -- start start 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1bb7fe640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 0x7fc1b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9ad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9afb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5650bb9024f0 con 0x5650bb99e050 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5650bb8a0870 con 0x5650bb99e970 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1cf027740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650bb8fe4b0 con 0x5650bb9ad760 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 0x7fc1b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51960/0 (socket says 172.21.15.73:51960) 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/2658009500 learned_addr learned my addr 172.21.15.73:0/2658009500 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1bb7fe640 1 -- 172.21.15.73:0/2536386792 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc1bc0cff40 con 0x7fc1c4106890 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.647+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9afb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/2658009500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 msgr2=0x5650bb9afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9afb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/2658009500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 msgr2=0x5650bb9ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/2658009500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bb8bc9b0 con 0x5650bb99e050 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 0x7fc1b00a5260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b4031ea0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000bc80 tx=0x7fc1c00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0b01a311051572e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1b8bfe640 1 -- 172.21.15.73:0/2658009500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001c480 con 0x5650bb99e050 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1b8bfe640 1 -- 172.21.15.73:0/2658009500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c001cad0 con 0x5650bb99e050 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.648+0000 7fc1b8bfe640 1 -- 172.21.15.73:0/2658009500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c00119a0 con 0x5650bb99e050 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- 172.21.15.73:0/2658009500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 msgr2=0x5650bb99e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000bc80 tx=0x7fc1c00107a0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- 172.21.15.73:0/2658009500 shutdown_connections 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9afb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2658009500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb99e430 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- 172.21.15.73:0/2658009500 >> 172.21.15.73:0/2658009500 conn(0x5650bb99a680 msgr2=0x5650bb99aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- 172.21.15.73:0/2658009500 shutdown_connections 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- 172.21.15.73:0/2658009500 wait complete. 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 -- start start 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.649+0000 7fc1cf027740 1 Processor -- start 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9aa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9aac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5650bb9b0390 con 0x5650bb9ad760 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5650bb88bae0 con 0x5650bb99e970 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650bb9b0510 con 0x5650bb99e050 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 10 client.0 _ll_register_callbacks cb 0x5650bb8c49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:24.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51972/0 (socket says 172.21.15.73:51972) 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2815091947 learned_addr learned my addr 172.21.15.73:0/2815091947 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9aac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 msgr2=0x5650bb9aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2815091947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 msgr2=0x5650bb9aac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9aac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/2815091947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bb616210 con 0x5650bb9ad760 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1c98af640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc09bdb0 tx=0x7fc1bc09ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3105d53a27cd0659 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1adffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc0794f0 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c001cad0 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.650+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc1bc098720 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc09c7b0 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 10 client.16338 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5650bb6e2620 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c001cad0 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc1bc0a00c0 con 0x5650bb9ad760 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 10 client.16338 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 10 client.16338 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 20 client.16338 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bb9bda00 0x5650bb9bfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 4 client.16338 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650bb915f10 con 0x5650bb9bda00 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.651+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bb9bda00 0x5650bb9bfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bb9bda00 0x5650bb9bfdf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9ea299870fcd1a8 server_cookie=c2264c18b805e038 in_seq=0 out_seq=0 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1adffb640 10 client.16338 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1adffb640 1 -- 172.21.15.73:0/2815091947 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650bb915f10 con 0x5650bb9bda00 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1adffb640 10 client.16338 handle_command_reply: tid=0 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 client.16338 shutdown 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 msgr2=0x5650bb9ab1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc09bdb0 tx=0x7fc1bc09ba10 comp rx=0 tx=0).stop 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 shutdown_connections 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bb9bda00 0x5650bb9bfdf0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9ab1c0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9aac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 --2- 172.21.15.73:0/2815091947 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9aa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1adffb640 0 client.16338 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 >> 172.21.15.73:0/2815091947 conn(0x5650bb99a680 msgr2=0x5650bb9a0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.652+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 shutdown_connections 2024-06-06T07:07:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.653+0000 7fc1cf027740 1 -- 172.21.15.73:0/2815091947 wait complete. 2024-06-06T07:07:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.653+0000 7fc1cf027740 20 client.16338 trim_cache size 0 max 16384 2024-06-06T07:07:24.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 Processor -- start 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 -- start start 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9c5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9c5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41acb60 con 0x5650bb99e050 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ae6a0 con 0x5650bb99e970 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1cf027740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ad230 con 0x5650bb9ad760 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51978/0 (socket says 172.21.15.73:51978) 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/188863611 learned_addr learned my addr 172.21.15.73:0/188863611 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9c5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1c98af640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/188863611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 msgr2=0x5650bb9c5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9c5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/188863611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 msgr2=0x5650bb9c5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9c5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 -- 172.21.15.73:0/188863611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650bb99e7a0 con 0x5650bb99e050 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1c98af640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.671+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005d00 tx=0x7fc1c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4ccf52324c8999 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1affff640 1 -- 172.21.15.73:0/188863611 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0005040 con 0x5650bb99e050 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1affff640 1 -- 172.21.15.73:0/188863611 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c0030070 con 0x5650bb99e050 2024-06-06T07:07:24.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1affff640 1 -- 172.21.15.73:0/188863611 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0004cd0 con 0x5650bb99e050 2024-06-06T07:07:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 -- 172.21.15.73:0/188863611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 msgr2=0x5650bb999660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005d00 tx=0x7fc1c0000bc0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 -- 172.21.15.73:0/188863611 shutdown_connections 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb9ad760 0x5650bb9c5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9c5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 --2- 172.21.15.73:0/188863611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb99e050 0x5650bb999660 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 -- 172.21.15.73:0/188863611 >> 172.21.15.73:0/188863611 conn(0x5650bb99a680 msgr2=0x5650bb998f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 -- 172.21.15.73:0/188863611 shutdown_connections 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.672+0000 7fc1cf027740 1 -- 172.21.15.73:0/188863611 wait complete. 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 -- start start 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 Processor -- start 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9b5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9b5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41adfd0 con 0x5650bb9ad760 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41ad900 con 0x5650bb99e970 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c41aab40 con 0x5650bb99e050 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1cf027740 10 client.0 _ll_register_callbacks cb 0x7fc1bc004f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca0b0640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9b5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51990/0 (socket says 172.21.15.73:51990) 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/3189553186 learned_addr learned my addr 172.21.15.73:0/3189553186 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1c98af640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9b5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 msgr2=0x5650bb9b5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9b5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/3189553186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 msgr2=0x5650bb9b5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9b5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca8b1640 1 -- 172.21.15.73:0/3189553186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650bb9a4ca0 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.673+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1ca8b1640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a0560 tx=0x7fc1bc00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab1e1117dd0b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc079280 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7fc1bc079420 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0030070 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc1bc09b500 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 10 client.16344 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5650bb9a26c0 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c0030070 con 0x5650bb9ad760 2024-06-06T07:07:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc1bc097e00 con 0x5650bb9ad760 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 10 client.16344 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 20 client.16344 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bba2d320 0x5650bba2f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 4 client.16344 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.674+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650bb90fc20 con 0x5650bba2d320 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.675+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bba2d320 0x5650bba2f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.675+0000 7fc1ca0b0640 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bba2d320 0x5650bba2f740 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d85014a001b7449f server_cookie=1964290597f207c9 in_seq=0 out_seq=0 2024-06-06T07:07:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.675+0000 7fc1adffb640 10 client.16344 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1adffb640 1 -- 172.21.15.73:0/3189553186 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5650bb90fc20 con 0x5650bba2d320 2024-06-06T07:07:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1adffb640 10 client.16344 handle_command_reply: tid=0 2024-06-06T07:07:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 client.16344 shutdown 2024-06-06T07:07:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 msgr2=0x5650bb9a4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc0a0560 tx=0x7fc1bc00efd0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 shutdown_connections 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650bba2d320 0x5650bba2f740 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650bb9ad760 0x5650bb9a4470 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650bb99e970 0x5650bb9b5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 --2- 172.21.15.73:0/3189553186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650bb99e050 0x5650bb9b5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 >> 172.21.15.73:0/3189553186 conn(0x5650bb99a680 msgr2=0x5650bb9aaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1adffb640 0 client.16344 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 shutdown_connections 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 1 -- 172.21.15.73:0/3189553186 wait complete. 2024-06-06T07:07:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.697+0000 7fc1cf027740 20 client.16344 trim_cache size 0 max 16384 2024-06-06T07:07:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:07:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:07:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 msgr2=0x7fc1b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 0x7fc1b00a5260 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4009900 tx=0x7fc1b4031ea0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 msgr2=0x7fc1c41a4050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00eb10 tx=0x7fc1bc00c2d0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 shutdown_connections 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc1c4106890 0x7fc1c41a4050 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc1c4101290 0x7fc1c4199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc1b00a2da0 0x7fc1b00a5260 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 --2- 172.21.15.73:0/2536386792 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc1c41008d0 0x7fc1c4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 >> 172.21.15.73:0/2536386792 conn(0x7fc1c4077a20 msgr2=0x7fc1c4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:24.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 shutdown_connections 2024-06-06T07:07:24.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:24.698+0000 7fc1cc314640 1 -- 172.21.15.73:0/2536386792 wait complete. 2024-06-06T07:07:25.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 132103 2.61 MB/sec execute 152 sec latency 21.570 ms 2024-06-06T07:07:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:25 smithi073 ceph-mon[32303]: pgmap v185: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:25 smithi136 ceph-mon[35901]: pgmap v185: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:25.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:25 smithi170 ceph-mon[36108]: pgmap v185: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:26.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 132568 2.61 MB/sec execute 153 sec latency 15.676 ms 2024-06-06T07:07:27.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 133123 2.61 MB/sec execute 154 sec latency 27.018 ms 2024-06-06T07:07:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:27 smithi073 ceph-mon[32303]: pgmap v186: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-06-06T07:07:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:27 smithi136 ceph-mon[35901]: pgmap v186: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-06-06T07:07:27.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:27 smithi170 ceph-mon[36108]: pgmap v186: 129 pgs: 129 active+clean; 89 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 1.4 MiB/s wr, 137 op/s 2024-06-06T07:07:28.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 133674 2.60 MB/sec execute 155 sec latency 18.622 ms 2024-06-06T07:07:29.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 134263 2.62 MB/sec execute 156 sec latency 15.473 ms 2024-06-06T07:07:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:29 smithi073 ceph-mon[32303]: pgmap v187: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:07:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:29 smithi136 ceph-mon[35901]: pgmap v187: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:07:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:29 smithi170 ceph-mon[36108]: pgmap v187: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:07:30.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 134774 2.64 MB/sec execute 157 sec latency 11.903 ms 2024-06-06T07:07:31.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 135135 2.63 MB/sec execute 158 sec latency 14.734 ms 2024-06-06T07:07:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:31 smithi073 ceph-mon[32303]: pgmap v188: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:31 smithi136 ceph-mon[35901]: pgmap v188: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:31 smithi170 ceph-mon[36108]: pgmap v188: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:32.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 135413 2.61 MB/sec execute 159 sec latency 12.187 ms 2024-06-06T07:07:33.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 135863 2.61 MB/sec execute 160 sec latency 15.801 ms 2024-06-06T07:07:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:33 smithi073 ceph-mon[32303]: pgmap v189: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:33 smithi136 ceph-mon[35901]: pgmap v189: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:33 smithi170 ceph-mon[36108]: pgmap v189: 129 pgs: 129 active+clean; 86 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 998 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:07:34.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 136357 2.61 MB/sec execute 161 sec latency 18.043 ms 2024-06-06T07:07:35.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 136961 2.60 MB/sec execute 162 sec latency 21.679 ms 2024-06-06T07:07:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:35 smithi073 ceph-mon[32303]: pgmap v190: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:35 smithi136 ceph-mon[35901]: pgmap v190: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:35 smithi170 ceph-mon[36108]: pgmap v190: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:36.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 137402 2.60 MB/sec execute 163 sec latency 20.574 ms 2024-06-06T07:07:37.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 138121 2.62 MB/sec execute 164 sec latency 14.803 ms 2024-06-06T07:07:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:37 smithi073 ceph-mon[32303]: pgmap v191: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:37 smithi136 ceph-mon[35901]: pgmap v191: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 -- 172.21.15.73:0/1895715651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81079d0 msgr2=0x7f11a8107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 --2- 172.21.15.73:0/1895715651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81079d0 0x7f11a8107db0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1198009930 tx=0x7f119802a420 comp rx=0 tx=0).stop 2024-06-06T07:07:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 -- 172.21.15.73:0/1895715651 shutdown_connections 2024-06-06T07:07:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 --2- 172.21.15.73:0/1895715651 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 0x7f11a810e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 --2- 172.21.15.73:0/1895715651 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81019c0 0x7f11a8101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 --2- 172.21.15.73:0/1895715651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81079d0 0x7f11a8107db0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 -- 172.21.15.73:0/1895715651 >> 172.21.15.73:0/1895715651 conn(0x7f11a80fd450 msgr2=0x7f11a80ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 -- 172.21.15.73:0/1895715651 shutdown_connections 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.801+0000 7f11adf29640 1 -- 172.21.15.73:0/1895715651 wait complete. 2024-06-06T07:07:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 Processor -- start 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 -- start start 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 0x7f11a819a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81079d0 0x7f11a819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8074ff0 con 0x7f11a81019c0 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8074e70 con 0x7f11a81079d0 2024-06-06T07:07:37.805 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.802+0000 7f11adf29640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8075170 con 0x7f11a8102380 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 0x7f11a819a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57336/0 (socket says 172.21.15.73:57336) 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81079d0 0x7f11a819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 -- 172.21.15.73:0/4144519606 learned_addr learned my addr 172.21.15.73:0/4144519606 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 -- 172.21.15.73:0/4144519606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 msgr2=0x7f11a819a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 0x7f11a819a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 -- 172.21.15.73:0/4144519606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81079d0 msgr2=0x7f11a819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81079d0 0x7f11a819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 -- 172.21.15.73:0/4144519606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a819f330 con 0x7f11a81019c0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1198005940 tx=0x7f11980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a4ff9640 1 -- 172.21.15.73:0/4144519606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119803f030 con 0x7f11a81019c0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a4ff9640 1 -- 172.21.15.73:0/4144519606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980318a0 con 0x7f11a81019c0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a819f5c0 con 0x7f11a81019c0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11a4ff9640 1 -- 172.21.15.73:0/4144519606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198035070 con 0x7f11a81019c0 2024-06-06T07:07:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.803+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a6eb0 con 0x7f11a81019c0 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11a4ff9640 1 -- 172.21.15.73:0/4144519606 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1198042030 con 0x7f11a81019c0 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 Processor -- start 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 -- start start 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 0x55c6d1610e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c6d156a4f0 con 0x55c6d1606a90 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6d1508870 con 0x55c6d1611350 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11b0c37740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6d15664b0 con 0x55c6d1606170 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 0x55c6d1610e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.806+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:58490/0 (socket says 172.21.15.73:58490) 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a7fff640 1 -- 172.21.15.73:0/2288140302 learned_addr learned my addr 172.21.15.73:0/2288140302 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:45396/0 (socket says 172.21.15.73:45396) 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a4ff9640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 0x7f11900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 -- 172.21.15.73:0/2288140302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 msgr2=0x55c6d1606550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 -- 172.21.15.73:0/2288140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 msgr2=0x55c6d1610e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 0x55c6d1610e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 -- 172.21.15.73:0/2288140302 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d15249b0 con 0x55c6d1611350 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a7fff640 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a77fe640 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 0x55c6d1610e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 0x7f11900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a4ff9640 1 -- 172.21.15.73:0/4144519606 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f11980efb70 con 0x7f11a81019c0 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f119c009bf0 tx=0x7f119c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3f54541c7766a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f118dffb640 1 -- 172.21.15.73:0/2288140302 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c009da0 con 0x55c6d1611350 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.807+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 0x7f11900a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0004ab0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f118dffb640 1 -- 172.21.15.73:0/2288140302 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c005020 con 0x55c6d1611350 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f118dffb640 1 -- 172.21.15.73:0/2288140302 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c004a90 con 0x55c6d1611350 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 -- 172.21.15.73:0/2288140302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 msgr2=0x55c6d1613740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f119c009bf0 tx=0x7f119c011810 comp rx=0 tx=0).stop 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 -- 172.21.15.73:0/2288140302 shutdown_connections 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1611350 0x55c6d1613740 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606a90 0x55c6d1610e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 --2- 172.21.15.73:0/2288140302 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1606550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 -- 172.21.15.73:0/2288140302 >> 172.21.15.73:0/2288140302 conn(0x55c6d1602680 msgr2=0x55c6d1602a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 -- 172.21.15.73:0/2288140302 shutdown_connections 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.808+0000 7f11b0c37740 1 -- 172.21.15.73:0/2288140302 wait complete. 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 -- start start 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 Processor -- start 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d160e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d160ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c6d1613f80 con 0x55c6d1606170 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6d14f3ae0 con 0x55c6d1606a90 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6d1614100 con 0x55c6d1611350 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11b0c37740 10 client.0 _ll_register_callbacks cb 0x55c6d152c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57366/0 (socket says 172.21.15.73:57366) 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11a77fe640 1 -- 172.21.15.73:0/3940627348 learned_addr learned my addr 172.21.15.73:0/3940627348 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11a7fff640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d160ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.809+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d160e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 -- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 msgr2=0x55c6d160ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d160ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 -- 172.21.15.73:0/3940627348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 msgr2=0x55c6d160e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d160e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 -- 172.21.15.73:0/3940627348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d127e210 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d160e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1198031440 tx=0x7f1198005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6761c38769528df5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198049040 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f11980b8400 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c005020 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f119803b460 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11b0c37740 10 client.16356 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.810+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6d134a620 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c005020 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f11980c44c0 con 0x55c6d1606170 2024-06-06T07:07:37.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 10 client.16356 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 10 client.16356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 20 client.16356 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d1621350 0x55c6d1623780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 4 client.16356 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6d157df10 con 0x55c6d1621350 2024-06-06T07:07:37.818 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.811+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d1621350 0x55c6d1623780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.812+0000 7f1186ffd640 10 client.16356 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.812+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d1621350 0x55c6d1623780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c2897184938166 server_cookie=37f05c552abb2de5 in_seq=0 out_seq=0 2024-06-06T07:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f1186ffd640 1 -- 172.21.15.73:0/3940627348 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6d157df10 con 0x55c6d1621350 2024-06-06T07:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f1186ffd640 10 client.16356 handle_command_reply: tid=0 2024-06-06T07:07:37.819 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 client.16356 shutdown 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 msgr2=0x55c6d160e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1198031440 tx=0x7f1198005c90 comp rx=0 tx=0).stop 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 shutdown_connections 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d1621350 0x55c6d1623780 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d160ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d160e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 --2- 172.21.15.73:0/3940627348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d160e230 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f1186ffd640 0 client.16356 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 >> 172.21.15.73:0/3940627348 conn(0x55c6d1602680 msgr2=0x55c6d1608400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 shutdown_connections 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 1 -- 172.21.15.73:0/3940627348 wait complete. 2024-06-06T07:07:37.820 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.813+0000 7f11b0c37740 20 client.16356 trim_cache size 0 max 16384 2024-06-06T07:07:37.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 Processor -- start 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 -- start start 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d16295a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d1600f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1070 con 0x55c6d1606170 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b2bb0 con 0x55c6d1606a90 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11b0c37740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1740 con 0x55c6d1611350 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57368/0 (socket says 172.21.15.73:57368) 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11a77fe640 1 -- 172.21.15.73:0/4109592295 learned_addr learned my addr 172.21.15.73:0/4109592295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:37.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.829+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d1600f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d16295a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 -- 172.21.15.73:0/4109592295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 msgr2=0x55c6d1600f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d1600f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 -- 172.21.15.73:0/4109592295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 msgr2=0x55c6d16295a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d16295a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 -- 172.21.15.73:0/4109592295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6d15fdf70 con 0x55c6d1606170 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f11980b81f0 tx=0x7f11980c4950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c333d8ebf86626 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f118d7fa640 1 -- 172.21.15.73:0/4109592295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198003e20 con 0x55c6d1606170 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f118d7fa640 1 -- 172.21.15.73:0/4109592295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980bbd90 con 0x55c6d1606170 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.830+0000 7f118d7fa640 1 -- 172.21.15.73:0/4109592295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980b7df0 con 0x55c6d1606170 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- 172.21.15.73:0/4109592295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 msgr2=0x55c6d1629060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f11980b81f0 tx=0x7f11980c4950 comp rx=0 tx=0).stop 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- 172.21.15.73:0/4109592295 shutdown_connections 2024-06-06T07:07:37.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1611350 0x55c6d1600f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d16295a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4109592295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1606170 0x55c6d1629060 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- 172.21.15.73:0/4109592295 >> 172.21.15.73:0/4109592295 conn(0x55c6d1602680 msgr2=0x55c6d1602a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- 172.21.15.73:0/4109592295 shutdown_connections 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- 172.21.15.73:0/4109592295 wait complete. 2024-06-06T07:07:37.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 -- start start 2024-06-06T07:07:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.831+0000 7f11b0c37740 1 Processor -- start 2024-06-06T07:07:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.836 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d1619a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 0x55c6d160aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b24e0 con 0x55c6d1611350 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81b1e10 con 0x55c6d1606a90 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81af050 con 0x55c6d1606170 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11b0c37740 10 client.0 _ll_register_callbacks cb 0x7f11980bbc50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:58526/0 (socket says 172.21.15.73:58526) 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 -- 172.21.15.73:0/4003905495 learned_addr learned my addr 172.21.15.73:0/4003905495 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d1619a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 0x55c6d160aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 -- 172.21.15.73:0/4003905495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 msgr2=0x55c6d1619a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d1619a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.837 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 -- 172.21.15.73:0/4003905495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 msgr2=0x55c6d160aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 0x55c6d160aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 -- 172.21.15.73:0/4003905495 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6d160b2d0 con 0x55c6d1606170 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d1619a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 0x55c6d160aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c00 tx=0x7f11980c4a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dd50511e12d7685 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.832+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198049030 con 0x55c6d1606170 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f119809adb0 con 0x55c6d1606170 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11980bbd90 con 0x55c6d1606170 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1198003e20 con 0x55c6d1606170 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 10 client.25897 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:37.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c6d160b2d0 con 0x55c6d1606170 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980bbd90 con 0x55c6d1606170 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f1198003890 con 0x55c6d1606170 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 10 client.25897 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 10 client.25897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 20 client.25897 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d168bdb0 0x55c6d168e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 4 client.25897 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6d1577c20 con 0x55c6d168bdb0 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.833+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d168bdb0 0x55c6d168e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.834+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d168bdb0 0x55c6d168e1a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c05f3b72d6f61c69 server_cookie=fe6ca4a923ae2c9c in_seq=0 out_seq=0 2024-06-06T07:07:37.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.834+0000 7f1186ffd640 10 client.25897 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:37.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:37 smithi170 ceph-mon[36108]: pgmap v191: 129 pgs: 129 active+clean; 91 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f1186ffd640 1 -- 172.21.15.73:0/4003905495 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6d1577c20 con 0x55c6d168bdb0 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f1186ffd640 10 client.25897 handle_command_reply: tid=0 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 client.25897 shutdown 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 msgr2=0x55c6d1619540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c00 tx=0x7f11980c4a90 comp rx=0 tx=0).stop 2024-06-06T07:07:37.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 shutdown_connections 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6d168bdb0 0x55c6d168e1a0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f1186ffd640 0 client.25897 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6d1611350 0x55c6d160aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6d1606a90 0x55c6d1619a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 --2- 172.21.15.73:0/4003905495 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6d1606170 0x55c6d1619540 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 >> 172.21.15.73:0/4003905495 conn(0x55c6d1602680 msgr2=0x55c6d160e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 shutdown_connections 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.855+0000 7f11b0c37740 1 -- 172.21.15.73:0/4003905495 wait complete. 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11b0c37740 20 client.25897 trim_cache size 0 max 16384 2024-06-06T07:07:37.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:07:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 msgr2=0x7f11900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 0x7f11900a5260 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009fd0 tx=0x7f11a0004ab0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 msgr2=0x7f11a819a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1198005940 tx=0x7f11980042f0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 shutdown_connections 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a81079d0 0x7f11a819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2da0 0x7f11900a5260 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8102380 0x7f11a819a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 --2- 172.21.15.73:0/4144519606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81019c0 0x7f11a819a390 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.856+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 >> 172.21.15.73:0/4144519606 conn(0x7f11a80fd450 msgr2=0x7f11a80fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.857+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 shutdown_connections 2024-06-06T07:07:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:37.857+0000 7f11adf29640 1 -- 172.21.15.73:0/4144519606 wait complete. 2024-06-06T07:07:38.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 138511 2.63 MB/sec execute 165 sec latency 15.334 ms 2024-06-06T07:07:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:38 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:07:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:38 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:07:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:38 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:07:38.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:38 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:07:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:38 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:07:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:38 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:07:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:38 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:07:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:38 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:07:38.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:38 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:07:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:38 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:07:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:38 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:07:38.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:38 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:07:39.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 138803 2.62 MB/sec execute 166 sec latency 14.928 ms 2024-06-06T07:07:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:39 smithi073 ceph-mon[32303]: pgmap v192: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:39 smithi136 ceph-mon[35901]: pgmap v192: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:39 smithi170 ceph-mon[36108]: pgmap v192: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:40.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 139191 2.61 MB/sec execute 167 sec latency 18.251 ms 2024-06-06T07:07:41.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 139654 2.60 MB/sec execute 168 sec latency 15.757 ms 2024-06-06T07:07:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:41 smithi073 ceph-mon[32303]: pgmap v193: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:41.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:41 smithi136 ceph-mon[35901]: pgmap v193: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:41.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:41 smithi170 ceph-mon[36108]: pgmap v193: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:42.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 140199 2.60 MB/sec execute 169 sec latency 22.662 ms 2024-06-06T07:07:43.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 140753 2.60 MB/sec execute 170 sec latency 43.072 ms 2024-06-06T07:07:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:43 smithi073 ceph-mon[32303]: pgmap v194: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:43 smithi136 ceph-mon[35901]: pgmap v194: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:43 smithi170 ceph-mon[36108]: pgmap v194: 129 pgs: 129 active+clean; 93 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1019 KiB/s rd, 1.4 MiB/s wr, 136 op/s 2024-06-06T07:07:44.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 141306 2.62 MB/sec execute 171 sec latency 61.729 ms 2024-06-06T07:07:45.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 141758 2.61 MB/sec execute 172 sec latency 14.907 ms 2024-06-06T07:07:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:45 smithi073 ceph-mon[32303]: pgmap v195: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:07:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:45 smithi136 ceph-mon[35901]: pgmap v195: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:07:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:45 smithi170 ceph-mon[36108]: pgmap v195: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:07:46.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 142213 2.62 MB/sec execute 173 sec latency 15.047 ms 2024-06-06T07:07:47.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 142489 2.61 MB/sec execute 174 sec latency 14.828 ms 2024-06-06T07:07:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:47 smithi073 ceph-mon[32303]: pgmap v196: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:47 smithi136 ceph-mon[35901]: pgmap v196: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:47.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:47 smithi170 ceph-mon[36108]: pgmap v196: 129 pgs: 129 active+clean; 90 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-06T07:07:48.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 142952 2.60 MB/sec execute 175 sec latency 14.943 ms 2024-06-06T07:07:49.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 143410 2.61 MB/sec execute 176 sec latency 67.230 ms 2024-06-06T07:07:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:49 smithi136 ceph-mon[35901]: pgmap v197: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:49.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:49 smithi170 ceph-mon[36108]: pgmap v197: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:49 smithi073 ceph-mon[32303]: pgmap v197: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 214 op/s 2024-06-06T07:07:50.440 INFO:tasks.workunit.client.0.smithi073.stdout: 1 143967 2.60 MB/sec execute 177 sec latency 25.455 ms 2024-06-06T07:07:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 -- 172.21.15.73:0/2933460516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197010bfb0 msgr2=0x7f197010e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2933460516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197010bfb0 0x7f197010e400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1968009ca0 tx=0x7f196802a950 comp rx=0 tx=0).stop 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 -- 172.21.15.73:0/2933460516 shutdown_connections 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2933460516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197010bfb0 0x7f197010e400 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2933460516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1970103590 0x7f197010ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2933460516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970102c70 0x7f1970103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 -- 172.21.15.73:0/2933460516 >> 172.21.15.73:0/2933460516 conn(0x7f1970077a20 msgr2=0x7f1970077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 -- 172.21.15.73:0/2933460516 shutdown_connections 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.947+0000 7f1977cf5640 1 -- 172.21.15.73:0/2933460516 wait complete. 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 Processor -- start 2024-06-06T07:07:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 -- start start 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970103590 0x7f1970100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f197010bfb0 0x7f1970101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1970111c10 con 0x7f1970102c70 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1970111a90 con 0x7f197010bfb0 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.948+0000 7f1977cf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1970111d90 con 0x7f1970103590 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46404/0 (socket says 172.21.15.73:46404) 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 -- 172.21.15.73:0/2778298061 learned_addr learned my addr 172.21.15.73:0/2778298061 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1976292640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f197010bfb0 0x7f1970101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975290640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970103590 0x7f1970100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 -- 172.21.15.73:0/2778298061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970103590 msgr2=0x7f1970100e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970103590 0x7f1970100e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 -- 172.21.15.73:0/2778298061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f197010bfb0 msgr2=0x7f1970101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f197010bfb0 0x7f1970101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 -- 172.21.15.73:0/2778298061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19701a6590 con 0x7f1970102c70 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1975a91640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ea60 tx=0x7f196c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1966ffd640 1 -- 172.21.15.73:0/2778298061 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c00cde0 con 0x7f1970102c70 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19701a6880 con 0x7f1970102c70 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1966ffd640 1 -- 172.21.15.73:0/2778298061 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f196c0035f0 con 0x7f1970102c70 2024-06-06T07:07:50.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1966ffd640 1 -- 172.21.15.73:0/2778298061 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196c010690 con 0x7f1970102c70 2024-06-06T07:07:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.949+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19701a6e20 con 0x7f1970102c70 2024-06-06T07:07:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.952+0000 7f197aa03740 1 Processor -- start 2024-06-06T07:07:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.952+0000 7f1966ffd640 1 -- 172.21.15.73:0/2778298061 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f196c0037a0 con 0x7f1970102c70 2024-06-06T07:07:50.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 -- start start 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 0x55f6785d9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785dbf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f67852a4f0 con 0x55f6785c69d0 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f6784c8870 con 0x55f6785d9b60 2024-06-06T07:07:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f197aa03740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6785264b0 con 0x55f6785c60b0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f1975a91640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 0x55f6785d9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f1976292640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f1975a91640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 0x55f6785d9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46408/0 (socket says 172.21.15.73:46408) 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f1975a91640 1 -- 172.21.15.73:0/4230850091 learned_addr learned my addr 172.21.15.73:0/4230850091 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.953+0000 7f1975290640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785dbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 -- 172.21.15.73:0/4230850091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 msgr2=0x55f6785dbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785dbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 -- 172.21.15.73:0/4230850091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 msgr2=0x55f6785d9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 0x55f6785d9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 -- 172.21.15.73:0/4230850091 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6784e49b0 con 0x55f6785c60b0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1975290640 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1966ffd640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 0x7f195c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1975a91640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 0x7f195c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1976292640 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c6490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1968005cc0 tx=0x7f19680093e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad60c862a4871301 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f1966ffd640 1 -- 172.21.15.73:0/2778298061 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f196c0cfa90 con 0x7f1970102c70 2024-06-06T07:07:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f195bfff640 1 -- 172.21.15.73:0/4230850091 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968048070 con 0x55f6785c60b0 2024-06-06T07:07:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f195bfff640 1 -- 172.21.15.73:0/4230850091 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1968003e20 con 0x55f6785c60b0 2024-06-06T07:07:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.954+0000 7f195bfff640 1 -- 172.21.15.73:0/4230850091 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968034040 con 0x55f6785c60b0 2024-06-06T07:07:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 -- 172.21.15.73:0/4230850091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 msgr2=0x55f6785c6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c6490 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1968005cc0 tx=0x7f19680093e0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f1975a91640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 0x7f195c0a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0d35e0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 -- 172.21.15.73:0/4230850091 shutdown_connections 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785dbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c69d0 0x55f6785d9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 --2- 172.21.15.73:0/4230850091 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c6490 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 -- 172.21.15.73:0/4230850091 >> 172.21.15.73:0/4230850091 conn(0x55f6785c2680 msgr2=0x55f6785c2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 -- 172.21.15.73:0/4230850091 shutdown_connections 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.955+0000 7f197aa03740 1 -- 172.21.15.73:0/4230850091 wait complete. 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 -- start start 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 Processor -- start 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 0x55f6785d7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785d75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f6785dc6d0 con 0x55f6785c60b0 2024-06-06T07:07:50.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55f6784b3ae0 con 0x55f6785d9b60 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55f6785dc850 con 0x55f6785c69d0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f197aa03740 10 client.0 _ll_register_callbacks cb 0x55f6784ec9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46410/0 (socket says 172.21.15.73:46410) 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f1976292640 1 -- 172.21.15.73:0/1889148654 learned_addr learned my addr 172.21.15.73:0/1889148654 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f1975290640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 0x55f6785d7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.956+0000 7f1976292640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785d75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 -- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 msgr2=0x55f6785d7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 0x55f6785d7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 -- 172.21.15.73:0/1889148654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 msgr2=0x55f6785d75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785d75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 -- 172.21.15.73:0/1889148654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f67823e210 con 0x55f6785c60b0 2024-06-06T07:07:50.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1975290640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 0x55f6785d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f1976292640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1968046040 tx=0x7f1968009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433130b798274710 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f19597fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968034050 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1968004000 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1968003e20 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1968059050 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f197aa03740 10 client.16374 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f67830a620 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.957+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1968003e20 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f196804e020 con 0x55f6785c60b0 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 10 client.16374 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 10 client.16374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:50.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 20 client.16374 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f6785e9b90 0x55f6785ebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 4 client.16374 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f67853df10 con 0x55f6785e9b90 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f1975290640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f6785e9b90 0x55f6785ebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f1975290640 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f6785e9b90 0x55f6785ebf80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4a1a9277050798 server_cookie=8214bebc379ba751 in_seq=0 out_seq=0 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.958+0000 7f19597fa640 10 client.16374 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f19597fa640 1 -- 172.21.15.73:0/1889148654 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f67853df10 con 0x55f6785e9b90 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f19597fa640 10 client.16374 handle_command_reply: tid=0 2024-06-06T07:07:50.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 client.16374 shutdown 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 msgr2=0x55f6785d6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f1968046040 tx=0x7f1968009580 comp rx=0 tx=0).stop 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 shutdown_connections 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f6785e9b90 0x55f6785ebf80 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785d9b60 0x55f6785d75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c69d0 0x55f6785d7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f19597fa640 0 client.16374 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 --2- 172.21.15.73:0/1889148654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785d6b20 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 >> 172.21.15.73:0/1889148654 conn(0x55f6785c2680 msgr2=0x55f6785c8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 shutdown_connections 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 1 -- 172.21.15.73:0/1889148654 wait complete. 2024-06-06T07:07:50.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.959+0000 7f197aa03740 20 client.16374 trim_cache size 0 max 16384 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 Processor -- start 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 -- start start 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785f1520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19701acab0 con 0x55f6785d9b60 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19701ae5f0 con 0x55f6785c69d0 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f197aa03740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19701ad180 con 0x55f6785c60b0 2024-06-06T07:07:50.977 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f1975290640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46426/0 (socket says 172.21.15.73:46426) 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.975+0000 7f1976292640 1 -- 172.21.15.73:0/1865675185 learned_addr learned my addr 172.21.15.73:0/1865675185 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1976292640 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 -- 172.21.15.73:0/1865675185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 msgr2=0x55f6785c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 -- 172.21.15.73:0/1865675185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 msgr2=0x55f6785f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 -- 172.21.15.73:0/1865675185 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6785d5ad0 con 0x55f6785c69d0 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1976292640 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f1975290640 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785f1520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f6785ec4c0 tx=0x7f1960005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d1d39a57f047c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f195b7fe640 1 -- 172.21.15.73:0/1865675185 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1960004310 con 0x55f6785c69d0 2024-06-06T07:07:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f195b7fe640 1 -- 172.21.15.73:0/1865675185 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19600044b0 con 0x55f6785c69d0 2024-06-06T07:07:50.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.976+0000 7f195b7fe640 1 -- 172.21.15.73:0/1865675185 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196001e890 con 0x55f6785c69d0 2024-06-06T07:07:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- 172.21.15.73:0/1865675185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 msgr2=0x55f6785f1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785f1520 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x55f6785ec4c0 tx=0x7f1960005f90 comp rx=0 tx=0).stop 2024-06-06T07:07:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- 172.21.15.73:0/1865675185 shutdown_connections 2024-06-06T07:07:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785d9b60 0x55f6785f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785f1520 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 --2- 172.21.15.73:0/1865675185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785c60b0 0x55f6785c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- 172.21.15.73:0/1865675185 >> 172.21.15.73:0/1865675185 conn(0x55f6785c2680 msgr2=0x55f6785c2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- 172.21.15.73:0/1865675185 shutdown_connections 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- 172.21.15.73:0/1865675185 wait complete. 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 -- start start 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.977+0000 7f197aa03740 1 Processor -- start 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785dd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785d9b60 0x55f6785dda80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f19701adf20 con 0x55f6785c60b0 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f19701ad850 con 0x55f6785c69d0 2024-06-06T07:07:50.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f19701aaab0 con 0x55f6785d9b60 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f197aa03740 10 client.0 _ll_register_callbacks cb 0x7f1968046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46434/0 (socket says 172.21.15.73:46434) 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 -- 172.21.15.73:0/3784180611 learned_addr learned my addr 172.21.15.73:0/3784180611 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785d9b60 0x55f6785dda80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 -- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785d9b60 msgr2=0x55f6785dda80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785d9b60 0x55f6785dda80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1975290640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785dd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 -- 172.21.15.73:0/3784180611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 msgr2=0x55f6785dd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785dd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 -- 172.21.15.73:0/3784180611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6785de170 con 0x55f6785c60b0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f1976292640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1968002230 tx=0x7f196802f480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb062af8d8009bd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:07:50.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.978+0000 7f19597fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1968056040 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9466+0+0 (secure 0 0 0) 0x7f1968050440 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19600044b0 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f196806e050 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 10 client.16380 fetch_fsmap learned FSMap version 36 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f678654400 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19600044b0 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f196804e040 con 0x55f6785c60b0 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 10 client.16380 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 20 client.16380 populate_metadata read hostname 'smithi073' 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f678654b50 0x55f678656f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:07:50.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 4 client.16380 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:07:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f678537c20 con 0x55f678654b50 2024-06-06T07:07:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.979+0000 7f1975290640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f678654b50 0x55f678656f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:07:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.980+0000 7f1975290640 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f678654b50 0x55f678656f70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc102d8a23fdd7a2 server_cookie=80ed12b0951a084f in_seq=0 out_seq=0 2024-06-06T07:07:50.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:50.980+0000 7f19597fa640 10 client.16380 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f19597fa640 1 -- 172.21.15.73:0/3784180611 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f678537c20 con 0x55f678654b50 2024-06-06T07:07:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f19597fa640 10 client.16380 handle_command_reply: tid=0 2024-06-06T07:07:51.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 client.16380 shutdown 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 msgr2=0x55f6785e2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1968002230 tx=0x7f196802f480 comp rx=0 tx=0).stop 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 shutdown_connections 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55f678654b50 0x55f678656f70 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.005+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55f6785d9b60 0x55f6785dda80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f19597fa640 0 client.16380 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55f6785c69d0 0x55f6785dd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 1 --2- 172.21.15.73:0/3784180611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f6785c60b0 0x55f6785e2180 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 >> 172.21.15.73:0/3784180611 conn(0x55f6785c2680 msgr2=0x55f6785c8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 shutdown_connections 2024-06-06T07:07:51.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 1 -- 172.21.15.73:0/3784180611 wait complete. 2024-06-06T07:07:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f197aa03740 20 client.16380 trim_cache size 0 max 16384 2024-06-06T07:07:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:07:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:07:51.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 msgr2=0x7f195c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 0x7f195c0a5260 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f196c000c00 tx=0x7f196c0d35e0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 msgr2=0x7f19701008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f196c00ea60 tx=0x7f196c00ef20 comp rx=0 tx=0).stop 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 shutdown_connections 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f197010bfb0 0x7f1970101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f195c0a2da0 0x7f195c0a5260 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1970103590 0x7f1970100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 --2- 172.21.15.73:0/2778298061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970102c70 0x7f19701008c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:07:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 >> 172.21.15.73:0/2778298061 conn(0x7f1970077a20 msgr2=0x7f19700fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:07:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 shutdown_connections 2024-06-06T07:07:51.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:07:51.006+0000 7f1977cf5640 1 -- 172.21.15.73:0/2778298061 wait complete. 2024-06-06T07:07:51.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 144459 2.59 MB/sec execute 178 sec latency 17.451 ms 2024-06-06T07:07:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:51 smithi136 ceph-mon[35901]: pgmap v198: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:51.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:51 smithi170 ceph-mon[36108]: pgmap v198: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:52.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:51 smithi073 ceph-mon[32303]: pgmap v198: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:52.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 145187 2.62 MB/sec execute 179 sec latency 15.935 ms 2024-06-06T07:07:53.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 145572 2.62 MB/sec execute 180 sec latency 17.166 ms 2024-06-06T07:07:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:53 smithi136 ceph-mon[35901]: pgmap v199: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:53.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:53 smithi170 ceph-mon[36108]: pgmap v199: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:54.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:53 smithi073 ceph-mon[32303]: pgmap v199: 129 pgs: 129 active+clean; 94 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-06T07:07:54.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 145879 2.61 MB/sec execute 181 sec latency 14.483 ms 2024-06-06T07:07:55.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 146267 2.60 MB/sec execute 182 sec latency 16.573 ms 2024-06-06T07:07:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:55 smithi136 ceph-mon[35901]: pgmap v200: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:07:55.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:55 smithi170 ceph-mon[36108]: pgmap v200: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:07:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:55 smithi073 ceph-mon[32303]: pgmap v200: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 217 op/s 2024-06-06T07:07:56.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 146746 2.60 MB/sec execute 183 sec latency 15.953 ms 2024-06-06T07:07:57.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 147288 2.60 MB/sec execute 184 sec latency 20.964 ms 2024-06-06T07:07:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:57 smithi136 ceph-mon[35901]: pgmap v201: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-06-06T07:07:57.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:57 smithi170 ceph-mon[36108]: pgmap v201: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-06-06T07:07:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:57 smithi073 ceph-mon[32303]: pgmap v201: 129 pgs: 129 active+clean; 96 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 140 op/s 2024-06-06T07:07:58.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 147853 2.59 MB/sec execute 185 sec latency 18.006 ms 2024-06-06T07:07:59.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 148470 2.61 MB/sec execute 186 sec latency 21.559 ms 2024-06-06T07:07:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:07:59 smithi136 ceph-mon[35901]: pgmap v202: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:07:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:07:59 smithi170 ceph-mon[36108]: pgmap v202: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:08:00.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:07:59 smithi073 ceph-mon[32303]: pgmap v202: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 215 op/s 2024-06-06T07:08:00.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 148978 2.62 MB/sec execute 187 sec latency 14.978 ms 2024-06-06T07:08:00.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:00 smithi170 ceph-mon[36108]: pgmap v203: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:01.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:00 smithi073 ceph-mon[32303]: pgmap v203: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:01.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:00 smithi136 ceph-mon[35901]: pgmap v203: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:01.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 149330 2.62 MB/sec execute 188 sec latency 17.685 ms 2024-06-06T07:08:02.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 149613 2.60 MB/sec execute 189 sec latency 18.167 ms 2024-06-06T07:08:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:02 smithi073 ceph-mon[32303]: pgmap v204: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:02 smithi136 ceph-mon[35901]: pgmap v204: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:03.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:02 smithi170 ceph-mon[36108]: pgmap v204: 129 pgs: 129 active+clean; 97 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-06T07:08:03.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 150104 2.60 MB/sec execute 190 sec latency 14.697 ms 2024-06-06T07:08:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.106+0000 7f4591f7e640 1 -- 172.21.15.73:0/2880938366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 msgr2=0x7f458c10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.106+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2880938366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c10d750 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f4584009cd0 tx=0x7f458402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 -- 172.21.15.73:0/2880938366 shutdown_connections 2024-06-06T07:08:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2880938366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c10d750 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2880938366 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 0x7f458c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2880938366 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 0x7f458c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 -- 172.21.15.73:0/2880938366 >> 172.21.15.73:0/2880938366 conn(0x7f458c077a20 msgr2=0x7f458c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 -- 172.21.15.73:0/2880938366 shutdown_connections 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 -- 172.21.15.73:0/2880938366 wait complete. 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 Processor -- start 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.107+0000 7f4591f7e640 1 -- start start 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 0x7f458c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 0x7f458c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f458c110b50 con 0x7f458c101290 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1109d0 con 0x7f458c1008d0 2024-06-06T07:08:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f458c110cd0 con 0x7f458c106890 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35540/0 (socket says 172.21.15.73:35540) 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 -- 172.21.15.73:0/2000129186 learned_addr learned my addr 172.21.15.73:0/2000129186 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 0x7f458c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458bfff640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 0x7f458c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 -- 172.21.15.73:0/2000129186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 msgr2=0x7f458c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 0x7f458c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 -- 172.21.15.73:0/2000129186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 msgr2=0x7f458c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 0x7f458c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 -- 172.21.15.73:0/2000129186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c19d010 con 0x7f458c101290 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f458affd640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f458000c920 tx=0x7f458000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4588ff9640 1 -- 172.21.15.73:0/2000129186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580007c70 con 0x7f458c101290 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c19d300 con 0x7f458c101290 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4588ff9640 1 -- 172.21.15.73:0/2000129186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4580007e10 con 0x7f458c101290 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4588ff9640 1 -- 172.21.15.73:0/2000129186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580018820 con 0x7f458c101290 2024-06-06T07:08:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.108+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1a4b90 con 0x7f458c101290 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.111+0000 7f4588ff9640 1 -- 172.21.15.73:0/2000129186 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f45800189c0 con 0x7f458c101290 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.111+0000 7f4594d6d740 1 Processor -- start 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.111+0000 7f4594d6d740 1 -- start start 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.111+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 0x55dbaa919450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.111+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa928220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4588ff9640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 0x7f45740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa92ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4594d6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa87d4f0 con 0x55dbaa919070 2024-06-06T07:08:04.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4594d6d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa81b870 con 0x55dbaa919990 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4594d6d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa8794b0 con 0x55dbaa928760 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 0x55dbaa919450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458affd640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 0x7f45740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 0x55dbaa919450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35550/0 (socket says 172.21.15.73:35550) 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458bfff640 1 -- 172.21.15.73:0/2016520268 learned_addr learned my addr 172.21.15.73:0/2016520268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa928220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458bfff640 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa92ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4588ff9640 1 -- 172.21.15.73:0/2000129186 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f45800cf930 con 0x7f458c101290 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 -- 172.21.15.73:0/2016520268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 msgr2=0x55dbaa92ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa92ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 -- 172.21.15.73:0/2016520268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 msgr2=0x55dbaa919450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 0x55dbaa919450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 -- 172.21.15.73:0/2016520268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbaa8379b0 con 0x55dbaa919990 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458affd640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 0x7f45740a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f458009b6f0 tx=0x7f4580000bc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa928220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f457c009c00 tx=0x7f457c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3f48b64a8077e54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4571ffb640 1 -- 172.21.15.73:0/2016520268 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004050 con 0x55dbaa919990 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4571ffb640 1 -- 172.21.15.73:0/2016520268 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f457c0054d0 con 0x55dbaa919990 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.112+0000 7f4571ffb640 1 -- 172.21.15.73:0/2016520268 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c004c60 con 0x55dbaa919990 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 -- 172.21.15.73:0/2016520268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 msgr2=0x55dbaa928220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa928220 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f457c009c00 tx=0x7f457c011810 comp rx=0 tx=0).stop 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 -- 172.21.15.73:0/2016520268 shutdown_connections 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa92ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa928220 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2016520268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919070 0x55dbaa919450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 -- 172.21.15.73:0/2016520268 >> 172.21.15.73:0/2016520268 conn(0x55dbaa915680 msgr2=0x55dbaa915a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 -- 172.21.15.73:0/2016520268 shutdown_connections 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.113+0000 7f4594d6d740 1 -- 172.21.15.73:0/2016520268 wait complete. 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 -- start start 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 Processor -- start 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa925630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa9260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa92b390 con 0x55dbaa919990 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa806ae0 con 0x55dbaa919070 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dbaa92b510 con 0x55dbaa928760 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f4594d6d740 10 client.0 _ll_register_callbacks cb 0x55dbaa83f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa925630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35560/0 (socket says 172.21.15.73:35560) 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa925630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53406/0 (socket says 172.21.15.73:53406) 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458b7fe640 1 -- 172.21.15.73:0/1177119079 learned_addr learned my addr 172.21.15.73:0/1177119079 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458bfff640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa9260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458b7fe640 1 -- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 msgr2=0x55dbaa9260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.114+0000 7f458b7fe640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa9260b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f458b7fe640 1 -- 172.21.15.73:0/1177119079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 msgr2=0x55dbaa925630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f458b7fe640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa925630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f458b7fe640 1 -- 172.21.15.73:0/1177119079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbaa591210 con 0x55dbaa919990 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f458b7fe640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f457c004500 tx=0x7f457c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef2dcd60935986de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f456affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f457c028020 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f457c0054d0 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f457c003a00 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f457c033050 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f4594d6d740 10 client.16392 fetch_fsmap learned FSMap version 36 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dbaa65d620 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.115+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f457c0054d0 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f457c018070 con 0x55dbaa919990 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 10 client.16392 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 10 client.16392 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 20 client.16392 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa93abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 4 client.16392 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dbaa890f10 con 0x55dbaa9387e0 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f458bfff640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa93abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f458bfff640 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa93abd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1572b94ce7fc4ee9 server_cookie=119c99a4b9e42c6a in_seq=0 out_seq=0 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.116+0000 7f456affd640 10 client.16392 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f456affd640 1 -- 172.21.15.73:0/1177119079 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dbaa890f10 con 0x55dbaa9387e0 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f456affd640 10 client.16392 handle_command_reply: tid=0 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 client.16392 shutdown 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 msgr2=0x55dbaa925b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f457c004500 tx=0x7f457c0046c0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 shutdown_connections 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa93abd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa928760 0x55dbaa9260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f456affd640 0 client.16392 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa919990 0x55dbaa925b70 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 --2- 172.21.15.73:0/1177119079 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa925630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 >> 172.21.15.73:0/1177119079 conn(0x55dbaa915680 msgr2=0x55dbaa91b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 shutdown_connections 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 1 -- 172.21.15.73:0/1177119079 wait complete. 2024-06-06T07:08:04.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.117+0000 7f4594d6d740 20 client.16392 trim_cache size 0 max 16384 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 Processor -- start 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 -- start start 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa91b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa940360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1accc0 con 0x55dbaa928760 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1ae800 con 0x55dbaa919990 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f4594d6d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1ad390 con 0x55dbaa919070 2024-06-06T07:08:04.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f458bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa940360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f458bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f458bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:60248/0 (socket says 172.21.15.73:60248) 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.134+0000 7f458bfff640 1 -- 172.21.15.73:0/3089166249 learned_addr learned my addr 172.21.15.73:0/3089166249 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458b7fe640 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa91b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 -- 172.21.15.73:0/3089166249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 msgr2=0x55dbaa91b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa91b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 -- 172.21.15.73:0/3089166249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 msgr2=0x55dbaa940360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa940360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 -- 172.21.15.73:0/3089166249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dbaa9203f0 con 0x55dbaa919070 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f458bfff640 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f458402ae00 tx=0x7f458402a8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6486be3a024c5a20 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f45717fa640 1 -- 172.21.15.73:0/3089166249 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458405c070 con 0x55dbaa919070 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f45717fa640 1 -- 172.21.15.73:0/3089166249 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4584036040 con 0x55dbaa919070 2024-06-06T07:08:04.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.135+0000 7f45717fa640 1 -- 172.21.15.73:0/3089166249 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584030800 con 0x55dbaa919070 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- 172.21.15.73:0/3089166249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 msgr2=0x55dbaa924570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f458402ae00 tx=0x7f458402a8e0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- 172.21.15.73:0/3089166249 shutdown_connections 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa940360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919990 0x55dbaa91b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 --2- 172.21.15.73:0/3089166249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919070 0x55dbaa924570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- 172.21.15.73:0/3089166249 >> 172.21.15.73:0/3089166249 conn(0x55dbaa915680 msgr2=0x55dbaa91abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- 172.21.15.73:0/3089166249 shutdown_connections 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- 172.21.15.73:0/3089166249 wait complete. 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.136+0000 7f4594d6d740 1 -- start start 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 Processor -- start 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa930cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1ae130 con 0x55dbaa928760 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1ada60 con 0x55dbaa919070 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1aacc0 con 0x55dbaa919990 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f4594d6d740 10 client.0 _ll_register_callbacks cb 0x7f4584009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa930cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35574/0 (socket says 172.21.15.73:35574) 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 -- 172.21.15.73:0/2815764957 learned_addr learned my addr 172.21.15.73:0/2815764957 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:60250/0 (socket says 172.21.15.73:60250) 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 -- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 msgr2=0x55dbaa9221f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 -- 172.21.15.73:0/2815764957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 msgr2=0x55dbaa930cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa930cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458bfff640 1 -- 172.21.15.73:0/2815764957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dbaa922d70 con 0x55dbaa928760 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.137+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f458bfff640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4584004af0 tx=0x7f4584005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef937810b714ee26 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f456affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:04.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4584066060 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9642+0+0 (secure 0 0 0) 0x7f4584004040 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4584036040 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f458405c040 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f4594d6d740 10 client.16398 fetch_fsmap learned FSMap version 36 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.138+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55dbaa9a3170 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4584036040 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4584055040 con 0x55dbaa928760 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 10 client.16398 fetch_fsmap finished waiting for FSMap version 36 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 20 client.16398 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa9a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 4 client.16398 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:08:04.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dbaa88ac20 con 0x55dbaa9387e0 2024-06-06T07:08:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.139+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa9a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.140+0000 7f456affd640 10 client.16398 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:04.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.140+0000 7f458b7fe640 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa9a57a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da44631e168a2fb server_cookie=9796a82ce99e0b0b in_seq=0 out_seq=0 2024-06-06T07:08:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.163+0000 7f456affd640 1 -- 172.21.15.73:0/2815764957 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dbaa88ac20 con 0x55dbaa9387e0 2024-06-06T07:08:04.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.163+0000 7f456affd640 10 client.16398 handle_command_reply: tid=0 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.163+0000 7f4594d6d740 1 client.16398 shutdown 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.163+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 msgr2=0x55dbaa922730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.163+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f4584004af0 tx=0x7f4584005da0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 shutdown_connections 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dbaa9387e0 0x55dbaa9a57a0 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dbaa928760 0x55dbaa922730 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dbaa919990 0x55dbaa9221f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 --2- 172.21.15.73:0/2815764957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dbaa919070 0x55dbaa930cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 >> 172.21.15.73:0/2815764957 conn(0x55dbaa915680 msgr2=0x55dbaa9147e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f456affd640 0 client.16398 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 shutdown_connections 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 1 -- 172.21.15.73:0/2815764957 wait complete. 2024-06-06T07:08:04.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.164+0000 7f4594d6d740 20 client.16398 trim_cache size 0 max 16384 2024-06-06T07:08:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:08:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:08:04.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 msgr2=0x7f45740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 0x7f45740a5260 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f458009b6f0 tx=0x7f4580000bc0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 msgr2=0x7f458c198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f458000c920 tx=0x7f458000cde0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 shutdown_connections 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f45740a2da0 0x7f45740a5260 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f458c106890 0x7f458c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f458c101290 0x7f458c198560 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 --2- 172.21.15.73:0/2000129186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f458c1008d0 0x7f458c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 >> 172.21.15.73:0/2000129186 conn(0x7f458c077a20 msgr2=0x7f458c0ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 shutdown_connections 2024-06-06T07:08:04.168 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:04.165+0000 7f4591f7e640 1 -- 172.21.15.73:0/2000129186 wait complete. 2024-06-06T07:08:04.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 150580 2.60 MB/sec execute 191 sec latency 22.238 ms 2024-06-06T07:08:05.148 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:04 smithi170 ceph-mon[36108]: pgmap v205: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-06-06T07:08:05.148 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:04 smithi170 ceph-mon[36108]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:08:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:04 smithi073 ceph-mon[32303]: pgmap v205: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-06-06T07:08:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:04 smithi073 ceph-mon[32303]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:08:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:04 smithi136 ceph-mon[35901]: pgmap v205: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.0 MiB/s wr, 235 op/s 2024-06-06T07:08:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:04 smithi136 ceph-mon[35901]: osdmap e103: 12 total, 12 up, 12 in 2024-06-06T07:08:05.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setuser ceph since I am not root 2024-06-06T07:08:05.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setgroup ceph since I am not root 2024-06-06T07:08:05.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:05 smithi136 ceph-mds[69750]: -- 172.21.15.136:0/1065645886 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e126dda40 con 0x563e126b0c00 2024-06-06T07:08:05.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: starting mds.e at 2024-06-06T07:08:05.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 151146 2.59 MB/sec execute 192 sec latency 19.380 ms 2024-06-06T07:08:06.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:05 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:08:06.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:05 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:08:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:05 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:08:06.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 151597 2.59 MB/sec execute 193 sec latency 16.412 ms 2024-06-06T07:08:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: pgmap v207: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-06-06T07:08:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:08:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/3276267646,v1:172.21.15.136:6835/3276267646] up:boot 2024-06-06T07:08:07.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:07.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:07.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:06 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: pgmap v207: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-06-06T07:08:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:08:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/3276267646,v1:172.21.15.136:6835/3276267646] up:boot 2024-06-06T07:08:07.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:07.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:07.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:06 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: pgmap v207: 129 pgs: 129 active+clean; 102 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 197 op/s 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: osdmap e104: 12 total, 12 up, 12 in 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/3276267646,v1:172.21.15.136:6835/3276267646] up:boot 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:06 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:07.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 152299 2.61 MB/sec execute 194 sec latency 15.327 ms 2024-06-06T07:08:08.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 152687 2.62 MB/sec execute 195 sec latency 15.762 ms 2024-06-06T07:08:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:08 smithi073 ceph-mon[32303]: pgmap v209: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:08 smithi136 ceph-mon[35901]: pgmap v209: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:09.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:08 smithi170 ceph-mon[36108]: pgmap v209: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:09.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 152972 2.61 MB/sec execute 196 sec latency 12.366 ms 2024-06-06T07:08:10.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 153321 2.60 MB/sec execute 197 sec latency 18.202 ms 2024-06-06T07:08:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:10 smithi073 ceph-mon[32303]: pgmap v210: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:10 smithi136 ceph-mon[35901]: pgmap v210: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:10 smithi170 ceph-mon[36108]: pgmap v210: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.2 MiB/s wr, 238 op/s 2024-06-06T07:08:11.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 153801 2.59 MB/sec execute 198 sec latency 18.152 ms 2024-06-06T07:08:12.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 154325 2.60 MB/sec execute 199 sec latency 25.202 ms 2024-06-06T07:08:13.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 154891 2.59 MB/sec execute 200 sec latency 22.553 ms 2024-06-06T07:08:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:13 smithi073 ceph-mon[32303]: pgmap v211: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T07:08:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:13 smithi073 ceph-mon[32303]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:08:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:13 smithi136 ceph-mon[35901]: pgmap v211: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T07:08:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:13 smithi136 ceph-mon[35901]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:08:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:13 smithi170 ceph-mon[36108]: pgmap v211: 129 pgs: 129 active+clean; 99 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 853 KiB/s rd, 1.0 MiB/s wr, 103 op/s 2024-06-06T07:08:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:13 smithi170 ceph-mon[36108]: osdmap e105: 12 total, 12 up, 12 in 2024-06-06T07:08:14.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 155449 2.60 MB/sec execute 201 sec latency 16.329 ms 2024-06-06T07:08:15.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 155909 2.60 MB/sec execute 202 sec latency 13.170 ms 2024-06-06T07:08:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:15 smithi073 ceph-mon[32303]: pgmap v213: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-06-06T07:08:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:15 smithi136 ceph-mon[35901]: pgmap v213: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-06-06T07:08:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:15 smithi170 ceph-mon[36108]: pgmap v213: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.8 MiB/s wr, 212 op/s 2024-06-06T07:08:16.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 156354 2.61 MB/sec execute 203 sec latency 14.018 ms 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- 172.21.15.73:0/253365850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00100880 msgr2=0x7fba00100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- 172.21.15.73:0/253365850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00100880 0x7fba00100d00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009930 tx=0x7fb9f002a420 comp rx=0 tx=0).stop 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- 172.21.15.73:0/253365850 shutdown_connections 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- 172.21.15.73:0/253365850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00101240 0x7fba0010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- 172.21.15.73:0/253365850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00100880 0x7fba00100d00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- 172.21.15.73:0/253365850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00106890 0x7fba00106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- 172.21.15.73:0/253365850 >> 172.21.15.73:0/253365850 conn(0x7fba000fc370 msgr2=0x7fba000fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- 172.21.15.73:0/253365850 shutdown_connections 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- 172.21.15.73:0/253365850 wait complete. 2024-06-06T07:08:17.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 Processor -- start 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- start start 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00101240 0x7fba001972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba00110b50 con 0x7fba00106890 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba001109d0 con 0x7fba00100880 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.261+0000 7fba082ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fba00110cd0 con 0x7fba00101240 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0608a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39332/0 (socket says 172.21.15.73:39332) 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0608a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47312/0 (socket says 172.21.15.73:47312) 2024-06-06T07:08:17.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 -- 172.21.15.73:0/1931874734 learned_addr learned my addr 172.21.15.73:0/1931874734 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba05889640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00101240 0x7fba001972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 -- 172.21.15.73:0/1931874734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00101240 msgr2=0x7fba001972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00101240 0x7fba001972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 -- 172.21.15.73:0/1931874734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 msgr2=0x7fba00196da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 -- 172.21.15.73:0/1931874734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001a6610 con 0x7fba00106890 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0608a640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba0688b640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009fd0 tx=0x7fb9f800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fb9f77fe640 1 -- 172.21.15.73:0/1931874734 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8017070 con 0x7fba00106890 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fb9f77fe640 1 -- 172.21.15.73:0/1931874734 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9f8003e20 con 0x7fba00106890 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001a6900 con 0x7fba00106890 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fb9f77fe640 1 -- 172.21.15.73:0/1931874734 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8010460 con 0x7fba00106890 2024-06-06T07:08:17.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.262+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba001a6dd0 con 0x7fba00106890 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.265+0000 7fb9f77fe640 1 -- 172.21.15.73:0/1931874734 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb9f8003970 con 0x7fba00106890 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.265+0000 7fba0aff7740 1 Processor -- start 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.265+0000 7fba0aff7740 1 -- start start 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 0x555620cb9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cba350 0x555620cbc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555620c134f0 con 0x555620caf170 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555620bb1870 con 0x555620cba350 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0aff7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555620c0f4b0 con 0x555620cafa90 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 0x555620cb9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39336/0 (socket says 172.21.15.73:39336) 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba05889640 1 -- 172.21.15.73:0/3844793570 learned_addr learned my addr 172.21.15.73:0/3844793570 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fb9f77fe640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 0x7fb9ec0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0688b640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 0x7fb9ec0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 -- 172.21.15.73:0/3844793570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cba350 msgr2=0x555620cbc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cba350 0x555620cbc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 -- 172.21.15.73:0/3844793570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 msgr2=0x555620caf550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 -- 172.21.15.73:0/3844793570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555620bcd9b0 con 0x555620cafa90 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba05889640 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fb9f77fe640 1 -- 172.21.15.73:0/1931874734 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7fb9f80cf600 con 0x7fba00106890 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0608a640 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 0x555620cb9e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f50 tx=0x7fb9fc0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daebd74df75f282 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fba0688b640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 0x7fb9ec0a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80ccf40 tx=0x7fb9f806a6f0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.266+0000 7fb9f4bfe640 1 -- 172.21.15.73:0/3844793570 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004030 con 0x555620cafa90 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fb9f4bfe640 1 -- 172.21.15.73:0/3844793570 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9fc005020 con 0x555620cafa90 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fb9f4bfe640 1 -- 172.21.15.73:0/3844793570 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004c40 con 0x555620cafa90 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 -- 172.21.15.73:0/3844793570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 msgr2=0x555620cb9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 0x555620cb9e10 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc009f50 tx=0x7fb9fc0118d0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 -- 172.21.15.73:0/3844793570 shutdown_connections 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cba350 0x555620cbc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620cafa90 0x555620cb9e10 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3844793570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620caf170 0x555620caf550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 -- 172.21.15.73:0/3844793570 >> 172.21.15.73:0/3844793570 conn(0x555620cab680 msgr2=0x555620caba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 -- 172.21.15.73:0/3844793570 shutdown_connections 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.267+0000 7fba0aff7740 1 -- 172.21.15.73:0/3844793570 wait complete. 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 -- start start 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 Processor -- start 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cb7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555620cbcf80 con 0x555620cba350 2024-06-06T07:08:17.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555620b9cae0 con 0x555620cafa90 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555620cbd100 con 0x555620caf170 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0aff7740 10 client.0 _ll_register_callbacks cb 0x555620bd59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39344/0 (socket says 172.21.15.73:39344) 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba05889640 1 -- 172.21.15.73:0/285959125 learned_addr learned my addr 172.21.15.73:0/285959125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba05889640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.268+0000 7fba0608a640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cb7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 -- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 msgr2=0x555620cb7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 -- 172.21.15.73:0/285959125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 msgr2=0x555620cb7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cb7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 -- 172.21.15.73:0/285959125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555620927210 con 0x555620cba350 2024-06-06T07:08:17.277 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba05889640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009230 tx=0x7fb9f0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4030b6acfb905bcd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fb9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0035040 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7fb9f003d600 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9fc005020 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb9f0053050 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba0aff7740 10 client.16416 fetch_fsmap learned FSMap version 38 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5556209f3620 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.269+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9fc005020 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb9f003a6d0 con 0x555620cba350 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 10 client.16416 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 10 client.16416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 20 client.16416 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:17.278 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620cca4b0 0x555620ccc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 4 client.16416 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555620c26f10 con 0x555620cca4b0 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0608a640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620cca4b0 0x555620ccc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fb9e9ffb640 10 client.16416 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.270+0000 7fba0608a640 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620cca4b0 0x555620ccc8a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b9a07cd2c8682ff server_cookie=14eb2b58e32969d6 in_seq=0 out_seq=0 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/285959125 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555620c26f10 con 0x555620cca4b0 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fb9e9ffb640 10 client.16416 handle_command_reply: tid=0 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 client.16416 shutdown 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 msgr2=0x555620cb7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009230 tx=0x7fb9f0005fd0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 shutdown_connections 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620cca4b0 0x555620ccc8a0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.279 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cb7da0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cb7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fb9e9ffb640 0 client.16416 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 --2- 172.21.15.73:0/285959125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 >> 172.21.15.73:0/285959125 conn(0x555620cab680 msgr2=0x555620cb1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 shutdown_connections 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 1 -- 172.21.15.73:0/285959125 wait complete. 2024-06-06T07:08:17.280 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.271+0000 7fba0aff7740 20 client.16416 trim_cache size 0 max 16384 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 Processor -- start 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 -- start start 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cd2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cd2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b1120 con 0x555620cba350 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b2c60 con 0x555620cafa90 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0aff7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b17f0 con 0x555620caf170 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cd2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba05889640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba05889640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43844/0 (socket says 172.21.15.73:43844) 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba05889640 1 -- 172.21.15.73:0/3956626430 learned_addr learned my addr 172.21.15.73:0/3956626430 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.288+0000 7fba0608a640 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cd2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 -- 172.21.15.73:0/3956626430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 msgr2=0x555620cb79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 -- 172.21.15.73:0/3956626430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 msgr2=0x555620cd2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cd2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 -- 172.21.15.73:0/3956626430 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555620ca5b50 con 0x555620cafa90 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba05889640 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cd2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fba0608a640 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cd2140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc01b040 tx=0x7fb9fc009d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1461df16f9739835 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fb9ebfff640 1 -- 172.21.15.73:0/3956626430 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc032070 con 0x555620cafa90 2024-06-06T07:08:17.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fb9ebfff640 1 -- 172.21.15.73:0/3956626430 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9fc02a070 con 0x555620cafa90 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.289+0000 7fb9ebfff640 1 -- 172.21.15.73:0/3956626430 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc004bb0 con 0x555620cafa90 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- 172.21.15.73:0/3956626430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 msgr2=0x555620cd2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cd2140 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc01b040 tx=0x7fb9fc009d20 comp rx=0 tx=0).stop 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- 172.21.15.73:0/3956626430 shutdown_connections 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cd2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cd2140 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 --2- 172.21.15.73:0/3956626430 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cb79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- 172.21.15.73:0/3956626430 >> 172.21.15.73:0/3956626430 conn(0x555620cab680 msgr2=0x555620cb72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- 172.21.15.73:0/3956626430 shutdown_connections 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- 172.21.15.73:0/3956626430 wait complete. 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 -- start start 2024-06-06T07:08:17.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.290+0000 7fba0aff7740 1 Processor -- start 2024-06-06T07:08:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cc2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cbdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b2590 con 0x555620cba350 2024-06-06T07:08:17.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fba001b1ec0 con 0x555620cafa90 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fba001af120 con 0x555620caf170 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 10 client.0 _ll_register_callbacks cb 0x7fb9f0009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39368/0 (socket says 172.21.15.73:39368) 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 -- 172.21.15.73:0/764179764 learned_addr learned my addr 172.21.15.73:0/764179764 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cc2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0608a640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cbdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 -- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 msgr2=0x555620cc2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cc2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 -- 172.21.15.73:0/764179764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 msgr2=0x555620cbdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cbdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 -- 172.21.15.73:0/764179764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555620cbec70 con 0x555620cba350 2024-06-06T07:08:17.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba05889640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005ec0 tx=0x7fb9f0043040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c61265d0ad97a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fb9e9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0035070 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9fc02a070 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.291+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 9554+0+0 (secure 0 0 0) 0x7fb9f003a600 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb9f004d2f0 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 10 client.16422 fetch_fsmap learned FSMap version 38 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555620d351b0 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9fc02a070 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb9f004d500 con 0x555620cba350 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 10 client.16422 fetch_fsmap finished waiting for FSMap version 38 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:17.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 20 client.16422 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620d35870 0x555620d37c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 4 client.16422 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555620c20c20 con 0x555620d35870 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.292+0000 7fba0608a640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620d35870 0x555620d37c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.293+0000 7fb9e9ffb640 10 client.16422 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:17.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.293+0000 7fba0608a640 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620d35870 0x555620d37c60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=620d0bad04c093d4 server_cookie=5940c5d571f6d25d in_seq=0 out_seq=0 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.315+0000 7fb9e9ffb640 1 -- 172.21.15.73:0/764179764 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555620c20c20 con 0x555620d35870 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.315+0000 7fb9e9ffb640 10 client.16422 handle_command_reply: tid=0 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.315+0000 7fba0aff7740 1 client.16422 shutdown 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 msgr2=0x555620cbe440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0005ec0 tx=0x7fb9f0043040 comp rx=0 tx=0).stop 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 shutdown_connections 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555620d35870 0x555620d37c60 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555620cba350 0x555620cbe440 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555620cafa90 0x555620cbdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fb9e9ffb640 0 client.16422 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 --2- 172.21.15.73:0/764179764 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555620caf170 0x555620cc2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 >> 172.21.15.73:0/764179764 conn(0x555620cab680 msgr2=0x555620cb13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 shutdown_connections 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 1 -- 172.21.15.73:0/764179764 wait complete. 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba0aff7740 20 client.16422 trim_cache size 0 max 16384 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:08:17.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 msgr2=0x7fb9ec0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 0x7fb9ec0a5260 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9f80ccf40 tx=0x7fb9f806a6f0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 msgr2=0x7fba00197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.316+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009fd0 tx=0x7fb9f800b260 comp rx=0 tx=0).stop 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 shutdown_connections 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba00106890 0x7fba00197820 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9ec0a2da0 0x7fb9ec0a5260 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fba00101240 0x7fba001972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 --2- 172.21.15.73:0/1931874734 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fba00100880 0x7fba00196da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 >> 172.21.15.73:0/1931874734 conn(0x7fba000fc370 msgr2=0x7fba000fcb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 shutdown_connections 2024-06-06T07:08:17.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:17.317+0000 7fba082ee640 1 -- 172.21.15.73:0/1931874734 wait complete. 2024-06-06T07:08:17.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 156627 2.60 MB/sec execute 204 sec latency 23.945 ms 2024-06-06T07:08:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:17 smithi073 ceph-mon[32303]: pgmap v214: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-06T07:08:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:17 smithi136 ceph-mon[35901]: pgmap v214: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-06T07:08:17.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:17 smithi170 ceph-mon[36108]: pgmap v214: 129 pgs: 129 active+clean; 100 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 171 op/s 2024-06-06T07:08:18.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setuser ceph since I am not root 2024-06-06T07:08:18.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setgroup ceph since I am not root 2024-06-06T07:08:18.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:18 smithi136 ceph-mds[69750]: -- 172.21.15.136:0/1785412123 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562184f45a40 con 0x562184f18800 2024-06-06T07:08:18.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:08:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: starting mds.e at 2024-06-06T07:08:18.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 157036 2.59 MB/sec execute 205 sec latency 19.400 ms 2024-06-06T07:08:18.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:18 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:08:18.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:18 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:08:18.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:18 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:08:19.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 157521 2.60 MB/sec execute 206 sec latency 44.935 ms 2024-06-06T07:08:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: pgmap v215: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-06T07:08:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:08:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/297801767,v1:172.21.15.136:6835/297801767] up:boot 2024-06-06T07:08:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:19.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:19 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: pgmap v215: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-06T07:08:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:08:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/297801767,v1:172.21.15.136:6835/297801767] up:boot 2024-06-06T07:08:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:19.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:19 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: pgmap v215: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 174 op/s 2024-06-06T07:08:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: osdmap e106: 12 total, 12 up, 12 in 2024-06-06T07:08:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/297801767,v1:172.21.15.136:6835/297801767] up:boot 2024-06-06T07:08:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:08:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:08:19.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:19 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:08:20.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 158020 2.59 MB/sec execute 207 sec latency 18.883 ms 2024-06-06T07:08:21.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 158548 2.58 MB/sec execute 208 sec latency 18.634 ms 2024-06-06T07:08:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:21 smithi073 ceph-mon[32303]: pgmap v217: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-06-06T07:08:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:21 smithi136 ceph-mon[35901]: pgmap v217: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-06-06T07:08:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:21 smithi170 ceph-mon[36108]: pgmap v217: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 218 op/s 2024-06-06T07:08:22.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 159287 2.60 MB/sec execute 209 sec latency 15.586 ms 2024-06-06T07:08:23.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 159659 2.61 MB/sec execute 210 sec latency 15.002 ms 2024-06-06T07:08:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:23 smithi073 ceph-mon[32303]: pgmap v218: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-06-06T07:08:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:23 smithi136 ceph-mon[35901]: pgmap v218: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-06-06T07:08:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:23 smithi170 ceph-mon[36108]: pgmap v218: 129 pgs: 129 active+clean; 101 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 187 op/s 2024-06-06T07:08:24.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 159986 2.60 MB/sec execute 211 sec latency 15.074 ms 2024-06-06T07:08:25.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 160251 2.59 MB/sec execute 212 sec latency 16.394 ms 2024-06-06T07:08:25.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:25 smithi073 ceph-mon[32303]: pgmap v219: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:25.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:25 smithi136 ceph-mon[35901]: pgmap v219: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:25.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:25 smithi170 ceph-mon[36108]: pgmap v219: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:26.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 160722 2.59 MB/sec execute 213 sec latency 15.040 ms 2024-06-06T07:08:27.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 161236 2.59 MB/sec execute 214 sec latency 18.544 ms 2024-06-06T07:08:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:27 smithi073 ceph-mon[32303]: pgmap v220: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:27 smithi136 ceph-mon[35901]: pgmap v220: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:27 smithi170 ceph-mon[36108]: pgmap v220: 129 pgs: 129 active+clean; 103 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.7 MiB/s wr, 170 op/s 2024-06-06T07:08:28.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 161822 2.58 MB/sec execute 215 sec latency 20.483 ms 2024-06-06T07:08:29.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 162257 2.58 MB/sec execute 216 sec latency 14.668 ms 2024-06-06T07:08:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:29 smithi073 ceph-mon[32303]: pgmap v221: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-06T07:08:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:29 smithi136 ceph-mon[35901]: pgmap v221: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-06T07:08:29.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:29 smithi170 ceph-mon[36108]: pgmap v221: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-06T07:08:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 -- 172.21.15.73:0/774495420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4106890 msgr2=0x7fe3c4106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.415 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 --2- 172.21.15.73:0/774495420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4106890 0x7fe3c4106c70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009930 tx=0x7fe3c002a420 comp rx=0 tx=0).stop 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 -- 172.21.15.73:0/774495420 shutdown_connections 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 --2- 172.21.15.73:0/774495420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c4101290 0x7fe3c410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 --2- 172.21.15.73:0/774495420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c41008d0 0x7fe3c4100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 --2- 172.21.15.73:0/774495420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4106890 0x7fe3c4106c70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 -- 172.21.15.73:0/774495420 >> 172.21.15.73:0/774495420 conn(0x7fe3c4077a20 msgr2=0x7fe3c4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 -- 172.21.15.73:0/774495420 shutdown_connections 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.414+0000 7fe3cb163640 1 -- 172.21.15.73:0/774495420 wait complete. 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 Processor -- start 2024-06-06T07:08:30.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 -- start start 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c41008d0 0x7fe3c4198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c4106890 0x7fe3c41a4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110b50 con 0x7fe3c4101290 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41109d0 con 0x7fe3c4106890 2024-06-06T07:08:30.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3cb163640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c4110cd0 con 0x7fe3c41008d0 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48900/0 (socket says 172.21.15.73:48900) 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3bbfff640 1 -- 172.21.15.73:0/3938583039 learned_addr learned my addr 172.21.15.73:0/3938583039 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.415+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c41008d0 0x7fe3c4198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c4106890 0x7fe3c41a4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 -- 172.21.15.73:0/3938583039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c41008d0 msgr2=0x7fe3c4198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c41008d0 0x7fe3c4198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 -- 172.21.15.73:0/3938583039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c4106890 msgr2=0x7fe3c41a4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c4106890 0x7fe3c41a4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 -- 172.21.15.73:0/3938583039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3c41a4590 con 0x7fe3c4101290 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3b9ffb640 1 -- 172.21.15.73:0/3938583039 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b4017030 con 0x7fe3c4101290 2024-06-06T07:08:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3b9ffb640 1 -- 172.21.15.73:0/3938583039 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3b4004020 con 0x7fe3c4101290 2024-06-06T07:08:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3c41a4760 con 0x7fe3c4101290 2024-06-06T07:08:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3b9ffb640 1 -- 172.21.15.73:0/3938583039 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40047a0 con 0x7fe3c4101290 2024-06-06T07:08:30.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.416+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3c41a4cb0 con 0x7fe3c4101290 2024-06-06T07:08:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.419+0000 7fe3b9ffb640 1 -- 172.21.15.73:0/3938583039 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe3b40041d0 con 0x7fe3c4101290 2024-06-06T07:08:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.419+0000 7fe3cde7b740 1 Processor -- start 2024-06-06T07:08:30.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.419+0000 7fe3cde7b740 1 -- start start 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e35740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5650f3d8c4f0 con 0x5650f3e33350 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5650f3d2a870 con 0x5650f3e28170 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650f3d884b0 con 0x5650f3e28a90 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e35740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56446/0 (socket says 172.21.15.73:56446) 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3b9ffb640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 0x7fe3ac0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 -- 172.21.15.73:0/3964126461 learned_addr learned my addr 172.21.15.73:0/3964126461 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:30.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 0x7fe3ac0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 -- 172.21.15.73:0/3964126461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 msgr2=0x5650f3e28550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e28550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 -- 172.21.15.73:0/3964126461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 msgr2=0x5650f3e35740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e35740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 -- 172.21.15.73:0/3964126461 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f3d469b0 con 0x5650f3e28a90 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3b9ffb640 1 -- 172.21.15.73:0/3938583039 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe3b40cf1a0 con 0x7fe3c4101290 2024-06-06T07:08:30.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.420+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a940 tx=0x7fe3c0031ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acd18456ab4b42f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 0x7fe3ac0a5260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3aaffd640 1 -- 172.21.15.73:0/3964126461 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0040070 con 0x5650f3e28a90 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3aaffd640 1 -- 172.21.15.73:0/3964126461 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3c0031e50 con 0x5650f3e28a90 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3aaffd640 1 -- 172.21.15.73:0/3964126461 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0035040 con 0x5650f3e28a90 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3964126461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 msgr2=0x5650f3e32e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe3c002a940 tx=0x7fe3c0031ba0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3964126461 shutdown_connections 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e35740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e32e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3964126461 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e28550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.421+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3964126461 >> 172.21.15.73:0/3964126461 conn(0x5650f3e24680 msgr2=0x5650f3e24a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3964126461 shutdown_connections 2024-06-06T07:08:30.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3964126461 wait complete. 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- start start 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 Processor -- start 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e30330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3e30db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5650f3e35f20 con 0x5650f3e28a90 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5650f3d15ae0 con 0x5650f3e33350 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5650f3e360a0 con 0x5650f3e28170 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.422+0000 7fe3cde7b740 10 client.0 _ll_register_callbacks cb 0x5650f3d4e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e30330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48926/0 (socket says 172.21.15.73:48926) 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3bbfff640 1 -- 172.21.15.73:0/4265422912 learned_addr learned my addr 172.21.15.73:0/4265422912 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:30.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c9700640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3e30db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 -- 172.21.15.73:0/4265422912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 msgr2=0x5650f3e30db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3e30db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 -- 172.21.15.73:0/4265422912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 msgr2=0x5650f3e30870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 -- 172.21.15.73:0/4265422912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f3aa0210 con 0x5650f3e28170 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e30330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0004620 tx=0x7fe3c0005490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=145a8ddc38c52dc5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3c004b020 con 0x5650f3e28170 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3c0031e50 con 0x5650f3e28170 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe3c0004770 con 0x5650f3e28170 2024-06-06T07:08:30.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe3c0056050 con 0x5650f3e28170 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3cde7b740 10 client.25972 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.423+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650f3b6c620 con 0x5650f3e28170 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3c0031e50 con 0x5650f3e28170 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe3c003c560 con 0x5650f3e28170 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 10 client.25972 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 10 client.25972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 20 client.25972 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3e43350 0x5650f3e45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 4 client.25972 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650f3d9ff10 con 0x5650f3e43350 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3e43350 0x5650f3e45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3a3fff640 10 client.25972 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:30.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.424+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3e43350 0x5650f3e45740 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353440b39e374842 server_cookie=6d51a52b6120b974 in_seq=0 out_seq=0 2024-06-06T07:08:30.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3a3fff640 1 -- 172.21.15.73:0/4265422912 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650f3d9ff10 con 0x5650f3e43350 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3a3fff640 10 client.25972 handle_command_reply: tid=0 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 client.25972 shutdown 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 msgr2=0x5650f3e30330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e30330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0004620 tx=0x7fe3c0005490 comp rx=0 tx=0).stop 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 shutdown_connections 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3e43350 0x5650f3e45740 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3e30db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e30870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3a3fff640 0 client.25972 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/4265422912 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e30330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 >> 172.21.15.73:0/4265422912 conn(0x5650f3e24680 msgr2=0x5650f3e2a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 shutdown_connections 2024-06-06T07:08:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 1 -- 172.21.15.73:0/4265422912 wait complete. 2024-06-06T07:08:30.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.426+0000 7fe3cde7b740 20 client.25972 trim_cache size 0 max 16384 2024-06-06T07:08:30.441 INFO:tasks.workunit.client.0.smithi073.stdout: 1 162944 2.60 MB/sec execute 217 sec latency 15.911 ms 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.445+0000 7fe3cde7b740 1 Processor -- start 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.445+0000 7fe3cde7b740 1 -- start start 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e48760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e48ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41aedc0 con 0x5650f3e33350 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41b0900 con 0x5650f3e28170 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41af490 con 0x5650f3e28a90 2024-06-06T07:08:30.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48930/0 (socket says 172.21.15.73:48930) 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 -- 172.21.15.73:0/3072189262 learned_addr learned my addr 172.21.15.73:0/3072189262 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e48760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e48ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 -- 172.21.15.73:0/3072189262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 msgr2=0x5650f3e48ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e48ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 -- 172.21.15.73:0/3072189262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 msgr2=0x5650f3e48760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e48760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 -- 172.21.15.73:0/3072189262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650f3e288c0 con 0x5650f3e33350 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3c9700640 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc010960 tx=0x7fe3bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fe47c2263cd07d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3aa7fc640 1 -- 172.21.15.73:0/3072189262 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc01f590 con 0x5650f3e33350 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3aa7fc640 1 -- 172.21.15.73:0/3072189262 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3bc01fbe0 con 0x5650f3e33350 2024-06-06T07:08:30.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.446+0000 7fe3aa7fc640 1 -- 172.21.15.73:0/3072189262 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3bc01cb90 con 0x5650f3e33350 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3072189262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 msgr2=0x5650f3e2d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc010960 tx=0x7fe3bc018000 comp rx=0 tx=0).stop 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3072189262 shutdown_connections 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e33350 0x5650f3e2d590 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28a90 0x5650f3e48ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/3072189262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e28170 0x5650f3e48760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3072189262 >> 172.21.15.73:0/3072189262 conn(0x5650f3e24680 msgr2=0x5650f3e2ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3072189262 shutdown_connections 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- 172.21.15.73:0/3072189262 wait complete. 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.447+0000 7fe3cde7b740 1 -- start start 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 Processor -- start 2024-06-06T07:08:30.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e45ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3ea5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41b0230 con 0x5650f3e28a90 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41afb60 con 0x5650f3e33350 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3c41acda0 con 0x5650f3e28170 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3cde7b740 10 client.0 _ll_register_callbacks cb 0x7fe3b409b0e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48934/0 (socket says 172.21.15.73:48934) 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 -- 172.21.15.73:0/1340028554 learned_addr learned my addr 172.21.15.73:0/1340028554 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e45ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3c9700640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3ea5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 -- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 msgr2=0x5650f3e45ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e45ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 -- 172.21.15.73:0/1340028554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 msgr2=0x5650f3ea5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3ea5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3bbfff640 1 -- 172.21.15.73:0/1340028554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650f3ea6280 con 0x5650f3e28a90 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3c9700640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3ea5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.448+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3bbfff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40cf5d0 tx=0x7fe3b40d4040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3707c30d610f0626 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40a3f80 con 0x5650f3e28a90 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe3b40aae80 con 0x5650f3e28a90 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3bc01fbe0 con 0x5650f3e28a90 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe3b409b310 con 0x5650f3e28a90 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 10 client.16446 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:30.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5650f3ead6a0 con 0x5650f3e28a90 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3bc01fbe0 con 0x5650f3e28a90 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe3b40b4080 con 0x5650f3e28a90 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 10 client.16446 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 20 client.16446 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3eade20 0x5650f3eb0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 4 client.16446 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.449+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650f3d99c20 con 0x5650f3eade20 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.450+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3eade20 0x5650f3eb0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.450+0000 7fe3a3fff640 10 client.16446 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:30.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.450+0000 7fe3c8eff640 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3eade20 0x5650f3eb0240 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7caa034188d13d8d server_cookie=ebac063326e9a79e in_seq=0 out_seq=0 2024-06-06T07:08:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3a3fff640 1 -- 172.21.15.73:0/1340028554 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5650f3d99c20 con 0x5650f3eade20 2024-06-06T07:08:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3a3fff640 10 client.16446 handle_command_reply: tid=0 2024-06-06T07:08:30.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 client.16446 shutdown 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 msgr2=0x5650f3e463e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fe3b40cf5d0 tx=0x7fe3b40d4040 comp rx=0 tx=0).stop 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 shutdown_connections 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5650f3eade20 0x5650f3eb0240 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5650f3e33350 0x5650f3ea5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3a3fff640 0 client.16446 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5650f3e28a90 0x5650f3e463e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 --2- 172.21.15.73:0/1340028554 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5650f3e28170 0x5650f3e45ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.476+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 >> 172.21.15.73:0/1340028554 conn(0x5650f3e24680 msgr2=0x5650f3e29b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 shutdown_connections 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cde7b740 1 -- 172.21.15.73:0/1340028554 wait complete. 2024-06-06T07:08:30.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cde7b740 20 client.16446 trim_cache size 0 max 16384 2024-06-06T07:08:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:08:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:08:30.479 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 msgr2=0x7fe3ac0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 0x7fe3ac0a5260 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc005fd0 tx=0x7fe3bc009500 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 msgr2=0x7fe3c4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fe3b400d950 tx=0x7fe3b400de10 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 shutdown_connections 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3c4106890 0x7fe3c41a4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3c4101290 0x7fe3c4199510 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3ac0a2da0 0x7fe3ac0a5260 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 --2- 172.21.15.73:0/3938583039 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3c41008d0 0x7fe3c4198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 >> 172.21.15.73:0/3938583039 conn(0x7fe3c4077a20 msgr2=0x7fe3c4103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 shutdown_connections 2024-06-06T07:08:30.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:30.477+0000 7fe3cb163640 1 -- 172.21.15.73:0/3938583039 wait complete. 2024-06-06T07:08:31.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 163349 2.60 MB/sec execute 218 sec latency 24.556 ms 2024-06-06T07:08:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:31 smithi073 ceph-mon[32303]: pgmap v222: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-06T07:08:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:31 smithi136 ceph-mon[35901]: pgmap v222: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-06T07:08:31.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:31 smithi170 ceph-mon[36108]: pgmap v222: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-06T07:08:32.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 163618 2.60 MB/sec execute 219 sec latency 14.198 ms 2024-06-06T07:08:33.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 163974 2.59 MB/sec execute 220 sec latency 16.887 ms 2024-06-06T07:08:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:33 smithi073 ceph-mon[32303]: pgmap v223: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-06T07:08:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:33 smithi136 ceph-mon[35901]: pgmap v223: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-06T07:08:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:33 smithi170 ceph-mon[36108]: pgmap v223: 129 pgs: 129 active+clean; 106 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-06T07:08:34.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 164422 2.58 MB/sec execute 221 sec latency 16.189 ms 2024-06-06T07:08:35.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 164948 2.59 MB/sec execute 222 sec latency 20.577 ms 2024-06-06T07:08:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:35 smithi073 ceph-mon[32303]: pgmap v224: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:35 smithi136 ceph-mon[35901]: pgmap v224: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:35.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:35 smithi170 ceph-mon[36108]: pgmap v224: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:36.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 165526 2.58 MB/sec execute 223 sec latency 17.811 ms 2024-06-06T07:08:37.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 166111 2.59 MB/sec execute 224 sec latency 11.124 ms 2024-06-06T07:08:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:37 smithi136 ceph-mon[35901]: pgmap v225: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-06T07:08:37.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:37 smithi170 ceph-mon[36108]: pgmap v225: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-06T07:08:38.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:37 smithi073 ceph-mon[32303]: pgmap v225: 129 pgs: 129 active+clean; 109 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-06T07:08:38.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 166555 2.59 MB/sec execute 225 sec latency 13.858 ms 2024-06-06T07:08:38.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:38 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:08:38.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:38 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:08:39.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:38 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:08:39.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 166989 2.60 MB/sec execute 226 sec latency 18.462 ms 2024-06-06T07:08:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:39 smithi136 ceph-mon[35901]: pgmap v226: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:39 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:08:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:39 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:08:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:39 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:08:39.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:39 smithi170 ceph-mon[36108]: pgmap v226: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:39 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:08:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:39 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:08:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:39 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:08:40.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:39 smithi073 ceph-mon[32303]: pgmap v226: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:40.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:39 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:08:40.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:39 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:08:40.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:39 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:08:40.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 167253 2.59 MB/sec execute 227 sec latency 14.922 ms 2024-06-06T07:08:41.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 167676 2.58 MB/sec execute 228 sec latency 17.087 ms 2024-06-06T07:08:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:41 smithi136 ceph-mon[35901]: pgmap v227: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:41.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:41 smithi170 ceph-mon[36108]: pgmap v227: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:42.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:41 smithi073 ceph-mon[32303]: pgmap v227: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:42.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 168167 2.59 MB/sec execute 229 sec latency 19.704 ms 2024-06-06T07:08:43.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 168689 2.58 MB/sec execute 230 sec latency 20.944 ms 2024-06-06T07:08:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 -- 172.21.15.73:0/1083656758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c103570 msgr2=0x7ff94c10ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 --2- 172.21.15.73:0/1083656758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c103570 0x7ff94c10ba50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff940009900 tx=0x7ff94002a480 comp rx=0 tx=0).stop 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 -- 172.21.15.73:0/1083656758 shutdown_connections 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 --2- 172.21.15.73:0/1083656758 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c10bf90 0x7ff94c10e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 --2- 172.21.15.73:0/1083656758 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c103570 0x7ff94c10ba50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 --2- 172.21.15.73:0/1083656758 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 -- 172.21.15.73:0/1083656758 >> 172.21.15.73:0/1083656758 conn(0x7ff94c077a20 msgr2=0x7ff94c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 -- 172.21.15.73:0/1083656758 shutdown_connections 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.576+0000 7ff95202a640 1 -- 172.21.15.73:0/1083656758 wait complete. 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 Processor -- start 2024-06-06T07:08:43.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 -- start start 2024-06-06T07:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 0x7ff94c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 0x7ff94c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c111c10 con 0x7ff94c10bf90 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c111a90 con 0x7ff94c103570 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff95202a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c111d90 con 0x7ff94c102c50 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 0x7ff94c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.577+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35338/0 (socket says 172.21.15.73:35338) 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 -- 172.21.15.73:0/4155413578 learned_addr learned my addr 172.21.15.73:0/4155413578 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94affd640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 0x7ff94c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 -- 172.21.15.73:0/4155413578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 msgr2=0x7ff94c1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 0x7ff94c1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 -- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 msgr2=0x7ff94c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 0x7ff94c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 -- 172.21.15.73:0/4155413578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94c19d160 con 0x7ff94c102c50 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94affd640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 0x7ff94c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94bfff640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 0x7ff94c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00e9f0 tx=0x7ff93c00edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff948ff9640 1 -- 172.21.15.73:0/4155413578 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c00cd80 con 0x7ff94c102c50 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff948ff9640 1 -- 172.21.15.73:0/4155413578 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93c0054d0 con 0x7ff94c102c50 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94c19d450 con 0x7ff94c102c50 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff948ff9640 1 -- 172.21.15.73:0/4155413578 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c010490 con 0x7ff94c102c50 2024-06-06T07:08:43.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.578+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94c1a4ce0 con 0x7ff94c102c50 2024-06-06T07:08:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff948ff9640 1 -- 172.21.15.73:0/4155413578 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff93c005020 con 0x7ff94c102c50 2024-06-06T07:08:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 Processor -- start 2024-06-06T07:08:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 -- start start 2024-06-06T07:08:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff948ff9640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 0x7ff9340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02020220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02020760 0x563a02022b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563a01f754f0 con 0x563a02011050 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563a01f13870 con 0x563a02020760 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff954e12740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563a01f714b0 con 0x563a02011970 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff94bfff640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 0x7ff9340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff948ff9640 1 -- 172.21.15.73:0/4155413578 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7ff93c0cfb60 con 0x7ff94c102c50 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41048/0 (socket says 172.21.15.73:41048) 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.581+0000 7ff94b7fe640 1 -- 172.21.15.73:0/771472296 learned_addr learned my addr 172.21.15.73:0/771472296 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94affd640 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02020220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94bfff640 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02020760 0x563a02022b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.589 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 -- 172.21.15.73:0/771472296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 msgr2=0x563a02020220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02020220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 -- 172.21.15.73:0/771472296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02020760 msgr2=0x563a02022b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02020760 0x563a02022b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 -- 172.21.15.73:0/771472296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a01f2f9b0 con 0x563a02011050 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94bfff640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 0x7ff9340a5260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff9440099e0 tx=0x7ff94400fe70 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0986a0 tx=0x7ff93c0980a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=729843e88ac5587f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff931ffb640 1 -- 172.21.15.73:0/771472296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c005680 con 0x563a02011050 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff931ffb640 1 -- 172.21.15.73:0/771472296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93c09c3d0 con 0x563a02011050 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.582+0000 7ff931ffb640 1 -- 172.21.15.73:0/771472296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0cd180 con 0x563a02011050 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- 172.21.15.73:0/771472296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 msgr2=0x563a02011430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0986a0 tx=0x7ff93c0980a0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- 172.21.15.73:0/771472296 shutdown_connections 2024-06-06T07:08:43.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02020760 0x563a02022b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02020220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 --2- 172.21.15.73:0/771472296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02011430 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- 172.21.15.73:0/771472296 >> 172.21.15.73:0/771472296 conn(0x563a0200d680 msgr2=0x563a0200da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- 172.21.15.73:0/771472296 shutdown_connections 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- 172.21.15.73:0/771472296 wait complete. 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.583+0000 7ff954e12740 1 -- start start 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 Processor -- start 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 0x563a0201d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a0201dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563a02023390 con 0x563a02020760 2024-06-06T07:08:43.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563a01efeae0 con 0x563a02011970 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563a02023510 con 0x563a02011050 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff954e12740 10 client.0 _ll_register_callbacks cb 0x563a01f379b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41060/0 (socket says 172.21.15.73:41060) 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 -- 172.21.15.73:0/2636183985 learned_addr learned my addr 172.21.15.73:0/2636183985 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 0x563a0201d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94affd640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a0201dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 -- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 msgr2=0x563a0201d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 0x563a0201d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 -- 172.21.15.73:0/2636183985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 msgr2=0x563a0201dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a0201dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94bfff640 1 -- 172.21.15.73:0/2636183985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a01c89210 con 0x563a02020760 2024-06-06T07:08:43.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.584+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 0x563a0201d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff94bfff640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff9440123c0 tx=0x7ff944004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f82273995b421de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944024030 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c09c3d0 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7ff9440046d0 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff944031050 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 10 client.16458 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563a01d55620 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93c09c3d0 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff94401f070 con 0x563a02020760 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 10 client.16458 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 20 client.16458 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:43.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.585+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a02030970 0x563a02032d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.586+0000 7ff954e12740 4 client.16458 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.586+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a01f88f10 con 0x563a02030970 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.586+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a02030970 0x563a02032d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.586+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a02030970 0x563a02032d60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5649f6e2caf0cd2 server_cookie=ed75eeb87d43308b in_seq=0 out_seq=0 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.586+0000 7ff92affd640 10 client.16458 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff92affd640 1 -- 172.21.15.73:0/2636183985 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a01f88f10 con 0x563a02030970 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff92affd640 10 client.16458 handle_command_reply: tid=0 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 client.16458 shutdown 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 msgr2=0x563a0201e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff9440123c0 tx=0x7ff944004ab0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 shutdown_connections 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a02030970 0x563a02032d60 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.594 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201e140 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a0201dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff92affd640 0 client.16458 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 --2- 172.21.15.73:0/2636183985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011050 0x563a0201d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 >> 172.21.15.73:0/2636183985 conn(0x563a0200d680 msgr2=0x563a02013470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 shutdown_connections 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.587+0000 7ff954e12740 1 -- 172.21.15.73:0/2636183985 wait complete. 2024-06-06T07:08:43.595 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.588+0000 7ff954e12740 20 client.16458 trim_cache size 0 max 16384 2024-06-06T07:08:43.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 Processor -- start 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 -- start start 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a02038b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02020760 0x563a0201a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1acbc0 con 0x563a02011050 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ae700 con 0x563a02011970 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff954e12740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ad290 con 0x563a02020760 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41068/0 (socket says 172.21.15.73:41068) 2024-06-06T07:08:43.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94b7fe640 1 -- 172.21.15.73:0/15343313 learned_addr learned my addr 172.21.15.73:0/15343313 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94bfff640 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02020760 0x563a0201a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94affd640 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a02038b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.605+0000 7ff94b7fe640 1 -- 172.21.15.73:0/15343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02020760 msgr2=0x563a0201a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02020760 0x563a0201a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff94b7fe640 1 -- 172.21.15.73:0/15343313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 msgr2=0x563a02038b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a02038b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff94b7fe640 1 -- 172.21.15.73:0/15343313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a020117a0 con 0x563a02011050 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c09bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d35d8aa3fab5c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff9317fa640 1 -- 172.21.15.73:0/15343313 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c09c3d0 con 0x563a02011050 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff9317fa640 1 -- 172.21.15.73:0/15343313 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93c0a0080 con 0x563a02011050 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.606+0000 7ff9317fa640 1 -- 172.21.15.73:0/15343313 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff93c0a4710 con 0x563a02011050 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- 172.21.15.73:0/15343313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 msgr2=0x563a02038630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7ff93c0097f0 tx=0x7ff93c09bbd0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- 172.21.15.73:0/15343313 shutdown_connections 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02020760 0x563a0201a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011970 0x563a02038b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 --2- 172.21.15.73:0/15343313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02011050 0x563a02038630 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- 172.21.15.73:0/15343313 >> 172.21.15.73:0/15343313 conn(0x563a0200d680 msgr2=0x563a0201df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- 172.21.15.73:0/15343313 shutdown_connections 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- 172.21.15.73:0/15343313 wait complete. 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 -- start start 2024-06-06T07:08:43.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.607+0000 7ff954e12740 1 Processor -- start 2024-06-06T07:08:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011050 0x563a02028a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02028fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ae030 con 0x563a02020760 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1ad960 con 0x563a02011050 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff94c1aaba0 con 0x563a02011970 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 10 client.0 _ll_register_callbacks cb 0x7ff944014e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41070/0 (socket says 172.21.15.73:41070) 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 -- 172.21.15.73:0/3021189569 learned_addr learned my addr 172.21.15.73:0/3021189569 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011050 0x563a02028a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94affd640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02028fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 -- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 msgr2=0x563a02028fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02028fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 -- 172.21.15.73:0/3021189569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011050 msgr2=0x563a02028a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011050 0x563a02028a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 -- 172.21.15.73:0/3021189569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a0201a950 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff94bfff640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff9440123c0 tx=0x7ff94400f8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2a79061a3550940 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff92affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94403a060 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7ff9440106c0 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff93c0a0080 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7ff944041050 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 10 client.16464 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.608+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x563a0209b1f0 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff93c0a0080 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7ff94401f040 con 0x563a02020760 2024-06-06T07:08:43.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 10 client.16464 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 20 client.16464 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a0209b910 0x563a0209dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 4 client.16464 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a01f82c20 con 0x563a0209b910 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a0209b910 0x563a0209dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff92affd640 10 client.16464 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:43.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.609+0000 7ff94b7fe640 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a0209b910 0x563a0209dd30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6191039e68c691fe server_cookie=629e395ef919dfdd in_seq=0 out_seq=0 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff92affd640 1 -- 172.21.15.73:0/3021189569 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563a01f82c20 con 0x563a0209b910 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff92affd640 10 client.16464 handle_command_reply: tid=0 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 client.16464 shutdown 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 msgr2=0x563a0201a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7ff9440123c0 tx=0x7ff94400f8a0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 shutdown_connections 2024-06-06T07:08:43.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563a0209b910 0x563a0209dd30 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563a02020760 0x563a0201a120 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563a02011970 0x563a02028fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 --2- 172.21.15.73:0/3021189569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563a02011050 0x563a02028a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff92affd640 0 client.16464 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 >> 172.21.15.73:0/3021189569 conn(0x563a0200d680 msgr2=0x563a0200bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 shutdown_connections 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 1 -- 172.21.15.73:0/3021189569 wait complete. 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.631+0000 7ff954e12740 20 client.16464 trim_cache size 0 max 16384 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:08:43.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 msgr2=0x7ff9340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 0x7ff9340a5260 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff9440099e0 tx=0x7ff94400fe70 comp rx=0 tx=0).stop 2024-06-06T07:08:43.634 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 msgr2=0x7ff94c198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff93c00e9f0 tx=0x7ff93c00edf0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 shutdown_connections 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff94c10bf90 0x7ff94c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff94c103570 0x7ff94c1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff9340a2da0 0x7ff9340a5260 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 --2- 172.21.15.73:0/4155413578 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff94c102c50 0x7ff94c198170 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 >> 172.21.15.73:0/4155413578 conn(0x7ff94c077a20 msgr2=0x7ff94c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 shutdown_connections 2024-06-06T07:08:43.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:43.632+0000 7ff95202a640 1 -- 172.21.15.73:0/4155413578 wait complete. 2024-06-06T07:08:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:43 smithi136 ceph-mon[35901]: pgmap v228: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:43.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:43 smithi170 ceph-mon[36108]: pgmap v228: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:43 smithi073 ceph-mon[32303]: pgmap v228: 129 pgs: 129 active+clean; 111 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.4 MiB/s wr, 139 op/s 2024-06-06T07:08:44.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 169168 2.58 MB/sec execute 231 sec latency 26.395 ms 2024-06-06T07:08:45.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 169900 2.59 MB/sec execute 232 sec latency 12.393 ms 2024-06-06T07:08:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:45 smithi136 ceph-mon[35901]: pgmap v229: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-06-06T07:08:45.825 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:45 smithi073 ceph-mon[32303]: pgmap v229: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-06-06T07:08:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:45 smithi170 ceph-mon[36108]: pgmap v229: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 208 op/s 2024-06-06T07:08:46.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 170277 2.60 MB/sec execute 233 sec latency 13.890 ms 2024-06-06T07:08:47.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 170619 2.59 MB/sec execute 234 sec latency 12.804 ms 2024-06-06T07:08:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:47 smithi136 ceph-mon[35901]: pgmap v230: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-06T07:08:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:47 smithi170 ceph-mon[36108]: pgmap v230: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-06T07:08:48.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:47 smithi073 ceph-mon[32303]: pgmap v230: 129 pgs: 129 active+clean; 115 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 1.2 MiB/s wr, 137 op/s 2024-06-06T07:08:48.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 170878 2.58 MB/sec execute 235 sec latency 16.301 ms 2024-06-06T07:08:49.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 171315 2.58 MB/sec execute 236 sec latency 45.165 ms 2024-06-06T07:08:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:49 smithi136 ceph-mon[35901]: pgmap v231: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:49.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:49 smithi170 ceph-mon[36108]: pgmap v231: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:49 smithi073 ceph-mon[32303]: pgmap v231: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 1.9 MiB/s wr, 211 op/s 2024-06-06T07:08:50.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 171792 2.58 MB/sec execute 237 sec latency 18.364 ms 2024-06-06T07:08:51.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 172339 2.57 MB/sec execute 238 sec latency 23.315 ms 2024-06-06T07:08:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:51 smithi136 ceph-mon[35901]: pgmap v232: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:51 smithi170 ceph-mon[36108]: pgmap v232: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:52.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:51 smithi073 ceph-mon[32303]: pgmap v232: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:52.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 172775 2.57 MB/sec execute 239 sec latency 21.997 ms 2024-06-06T07:08:53.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 173514 2.59 MB/sec execute 240 sec latency 16.295 ms 2024-06-06T07:08:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:53 smithi136 ceph-mon[35901]: pgmap v233: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:53 smithi170 ceph-mon[36108]: pgmap v233: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:54.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:53 smithi073 ceph-mon[32303]: pgmap v233: 129 pgs: 129 active+clean; 113 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-06T07:08:54.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 173892 2.59 MB/sec execute 241 sec latency 14.937 ms 2024-06-06T07:08:54.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:54 smithi170 ceph-mon[36108]: pgmap v234: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-06-06T07:08:55.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:54 smithi073 ceph-mon[32303]: pgmap v234: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-06-06T07:08:55.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:54 smithi136 ceph-mon[35901]: pgmap v234: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 207 op/s 2024-06-06T07:08:55.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 174199 2.59 MB/sec execute 242 sec latency 12.514 ms 2024-06-06T07:08:56.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 174513 2.58 MB/sec execute 243 sec latency 14.683 ms 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- 172.21.15.73:0/2570390669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a41008d0 msgr2=0x7fe1a4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/2570390669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a41008d0 0x7fe1a4100d50 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009930 tx=0x7fe18c02a420 comp rx=0 tx=0).stop 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- 172.21.15.73:0/2570390669 shutdown_connections 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/2570390669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a4101290 0x7fe1a410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/2570390669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a41008d0 0x7fe1a4100d50 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/2570390669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4106890 0x7fe1a4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- 172.21.15.73:0/2570390669 >> 172.21.15.73:0/2570390669 conn(0x7fe1a4077a20 msgr2=0x7fe1a4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- 172.21.15.73:0/2570390669 shutdown_connections 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- 172.21.15.73:0/2570390669 wait complete. 2024-06-06T07:08:56.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 Processor -- start 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 -- start start 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.730+0000 7fe1a97ed640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 0x7fe1a4197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 0x7fe1a4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4110b50 con 0x7fe1a4106890 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41109d0 con 0x7fe1a41008d0 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a4110cd0 con 0x7fe1a4101290 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 0x7fe1a4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 0x7fe1a4197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60282/0 (socket says 172.21.15.73:60282) 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 0x7fe1a4197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55918/0 (socket says 172.21.15.73:55918) 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 -- 172.21.15.73:0/607248623 learned_addr learned my addr 172.21.15.73:0/607248623 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 0x7fe1a4197570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:33796/0 (socket says 172.21.15.73:33796) 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 -- 172.21.15.73:0/607248623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 msgr2=0x7fe1a4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 0x7fe1a4197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 -- 172.21.15.73:0/607248623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 msgr2=0x7fe1a4197030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 0x7fe1a4197030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 -- 172.21.15.73:0/607248623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1a419c020 con 0x7fe1a4106890 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a37fe640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00eb10 tx=0x7fe19c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a0ff9640 1 -- 172.21.15.73:0/607248623 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c004b20 con 0x7fe1a4106890 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1a419c310 con 0x7fe1a4106890 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a0ff9640 1 -- 172.21.15.73:0/607248623 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe19c004cc0 con 0x7fe1a4106890 2024-06-06T07:08:56.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a0ff9640 1 -- 172.21.15.73:0/607248623 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19c010690 con 0x7fe1a4106890 2024-06-06T07:08:56.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.731+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1a41a3ba0 con 0x7fe1a4106890 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1a0ff9640 1 -- 172.21.15.73:0/607248623 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe19c005020 con 0x7fe1a4106890 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 Processor -- start 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 -- start start 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x56319536a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 0x563195377740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5631952ce4f0 con 0x56319536aa90 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56319526c870 con 0x56319536a170 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.734+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5631952ca4b0 con 0x563195375350 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a0ff9640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 0x7fe1900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60292/0 (socket says 172.21.15.73:60292) 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/2371569502 learned_addr learned my addr 172.21.15.73:0/2371569502 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 0x563195377740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a37fe640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 0x7fe1900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x56319536a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a0ff9640 1 -- 172.21.15.73:0/607248623 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe19c0cfd90 con 0x7fe1a4106890 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 -- 172.21.15.73:0/2371569502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 msgr2=0x56319536a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x56319536a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 -- 172.21.15.73:0/2371569502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 msgr2=0x563195374e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 -- 172.21.15.73:0/2371569502 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631952889b0 con 0x563195375350 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe19bfff640 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 0x563195377740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe18c031440 tx=0x7fe18c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e332cc08ab223d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1a37fe640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 0x7fe1900a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe19c09bc00 tx=0x7fe19c00cee0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.735+0000 7fe1997fa640 1 -- 172.21.15.73:0/2371569502 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c035070 con 0x563195375350 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1997fa640 1 -- 172.21.15.73:0/2371569502 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe18c0318e0 con 0x563195375350 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1997fa640 1 -- 172.21.15.73:0/2371569502 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0053a0 con 0x563195375350 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/2371569502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 msgr2=0x563195377740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 0x563195377740 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe18c031440 tx=0x7fe18c005fd0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/2371569502 shutdown_connections 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563195375350 0x563195377740 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195374e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/2371569502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x56319536a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/2371569502 >> 172.21.15.73:0/2371569502 conn(0x563195366680 msgr2=0x563195366a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/2371569502 shutdown_connections 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.736+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/2371569502 wait complete. 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 -- start start 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 Processor -- start 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536a170 0x563195372340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x563195372dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563195377ec0 con 0x56319536aa90 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563195257ae0 con 0x563195375350 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563195378040 con 0x56319536a170 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1ac5cf740 10 client.0 _ll_register_callbacks cb 0x5631952909b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60300/0 (socket says 172.21.15.73:60300) 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536a170 0x563195372340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3006065939 learned_addr learned my addr 172.21.15.73:0/3006065939 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x563195372dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536a170 msgr2=0x563195372340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536a170 0x563195372340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3006065939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 msgr2=0x563195372dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x563195372dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3006065939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563194fe2210 con 0x56319536aa90 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.737+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x563195372dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:08:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe19400ba30 tx=0x7fe19400bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4581d44c872bd182 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe194011440 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe1940115e0 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5631952e1f10 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe1940117e0 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 10 client.16476 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5631950ae620 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5631952e1f10 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe19401b920 con 0x56319536aa90 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 10 client.16476 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 10 client.16476 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 20 client.16476 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563195385550 0x563195387960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 4 client.16476 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.738+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631952dbc20 con 0x563195385550 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.739+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563195385550 0x563195387960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.739+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563195385550 0x563195387960 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acbe694e5ef05a93 server_cookie=5e2d410c61605f5 in_seq=0 out_seq=0 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.739+0000 7fe18a7fc640 10 client.16476 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe18a7fc640 1 -- 172.21.15.73:0/3006065939 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5631952dbc20 con 0x563195385550 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe18a7fc640 10 client.16476 handle_command_reply: tid=0 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 client.16476 shutdown 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 msgr2=0x563195372880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe19400ba30 tx=0x7fe19400bef0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 shutdown_connections 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563195385550 0x563195387960 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe18a7fc640 0 client.16476 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:56.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x563195372dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536aa90 0x563195372880 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3006065939 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536a170 0x563195372340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 >> 172.21.15.73:0/3006065939 conn(0x563195366680 msgr2=0x56319536c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 shutdown_connections 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3006065939 wait complete. 2024-06-06T07:08:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.740+0000 7fe1ac5cf740 20 client.16476 trim_cache size 0 max 16384 2024-06-06T07:08:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.756+0000 7fe1ac5cf740 1 Processor -- start 2024-06-06T07:08:56.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.756+0000 7fe1ac5cf740 1 -- start start 2024-06-06T07:08:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x563195365f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41acbf0 con 0x563195375350 2024-06-06T07:08:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41ae730 con 0x56319536a170 2024-06-06T07:08:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41ad2c0 con 0x56319536aa90 2024-06-06T07:08:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60316/0 (socket says 172.21.15.73:60316) 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe19bfff640 1 -- 172.21.15.73:0/3911763614 learned_addr learned my addr 172.21.15.73:0/3911763614 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x563195365f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3911763614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 msgr2=0x563195365f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x563195365f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3911763614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 msgr2=0x56319538d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 -- 172.21.15.73:0/3911763614 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563195367000 con 0x56319536aa90 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe19bfff640 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.757+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536cb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe19401bf80 tx=0x7fe194003a80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ffaf82b338a17acb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.758+0000 7fe198ff9640 1 -- 172.21.15.73:0/3911763614 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19402d070 con 0x56319536aa90 2024-06-06T07:08:56.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.758+0000 7fe198ff9640 1 -- 172.21.15.73:0/3911763614 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe194011440 con 0x56319536aa90 2024-06-06T07:08:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.758+0000 7fe198ff9640 1 -- 172.21.15.73:0/3911763614 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1940054d0 con 0x56319536aa90 2024-06-06T07:08:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.758+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3911763614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 msgr2=0x56319536cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.758+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536cb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe19401bf80 tx=0x7fe194003a80 comp rx=0 tx=0).stop 2024-06-06T07:08:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3911763614 shutdown_connections 2024-06-06T07:08:56.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563195375350 0x56319538d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536cb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/3911763614 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56319536a170 0x563195365f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3911763614 >> 172.21.15.73:0/3911763614 conn(0x563195366680 msgr2=0x563195372b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3911763614 shutdown_connections 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/3911763614 wait complete. 2024-06-06T07:08:56.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- start start 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 Processor -- start 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x56319536f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41ae060 con 0x56319536a170 2024-06-06T07:08:56.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41ad990 con 0x563195375350 2024-06-06T07:08:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe1a41aabd0 con 0x56319536aa90 2024-06-06T07:08:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.759+0000 7fe1ac5cf740 10 client.0 _ll_register_callbacks cb 0x7fe1940042d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:08:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60328/0 (socket says 172.21.15.73:60328) 2024-06-06T07:08:56.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 -- 172.21.15.73:0/1670925199 learned_addr learned my addr 172.21.15.73:0/1670925199 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe1a2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x56319536f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 -- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 msgr2=0x56319536f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 -- 172.21.15.73:0/1670925199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 msgr2=0x56319536f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x56319536f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 -- 172.21.15.73:0/1670925199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56319536fdd0 con 0x56319536a170 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe19bfff640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe18c004620 tx=0x7fe18c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f96b043ddda8ec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe18a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c035040 con 0x56319536a170 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9466+0+0 (secure 0 0 0) 0x7fe18c005020 con 0x56319536a170 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.760+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe194011440 con 0x56319536a170 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7fe18c063050 con 0x56319536a170 2024-06-06T07:08:56.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 10 client.16482 fetch_fsmap learned FSMap version 40 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5631953f01c0 con 0x56319536a170 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe194011440 con 0x56319536a170 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) ==== 8362+0+0 (secure 0 0 0) 0x7fe18c03c700 con 0x56319536a170 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 10 client.16482 fetch_fsmap finished waiting for FSMap version 40 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 10 client.16482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 20 client.16482 populate_metadata read hostname 'smithi073' 2024-06-06T07:08:56.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5631953f0550 0x5631953f2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:08:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 4 client.16482 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:08:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631952d5f70 con 0x5631953f0550 2024-06-06T07:08:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.761+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5631953f0550 0x5631953f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:08:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.762+0000 7fe1a2ffd640 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5631953f0550 0x5631953f2970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecca4581f412c54 server_cookie=205abfb2ca546595 in_seq=0 out_seq=0 2024-06-06T07:08:56.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.762+0000 7fe18a7fc640 10 client.16482 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.790+0000 7fe18a7fc640 1 -- 172.21.15.73:0/1670925199 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5631952d5f70 con 0x5631953f0550 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.790+0000 7fe18a7fc640 10 client.16482 handle_command_reply: tid=0 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.790+0000 7fe1ac5cf740 1 client.16482 shutdown 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 msgr2=0x56319537d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe18c004620 tx=0x7fe18c000bc0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 shutdown_connections 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5631953f0550 0x5631953f2970 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe18a7fc640 0 client.16482 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563195375350 0x56319536f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56319536aa90 0x56319536f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 --2- 172.21.15.73:0/1670925199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56319536a170 0x56319537d9e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 >> 172.21.15.73:0/1670925199 conn(0x563195366680 msgr2=0x563195380110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 shutdown_connections 2024-06-06T07:08:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 1 -- 172.21.15.73:0/1670925199 wait complete. 2024-06-06T07:08:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.791+0000 7fe1ac5cf740 20 client.16482 trim_cache size 0 max 16384 2024-06-06T07:08:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:08:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:08:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:08:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 msgr2=0x7fe1900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 0x7fe1900a5260 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe19c09bc00 tx=0x7fe19c00cee0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 msgr2=0x7fe1a419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:08:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe19c00eb10 tx=0x7fe19c00c2d0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.794 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 shutdown_connections 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1a4106890 0x7fe1a419b8f0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe1900a2da0 0x7fe1900a5260 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe1a4101290 0x7fe1a4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 --2- 172.21.15.73:0/607248623 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe1a41008d0 0x7fe1a4197030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 >> 172.21.15.73:0/607248623 conn(0x7fe1a4077a20 msgr2=0x7fe1a4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 shutdown_connections 2024-06-06T07:08:56.795 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:08:56.792+0000 7fe1a97ed640 1 -- 172.21.15.73:0/607248623 wait complete. 2024-06-06T07:08:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:56 smithi073 ceph-mon[32303]: pgmap v235: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-06T07:08:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:56 smithi136 ceph-mon[35901]: pgmap v235: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-06T07:08:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:56 smithi170 ceph-mon[36108]: pgmap v235: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 918 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-06T07:08:57.442 INFO:tasks.workunit.client.0.smithi073.stdout: 1 174931 2.57 MB/sec execute 244 sec latency 27.466 ms 2024-06-06T07:08:58.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 175435 2.58 MB/sec execute 245 sec latency 29.226 ms 2024-06-06T07:08:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:08:58 smithi073 ceph-mon[32303]: pgmap v236: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:08:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:08:58 smithi136 ceph-mon[35901]: pgmap v236: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:08:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:08:58 smithi170 ceph-mon[36108]: pgmap v236: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 213 op/s 2024-06-06T07:08:59.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 176009 2.57 MB/sec execute 246 sec latency 43.195 ms 2024-06-06T07:09:00.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 176430 2.57 MB/sec execute 247 sec latency 16.075 ms 2024-06-06T07:09:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:00 smithi073 ceph-mon[32303]: pgmap v237: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:00 smithi136 ceph-mon[35901]: pgmap v237: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:01.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:00 smithi170 ceph-mon[36108]: pgmap v237: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:01.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 177116 2.58 MB/sec execute 248 sec latency 15.011 ms 2024-06-06T07:09:02.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 177515 2.59 MB/sec execute 249 sec latency 15.586 ms 2024-06-06T07:09:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:02 smithi073 ceph-mon[32303]: pgmap v238: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:02 smithi136 ceph-mon[35901]: pgmap v238: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:03.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:02 smithi170 ceph-mon[36108]: pgmap v238: 129 pgs: 129 active+clean; 110 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-06T07:09:03.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 177788 2.58 MB/sec execute 250 sec latency 17.601 ms 2024-06-06T07:09:04.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 178121 2.57 MB/sec execute 251 sec latency 20.609 ms 2024-06-06T07:09:05.149 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:04 smithi170 ceph-mon[36108]: pgmap v239: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-06-06T07:09:05.149 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:04 smithi170 ceph-mon[36108]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:09:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:04 smithi073 ceph-mon[32303]: pgmap v239: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-06-06T07:09:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:04 smithi073 ceph-mon[32303]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:09:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:04 smithi136 ceph-mon[35901]: pgmap v239: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.9 MiB/s wr, 224 op/s 2024-06-06T07:09:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:04 smithi136 ceph-mon[35901]: osdmap e107: 12 total, 12 up, 12 in 2024-06-06T07:09:05.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 178573 2.57 MB/sec execute 252 sec latency 18.878 ms 2024-06-06T07:09:06.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 179070 2.57 MB/sec execute 253 sec latency 26.080 ms 2024-06-06T07:09:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:06 smithi073 ceph-mon[32303]: pgmap v241: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-06T07:09:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:06 smithi136 ceph-mon[35901]: pgmap v241: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-06T07:09:07.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:06 smithi170 ceph-mon[36108]: pgmap v241: 129 pgs: 129 active+clean; 113 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.7 MiB/s wr, 192 op/s 2024-06-06T07:09:07.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 179694 2.57 MB/sec execute 254 sec latency 19.601 ms 2024-06-06T07:09:08.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 180243 2.57 MB/sec execute 255 sec latency 13.561 ms 2024-06-06T07:09:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:08 smithi073 ceph-mon[32303]: pgmap v242: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:08 smithi136 ceph-mon[35901]: pgmap v242: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:09.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:08 smithi170 ceph-mon[36108]: pgmap v242: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:09.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 180693 2.58 MB/sec execute 256 sec latency 77.999 ms 2024-06-06T07:09:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.894+0000 7f9402761640 1 -- 172.21.15.73:0/4240975090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc102c30 msgr2=0x7f93fc103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.894+0000 7f9402761640 1 --2- 172.21.15.73:0/4240975090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc102c30 0x7f93fc103010 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec02a480 comp rx=0 tx=0).stop 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- 172.21.15.73:0/4240975090 shutdown_connections 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- 172.21.15.73:0/4240975090 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc10c000 0x7f93fc10e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- 172.21.15.73:0/4240975090 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc103550 0x7f93fc10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- 172.21.15.73:0/4240975090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc102c30 0x7f93fc103010 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- 172.21.15.73:0/4240975090 >> 172.21.15.73:0/4240975090 conn(0x7f93fc0779c0 msgr2=0x7f93fc077dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- 172.21.15.73:0/4240975090 shutdown_connections 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- 172.21.15.73:0/4240975090 wait complete. 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 Processor -- start 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- start start 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc102c30 0x7f93fc198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc10c000 0x7f93fc19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc111c40 con 0x7f93fc103550 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc111ac0 con 0x7f93fc10c000 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f9402761640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc111dc0 con 0x7f93fc102c30 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55820/0 (socket says 172.21.15.73:55820) 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.895+0000 7f93f3fff640 1 -- 172.21.15.73:0/24315492 learned_addr learned my addr 172.21.15.73:0/24315492 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93fbfff640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc102c30 0x7f93fc198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f9400cfe640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc10c000 0x7f93fc19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 -- 172.21.15.73:0/24315492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc102c30 msgr2=0x7f93fc198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc102c30 0x7f93fc198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 -- 172.21.15.73:0/24315492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc10c000 msgr2=0x7f93fc19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc10c000 0x7f93fc19c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 -- 172.21.15.73:0/24315492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93fc19d120 con 0x7f93fc103550 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f3fff640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f9ffb640 1 -- 172.21.15.73:0/24315492 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400cd80 con 0x7f93fc103550 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f9ffb640 1 -- 172.21.15.73:0/24315492 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e40108e0 con 0x7f93fc103550 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93fc19d410 con 0x7f93fc103550 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f93f9ffb640 1 -- 172.21.15.73:0/24315492 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e4018920 con 0x7f93fc103550 2024-06-06T07:09:09.901 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.896+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93fc1a4ca0 con 0x7f93fc103550 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.898+0000 7f93f9ffb640 1 -- 172.21.15.73:0/24315492 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f93e4010a90 con 0x7f93fc103550 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 Processor -- start 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 -- start start 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93f9ffb640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 0x7f93e80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9f4350 0x562fdd9f6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562fdd94d4f0 con 0x562fdd9e9170 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562fdd8eb870 con 0x562fdd9f4350 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f940546d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562fdd9494b0 con 0x562fdd9e9a90 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f9400cfe640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 0x7f93e80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55836/0 (socket says 172.21.15.73:55836) 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93fbfff640 1 -- 172.21.15.73:0/2837297190 learned_addr learned my addr 172.21.15.73:0/2837297190 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93f9ffb640 1 -- 172.21.15.73:0/24315492 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9642+0+0 (secure 0 0 0) 0x7f93e40cfae0 con 0x7f93fc103550 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f9400cfe640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9f4350 0x562fdd9f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.899+0000 7f93f3fff640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.907 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 -- 172.21.15.73:0/2837297190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 msgr2=0x562fdd9f3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 -- 172.21.15.73:0/2837297190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9f4350 msgr2=0x562fdd9f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9f4350 0x562fdd9f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 -- 172.21.15.73:0/2837297190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdd9079b0 con 0x562fdd9e9170 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93f3fff640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f9400cfe640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 0x7f93e80a52e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009fd0 tx=0x7f93f4009b00 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93fbfff640 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec032b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fd6f728d4c2081 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93f27fc640 1 -- 172.21.15.73:0/2837297190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec040070 con 0x562fdd9e9170 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93f27fc640 1 -- 172.21.15.73:0/2837297190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93ec032df0 con 0x562fdd9e9170 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.900+0000 7f93f27fc640 1 -- 172.21.15.73:0/2837297190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93ec035040 con 0x562fdd9e9170 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- 172.21.15.73:0/2837297190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 msgr2=0x562fdd9e9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93ec009900 tx=0x7f93ec032b00 comp rx=0 tx=0).stop 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- 172.21.15.73:0/2837297190 shutdown_connections 2024-06-06T07:09:09.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9f4350 0x562fdd9f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 --2- 172.21.15.73:0/2837297190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9170 0x562fdd9e9550 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- 172.21.15.73:0/2837297190 >> 172.21.15.73:0/2837297190 conn(0x562fdd9e5680 msgr2=0x562fdd9e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- 172.21.15.73:0/2837297190 shutdown_connections 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- 172.21.15.73:0/2837297190 wait complete. 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 -- start start 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.901+0000 7f940546d740 1 Processor -- start 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9f12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562fdd9fb330 con 0x562fdd9f4350 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562fdd8d6ae0 con 0x562fdd9e9170 2024-06-06T07:09:09.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562fdd9fb4b0 con 0x562fdd9e9a90 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f940546d740 10 client.0 _ll_register_callbacks cb 0x562fdd90f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9f12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:59162/0 (socket says 172.21.15.73:59162) 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f93f3fff640 1 -- 172.21.15.73:0/313954480 learned_addr learned my addr 172.21.15.73:0/313954480 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 -- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 msgr2=0x562fdd9f1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 -- 172.21.15.73:0/313954480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 msgr2=0x562fdd9f12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9f12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.902+0000 7f9400cfe640 1 -- 172.21.15.73:0/313954480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdd661210 con 0x562fdd9f4350 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f9400cfe640 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9f1d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x562fdd9e6810 tx=0x7f93f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8952aa23fef7f2cf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4010b50 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9642+0+0 (secure 0 0 0) 0x7f93f401c040 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93ec032df0 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f93f4010cf0 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 10 client.16494 fetch_fsmap learned FSMap version 41 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562fdd72d620 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93ec032df0 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f93f4020050 con 0x562fdd9f4350 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 10 client.16494 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 10 client.16494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 20 client.16494 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda088a0 0x562fdda0acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 4 client.16494 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:09:09.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.903+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fdd960f10 con 0x562fdda088a0 2024-06-06T07:09:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.904+0000 7f93fbfff640 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda088a0 0x562fdda0acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.904+0000 7f93db7fe640 10 client.16494 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:09.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.904+0000 7f93fbfff640 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda088a0 0x562fdda0acb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=885c045300d21068 server_cookie=177c165c495025fb in_seq=0 out_seq=0 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f93db7fe640 1 -- 172.21.15.73:0/313954480 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562fdd960f10 con 0x562fdda088a0 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f93db7fe640 10 client.16494 handle_command_reply: tid=0 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 client.16494 shutdown 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 msgr2=0x562fdd9f1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9f1d70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x562fdd9e6810 tx=0x7f93f4004ab0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 shutdown_connections 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda088a0 0x562fdda0acb0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9f1d70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f93db7fe640 0 client.16494 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:09.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9a90 0x562fdd9f1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 --2- 172.21.15.73:0/313954480 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9f12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 >> 172.21.15.73:0/313954480 conn(0x562fdd9e5680 msgr2=0x562fdd9eb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 shutdown_connections 2024-06-06T07:09:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 1 -- 172.21.15.73:0/313954480 wait complete. 2024-06-06T07:09:09.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.905+0000 7f940546d740 20 client.16494 trim_cache size 0 max 16384 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 Processor -- start 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 -- start start 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9ebb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 0x562fdda105c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1aceb0 con 0x562fdd9e9a90 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1aea20 con 0x562fdd9e9170 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f940546d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1ad580 con 0x562fdd9f4350 2024-06-06T07:09:09.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f93f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55848/0 (socket says 172.21.15.73:55848) 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.922+0000 7f93f3fff640 1 -- 172.21.15.73:0/937375765 learned_addr learned my addr 172.21.15.73:0/937375765 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f9400cfe640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 0x562fdda105c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93fbfff640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9ebb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 -- 172.21.15.73:0/937375765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 msgr2=0x562fdda105c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 0x562fdda105c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 -- 172.21.15.73:0/937375765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 msgr2=0x562fdd9ebb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9ebb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 -- 172.21.15.73:0/937375765 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fdda0b330 con 0x562fdd9e9a90 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f9400cfe640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 0x562fdda105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f3fff640 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f93e409b950 tx=0x7f93e40b4da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2ebd80ae7d94fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f1ffb640 1 -- 172.21.15.73:0/937375765 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e409ff50 con 0x562fdd9e9a90 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f1ffb640 1 -- 172.21.15.73:0/937375765 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e40a4810 con 0x562fdd9e9a90 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.923+0000 7f93f1ffb640 1 -- 172.21.15.73:0/937375765 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40b4450 con 0x562fdd9e9a90 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 -- 172.21.15.73:0/937375765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 msgr2=0x562fdd9f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f93e409b950 tx=0x7f93e40b4da0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 -- 172.21.15.73:0/937375765 shutdown_connections 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9f4350 0x562fdda105c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9e9a90 0x562fdd9f3230 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 --2- 172.21.15.73:0/937375765 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9170 0x562fdd9ebb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 -- 172.21.15.73:0/937375765 >> 172.21.15.73:0/937375765 conn(0x562fdd9e5680 msgr2=0x562fdd9f2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 -- 172.21.15.73:0/937375765 shutdown_connections 2024-06-06T07:09:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.924+0000 7f940546d740 1 -- 172.21.15.73:0/937375765 wait complete. 2024-06-06T07:09:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 -- start start 2024-06-06T07:09:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 Processor -- start 2024-06-06T07:09:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 0x562fdd9fc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9a90 0x562fdd9fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1ae350 con 0x562fdd9f4350 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1adc80 con 0x562fdd9e9a90 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f93fc1aaeb0 con 0x562fdd9e9170 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f940546d740 10 client.0 _ll_register_callbacks cb 0x7f93f40138b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 0x562fdd9fc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55862/0 (socket says 172.21.15.73:55862) 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f93fbfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 0x562fdd9fc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:59184/0 (socket says 172.21.15.73:59184) 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 -- 172.21.15.73:0/3048202957 learned_addr learned my addr 172.21.15.73:0/3048202957 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f93f3fff640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9a90 0x562fdd9fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 -- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 msgr2=0x562fdd9fc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 0x562fdd9fc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 -- 172.21.15.73:0/3048202957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9a90 msgr2=0x562fdd9fcb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9a90 0x562fdd9fcb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.925+0000 7f9400cfe640 1 -- 172.21.15.73:0/3048202957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fdda72690 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f9400cfe640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006010 tx=0x7f93f4013470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7901a592d59aed server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f4010b50 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9642+0+0 (secure 0 0 0) 0x7f93f401fc90 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e40a4810 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f93f4004b20 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 10 client.16500 fetch_fsmap learned FSMap version 41 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x562fdda73910 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93e40a4810 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f93f4004040 con 0x562fdd9f4350 2024-06-06T07:09:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 10 client.16500 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 20 client.16500 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda740e0 0x562fdda76500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 4 client.16500 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562fdd95ac20 con 0x562fdda740e0 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.926+0000 7f93fbfff640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda740e0 0x562fdda76500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.927+0000 7f93fbfff640 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda740e0 0x562fdda76500 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e9d6814e5c43cb8 server_cookie=694fe144515e9dec in_seq=0 out_seq=0 2024-06-06T07:09:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.927+0000 7f93db7fe640 10 client.16500 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f93db7fe640 1 -- 172.21.15.73:0/3048202957 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562fdd95ac20 con 0x562fdda740e0 2024-06-06T07:09:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f93db7fe640 10 client.16500 handle_command_reply: tid=0 2024-06-06T07:09:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 client.16500 shutdown 2024-06-06T07:09:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 msgr2=0x562fdd9fd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f93f4006010 tx=0x7f93f4013470 comp rx=0 tx=0).stop 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 shutdown_connections 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562fdda740e0 0x562fdda76500 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562fdd9f4350 0x562fdd9fd050 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f93db7fe640 0 client.16500 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562fdd9e9a90 0x562fdd9fcb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 --2- 172.21.15.73:0/3048202957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562fdd9e9170 0x562fdd9fc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 >> 172.21.15.73:0/3048202957 conn(0x562fdd9e5680 msgr2=0x562fdd9f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 shutdown_connections 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 1 -- 172.21.15.73:0/3048202957 wait complete. 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.952+0000 7f940546d740 20 client.16500 trim_cache size 0 max 16384 2024-06-06T07:09:09.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:09:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:09:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:09:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 msgr2=0x7f93e80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 0x7f93e80a52e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f93f4009fd0 tx=0x7f93f4009b00 comp rx=0 tx=0).stop 2024-06-06T07:09:09.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 msgr2=0x7f93fc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f93e400e9f0 tx=0x7f93e400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 shutdown_connections 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f93fc10c000 0x7f93fc19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93fc103550 0x7f93fc1986c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f93e80a2e20 0x7f93e80a52e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 --2- 172.21.15.73:0/24315492 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f93fc102c30 0x7f93fc198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 >> 172.21.15.73:0/24315492 conn(0x7f93fc0779c0 msgr2=0x7f93fc0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 shutdown_connections 2024-06-06T07:09:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:09.953+0000 7f9402761640 1 -- 172.21.15.73:0/24315492 wait complete. 2024-06-06T07:09:10.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 181107 2.58 MB/sec execute 257 sec latency 26.268 ms 2024-06-06T07:09:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:10 smithi073 ceph-mon[32303]: pgmap v243: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:10 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:10 smithi136 ceph-mon[35901]: pgmap v243: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:10 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:10 smithi170 ceph-mon[36108]: pgmap v243: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:10 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:11.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 181372 2.57 MB/sec execute 258 sec latency 14.938 ms 2024-06-06T07:09:12.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 181776 2.57 MB/sec execute 259 sec latency 16.859 ms 2024-06-06T07:09:13.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 182237 2.57 MB/sec execute 260 sec latency 16.530 ms 2024-06-06T07:09:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:13 smithi073 ceph-mon[32303]: pgmap v244: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:13 smithi073 ceph-mon[32303]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:09:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:13 smithi136 ceph-mon[35901]: pgmap v244: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:13 smithi136 ceph-mon[35901]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:09:13.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:13 smithi170 ceph-mon[36108]: pgmap v244: 129 pgs: 129 active+clean; 115 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-06T07:09:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:13 smithi170 ceph-mon[36108]: osdmap e108: 12 total, 12 up, 12 in 2024-06-06T07:09:14.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 182759 2.57 MB/sec execute 261 sec latency 20.284 ms 2024-06-06T07:09:15.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 183314 2.56 MB/sec execute 262 sec latency 22.351 ms 2024-06-06T07:09:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:15 smithi073 ceph-mon[32303]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-06T07:09:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:15 smithi136 ceph-mon[35901]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-06T07:09:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:15 smithi170 ceph-mon[36108]: pgmap v246: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-06T07:09:16.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 183930 2.57 MB/sec execute 263 sec latency 15.533 ms 2024-06-06T07:09:17.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184424 2.58 MB/sec execute 264 sec latency 16.890 ms 2024-06-06T07:09:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:17 smithi073 ceph-mon[32303]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-06-06T07:09:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:17 smithi136 ceph-mon[35901]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-06-06T07:09:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:17 smithi170 ceph-mon[36108]: pgmap v247: 129 pgs: 129 active+clean; 117 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.5 MiB/s wr, 170 op/s 2024-06-06T07:09:18.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184528 2.57 MB/sec execute 265 sec latency 799.454 ms 2024-06-06T07:09:19.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184528 2.56 MB/sec execute 266 sec latency 1799.500 ms 2024-06-06T07:09:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:19 smithi073 ceph-mon[32303]: pgmap v248: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:19 smithi136 ceph-mon[35901]: pgmap v248: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:19 smithi170 ceph-mon[36108]: pgmap v248: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:20.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184595 2.55 MB/sec execute 267 sec latency 2089.057 ms 2024-06-06T07:09:21.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184595 2.54 MB/sec execute 268 sec latency 1474.441 ms 2024-06-06T07:09:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:21 smithi073 ceph-mon[32303]: pgmap v249: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:21 smithi136 ceph-mon[35901]: pgmap v249: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:21 smithi170 ceph-mon[36108]: pgmap v249: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:22.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184595 2.54 MB/sec execute 269 sec latency 2474.495 ms 2024-06-06T07:09:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 -- 172.21.15.73:0/1858856473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081068d0 msgr2=0x7f5608106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 --2- 172.21.15.73:0/1858856473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081068d0 0x7f5608106cb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f55f8009930 tx=0x7f55f802a420 comp rx=0 tx=0).stop 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 -- 172.21.15.73:0/1858856473 shutdown_connections 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 --2- 172.21.15.73:0/1858856473 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f5608101280 0x7f560810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 --2- 172.21.15.73:0/1858856473 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f56081008c0 0x7f5608100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 --2- 172.21.15.73:0/1858856473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081068d0 0x7f5608106cb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 -- 172.21.15.73:0/1858856473 >> 172.21.15.73:0/1858856473 conn(0x7f5608076c20 msgr2=0x7f5608077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 -- 172.21.15.73:0/1858856473 shutdown_connections 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.049+0000 7f560ce93640 1 -- 172.21.15.73:0/1858856473 wait complete. 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.050+0000 7f560ce93640 1 Processor -- start 2024-06-06T07:09:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.050+0000 7f560ce93640 1 -- start start 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.050+0000 7f560ce93640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.050+0000 7f560ce93640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5608101280 0x7f56081985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56081068d0 0x7f560819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5608110b90 con 0x7f56081008c0 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f5608110a10 con 0x7f5608101280 2024-06-06T07:09:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f5608110d10 con 0x7f56081068d0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40272/0 (socket says 172.21.15.73:40272) 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 -- 172.21.15.73:0/4166041914 learned_addr learned my addr 172.21.15.73:0/4166041914 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56081068d0 0x7f560819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f5605d9b640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5608101280 0x7f56081985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 -- 172.21.15.73:0/4166041914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56081068d0 msgr2=0x7f560819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56081068d0 0x7f560819c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 -- 172.21.15.73:0/4166041914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5608101280 msgr2=0x7f56081985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5608101280 0x7f56081985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 -- 172.21.15.73:0/4166041914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560819d0a0 con 0x7f56081008c0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560659c640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f55f8004770 tx=0x7f55f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f55f77fe640 1 -- 172.21.15.73:0/4166041914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f803f070 con 0x7f56081008c0 2024-06-06T07:09:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560819d330 con 0x7f56081008c0 2024-06-06T07:09:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f55f77fe640 1 -- 172.21.15.73:0/4166041914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55f803b460 con 0x7f56081008c0 2024-06-06T07:09:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f55f77fe640 1 -- 172.21.15.73:0/4166041914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8035070 con 0x7f56081008c0 2024-06-06T07:09:23.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.051+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081a4c20 con 0x7f56081008c0 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.054+0000 7f55f77fe640 1 -- 172.21.15.73:0/4166041914 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f55f803b610 con 0x7f56081008c0 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.054+0000 7f560fba0740 1 Processor -- start 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.054+0000 7f560fba0740 1 -- start start 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114315220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x559114317b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55911426a4f0 con 0x559114306050 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559114208870 con 0x559114306970 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560fba0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591142664b0 con 0x559114315760 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f55f77fe640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 0x7f55f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40276/0 (socket says 172.21.15.73:40276) 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 -- 172.21.15.73:0/4191903095 learned_addr learned my addr 172.21.15.73:0/4191903095 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 0x7f55f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5605d9b640 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x559114317b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f55f77fe640 1 -- 172.21.15.73:0/4166041914 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55f80efb20 con 0x7f56081008c0 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f560659c640 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114315220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 -- 172.21.15.73:0/4191903095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 msgr2=0x559114317b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x559114317b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.061 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 -- 172.21.15.73:0/4191903095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 msgr2=0x559114315220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114315220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 -- 172.21.15.73:0/4191903095 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591142249b0 con 0x559114306050 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.055+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5600012d30 tx=0x7f5600010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2bad08122dcbbf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 0x7f55f00a5260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5600010c40 tx=0x7f560000c000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f55f4ff9640 1 -- 172.21.15.73:0/4191903095 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600024070 con 0x559114306050 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f55f4ff9640 1 -- 172.21.15.73:0/4191903095 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5600009d20 con 0x559114306050 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f55f4ff9640 1 -- 172.21.15.73:0/4191903095 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5600005280 con 0x559114306050 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 -- 172.21.15.73:0/4191903095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 msgr2=0x559114306430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f5600012d30 tx=0x7f5600010740 comp rx=0 tx=0).stop 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 -- 172.21.15.73:0/4191903095 shutdown_connections 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x559114317b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.062 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114315220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 --2- 172.21.15.73:0/4191903095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114306430 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.056+0000 7f560fba0740 1 -- 172.21.15.73:0/4191903095 >> 172.21.15.73:0/4191903095 conn(0x559114302680 msgr2=0x559114302a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- 172.21.15.73:0/4191903095 shutdown_connections 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- 172.21.15.73:0/4191903095 wait complete. 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- start start 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 Processor -- start 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114306970 0x559114312c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114315760 0x5591143131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559114318330 con 0x559114306050 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5591141f3ae0 con 0x559114315760 2024-06-06T07:09:23.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5591143184b0 con 0x559114306970 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560fba0740 10 client.0 _ll_register_callbacks cb 0x55911422c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40292/0 (socket says 172.21.15.73:40292) 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f560659c640 1 -- 172.21.15.73:0/3619391907 learned_addr learned my addr 172.21.15.73:0/3619391907 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.057+0000 7f5605d9b640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114306970 0x559114312c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f5606d9d640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114315760 0x5591143131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 -- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114306970 msgr2=0x559114312c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114306970 0x559114312c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 -- 172.21.15.73:0/3619391907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114315760 msgr2=0x5591143131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114315760 0x5591143131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 -- 172.21.15.73:0/3619391907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559113f7e210 con 0x559114306050 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560659c640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005c20 tx=0x7f55f80bb950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88755253fb0d1fcc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8048040 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55f803b940 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55911427df10 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f55f80bffe0 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 10 client.16512 fetch_fsmap learned FSMap version 41 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55911404a620 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55911427df10 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f55f804b060 con 0x559114306050 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 10 client.16512 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 20 client.16512 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591143257f0 0x559114327be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 4 client.16512 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:09:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.058+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559114277c20 con 0x5591143257f0 2024-06-06T07:09:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.059+0000 7f5605d9b640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591143257f0 0x559114327be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.059+0000 7f5605d9b640 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591143257f0 0x559114327be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a12cd60e8c4800 server_cookie=3909da2087786cb3 in_seq=0 out_seq=0 2024-06-06T07:09:23.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.059+0000 7f55edffb640 10 client.16512 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f55edffb640 1 -- 172.21.15.73:0/3619391907 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559114277c20 con 0x5591143257f0 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f55edffb640 10 client.16512 handle_command_reply: tid=0 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 client.16512 shutdown 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 msgr2=0x559114312720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f55f8005c20 tx=0x7f55f80bb950 comp rx=0 tx=0).stop 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 shutdown_connections 2024-06-06T07:09:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5591143257f0 0x559114327be0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114315760 0x5591143131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f55edffb640 0 client.16512 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114306970 0x559114312c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 --2- 172.21.15.73:0/3619391907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114312720 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 >> 172.21.15.73:0/3619391907 conn(0x559114302680 msgr2=0x559114308470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 shutdown_connections 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 1 -- 172.21.15.73:0/3619391907 wait complete. 2024-06-06T07:09:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.060+0000 7f560fba0740 20 client.16512 trim_cache size 0 max 16384 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 Processor -- start 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 -- start start 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 0x559114312df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911432d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56081acce0 con 0x559114306970 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56081ae850 con 0x559114306050 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560fba0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56081ad3b0 con 0x559114315760 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40298/0 (socket says 172.21.15.73:40298) 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 -- 172.21.15.73:0/4233907632 learned_addr learned my addr 172.21.15.73:0/4233907632 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:23.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5606d9d640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911432d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560659c640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 0x559114312df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 -- 172.21.15.73:0/4233907632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 msgr2=0x55911432d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911432d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 -- 172.21.15.73:0/4233907632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 msgr2=0x559114312df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 0x559114312df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 -- 172.21.15.73:0/4233907632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559114327f50 con 0x559114306970 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f560659c640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 0x559114312df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.076+0000 7f5605d9b640 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000c00 tx=0x7f55fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f31d15b82c229e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f55effff640 1 -- 172.21.15.73:0/4233907632 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc003d40 con 0x559114306970 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f55effff640 1 -- 172.21.15.73:0/4233907632 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55fc0054d0 con 0x559114306970 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f55effff640 1 -- 172.21.15.73:0/4233907632 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55fc0116a0 con 0x559114306970 2024-06-06T07:09:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- 172.21.15.73:0/4233907632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 msgr2=0x55911432d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f55fc000c00 tx=0x7f55fc00b390 comp rx=0 tx=0).stop 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- 172.21.15.73:0/4233907632 shutdown_connections 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911432d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306970 0x55911432d3c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 --2- 172.21.15.73:0/4233907632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306050 0x559114312df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- 172.21.15.73:0/4233907632 >> 172.21.15.73:0/4233907632 conn(0x559114302680 msgr2=0x5591143126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- 172.21.15.73:0/4233907632 shutdown_connections 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- 172.21.15.73:0/4233907632 wait complete. 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.077+0000 7f560fba0740 1 -- start start 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 Processor -- start 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114319480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911431d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f56081ae180 con 0x559114306050 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f56081adab0 con 0x559114306970 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f56081aace0 con 0x559114315760 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560fba0740 10 client.0 _ll_register_callbacks cb 0x7f55f8009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40308/0 (socket says 172.21.15.73:40308) 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 -- 172.21.15.73:0/2859771809 learned_addr learned my addr 172.21.15.73:0/2859771809 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f5605d9b640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114319480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 -- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 msgr2=0x55911431d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911431d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 -- 172.21.15.73:0/2859771809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 msgr2=0x559114319480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114319480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 -- 172.21.15.73:0/2859771809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55911431daa0 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f5605d9b640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114319480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f560659c640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55f80b9f90 tx=0x7f55f80092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1b4d01544c29eb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f55edffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f8048070 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.078+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f55f80bb130 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55fc0054d0 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f55f80da610 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 10 client.16518 fetch_fsmap learned FSMap version 41 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55911431daa0 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55fc0054d0 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 41) ==== 8366+0+0 (secure 0 0 0) 0x7f55f80b7bf0 con 0x559114306050 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 10 client.16518 fetch_fsmap finished waiting for FSMap version 41 2024-06-06T07:09:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 20 client.16518 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559114390140 0x559114392580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 4 client.16518 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559114271f70 con 0x559114390140 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.079+0000 7f5605d9b640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559114390140 0x559114392580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.080+0000 7f55edffb640 10 client.16518 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.080+0000 7f5605d9b640 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559114390140 0x559114392580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a96062434c1426 server_cookie=a0f42515f58f6b00 in_seq=0 out_seq=0 2024-06-06T07:09:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f55edffb640 1 -- 172.21.15.73:0/2859771809 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559114271f70 con 0x559114390140 2024-06-06T07:09:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f55edffb640 10 client.16518 handle_command_reply: tid=0 2024-06-06T07:09:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 client.16518 shutdown 2024-06-06T07:09:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 msgr2=0x559114318f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f55f80b9f90 tx=0x7f55f80092b0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 shutdown_connections 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559114390140 0x559114392580 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559114315760 0x55911431d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559114306970 0x559114319480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 --2- 172.21.15.73:0/2859771809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559114306050 0x559114318f40 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f55edffb640 0 client.16518 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 >> 172.21.15.73:0/2859771809 conn(0x559114302680 msgr2=0x5591143126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 shutdown_connections 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 1 -- 172.21.15.73:0/2859771809 wait complete. 2024-06-06T07:09:23.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.111+0000 7f560fba0740 20 client.16518 trim_cache size 0 max 16384 2024-06-06T07:09:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:09:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:09:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:09:23.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 msgr2=0x7f55f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 0x7f55f00a5260 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5600010c40 tx=0x7f560000c000 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 msgr2=0x7f56081980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f55f8004770 tx=0x7f55f80042f0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 shutdown_connections 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f55f00a2da0 0x7f55f00a5260 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f56081068d0 0x7f560819c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f5608101280 0x7f56081985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 --2- 172.21.15.73:0/4166041914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f56081008c0 0x7f56081980b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 >> 172.21.15.73:0/4166041914 conn(0x7f5608076c20 msgr2=0x7f560806a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 shutdown_connections 2024-06-06T07:09:23.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:23.112+0000 7f560ce93640 1 -- 172.21.15.73:0/4166041914 wait complete. 2024-06-06T07:09:23.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184595 2.53 MB/sec execute 270 sec latency 3474.568 ms 2024-06-06T07:09:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:23 smithi136 ceph-mon[35901]: pgmap v250: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:23 smithi170 ceph-mon[36108]: pgmap v250: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:23 smithi073 ceph-mon[32303]: pgmap v250: 129 pgs: 129 active+clean; 118 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.7 MiB/s wr, 167 op/s 2024-06-06T07:09:24.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184595 2.52 MB/sec execute 271 sec latency 4474.651 ms 2024-06-06T07:09:25.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 184784 2.51 MB/sec execute 272 sec latency 4768.403 ms 2024-06-06T07:09:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:25 smithi136 ceph-mon[35901]: pgmap v251: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-06-06T07:09:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:25 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:25 smithi170 ceph-mon[36108]: pgmap v251: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-06-06T07:09:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:25 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:26.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:25 smithi073 ceph-mon[32303]: pgmap v251: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 163 op/s 2024-06-06T07:09:26.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:25 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:26.443 INFO:tasks.workunit.client.0.smithi073.stdout: 1 185062 2.50 MB/sec execute 273 sec latency 15.684 ms 2024-06-06T07:09:27.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 185528 2.50 MB/sec execute 274 sec latency 15.121 ms 2024-06-06T07:09:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:27 smithi136 ceph-mon[35901]: pgmap v252: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 702 KiB/s wr, 80 op/s 2024-06-06T07:09:27.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:27 smithi170 ceph-mon[36108]: pgmap v252: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 702 KiB/s wr, 80 op/s 2024-06-06T07:09:28.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:27 smithi073 ceph-mon[32303]: pgmap v252: 129 pgs: 129 active+clean; 119 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 702 KiB/s wr, 80 op/s 2024-06-06T07:09:28.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 186050 2.50 MB/sec execute 275 sec latency 18.996 ms 2024-06-06T07:09:29.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 186625 2.50 MB/sec execute 276 sec latency 44.991 ms 2024-06-06T07:09:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:29 smithi136 ceph-mon[35901]: pgmap v253: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-06T07:09:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:29 smithi170 ceph-mon[36108]: pgmap v253: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-06T07:09:30.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:29 smithi073 ceph-mon[32303]: pgmap v253: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 882 KiB/s rd, 1.1 MiB/s wr, 130 op/s 2024-06-06T07:09:30.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 187052 2.50 MB/sec execute 277 sec latency 15.603 ms 2024-06-06T07:09:31.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 187761 2.51 MB/sec execute 278 sec latency 14.851 ms 2024-06-06T07:09:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:31 smithi136 ceph-mon[35901]: pgmap v254: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:31.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:31 smithi170 ceph-mon[36108]: pgmap v254: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:32.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:31 smithi073 ceph-mon[32303]: pgmap v254: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:32.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188073 2.51 MB/sec execute 279 sec latency 256.235 ms 2024-06-06T07:09:33.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188073 2.50 MB/sec execute 280 sec latency 1256.265 ms 2024-06-06T07:09:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:33 smithi136 ceph-mon[35901]: pgmap v255: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:33 smithi170 ceph-mon[36108]: pgmap v255: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:34.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:33 smithi073 ceph-mon[32303]: pgmap v255: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 475 KiB/s wr, 64 op/s 2024-06-06T07:09:34.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188073 2.49 MB/sec execute 281 sec latency 2256.317 ms 2024-06-06T07:09:35.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188140 2.49 MB/sec execute 282 sec latency 2547.628 ms 2024-06-06T07:09:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:35 smithi136 ceph-mon[35901]: pgmap v256: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:35 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:35 smithi170 ceph-mon[36108]: pgmap v256: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:35 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:36.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:35 smithi073 ceph-mon[32303]: pgmap v256: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 858 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:36.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:35 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 -- 172.21.15.73:0/1630850300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1068b0 msgr2=0x7fdb9c106c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1630850300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c106c90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c009930 tx=0x7fdb8c02a420 comp rx=0 tx=0).stop 2024-06-06T07:09:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 -- 172.21.15.73:0/1630850300 shutdown_connections 2024-06-06T07:09:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1630850300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c101260 0x7fdb9c10d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1630850300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1630850300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c106c90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 -- 172.21.15.73:0/1630850300 >> 172.21.15.73:0/1630850300 conn(0x7fdb9c0779b0 msgr2=0x7fdb9c077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 -- 172.21.15.73:0/1630850300 shutdown_connections 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.206+0000 7fdba17f9640 1 -- 172.21.15.73:0/1630850300 wait complete. 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 Processor -- start 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 -- start start 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 0x7fdb9c197610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c110b70 con 0x7fdb9c1008a0 2024-06-06T07:09:36.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1109f0 con 0x7fdb9c1068b0 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdba17f9640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c110cf0 con 0x7fdb9c101260 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42390/0 (socket says 172.21.15.73:42390) 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdb9affd640 1 -- 172.21.15.73:0/1824791310 learned_addr learned my addr 172.21.15.73:0/1824791310 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.207+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 0x7fdb9c197610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 -- 172.21.15.73:0/1824791310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 msgr2=0x7fdb9c197610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 0x7fdb9c197610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 -- 172.21.15.73:0/1824791310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 msgr2=0x7fdb9c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 -- 172.21.15.73:0/1824791310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c19c0c0 con 0x7fdb9c1008a0 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 0x7fdb9c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb9affd640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c004770 tx=0x7fdb8c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb93fff640 1 -- 172.21.15.73:0/1824791310 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c035050 con 0x7fdb9c1008a0 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c19c350 con 0x7fdb9c1008a0 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb93fff640 1 -- 172.21.15.73:0/1824791310 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb8c003aa0 con 0x7fdb9c1008a0 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdb93fff640 1 -- 172.21.15.73:0/1824791310 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c038ae0 con 0x7fdb9c1008a0 2024-06-06T07:09:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.208+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c069b60 con 0x7fdb9c1008a0 2024-06-06T07:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.210+0000 7fdb93fff640 1 -- 172.21.15.73:0/1824791310 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb8c003c50 con 0x7fdb9c1008a0 2024-06-06T07:09:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 Processor -- start 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 -- start start 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb15550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15a90 0x55e82bb1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e82ba794f0 con 0x55e82bb20350 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e82ba17870 con 0x55e82bb15170 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdba45e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e82ba754b0 con 0x55e82bb15a90 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb93fff640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 0x7fdb880a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42402/0 (socket says 172.21.15.73:42402) 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/851633817 learned_addr learned my addr 172.21.15.73:0/851633817 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:36.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 0x7fdb880a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb93fff640 1 -- 172.21.15.73:0/1824791310 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fdb8c0f18d0 con 0x7fdb9c1008a0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9affd640 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15a90 0x55e82bb1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.211+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb15550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/851633817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15a90 msgr2=0x55e82bb1fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15a90 0x55e82bb1fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/851633817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 msgr2=0x55e82bb15550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb15550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/851633817 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e82ba339b0 con 0x55e82bb20350 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 0x7fdb880a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005c40 tx=0x7fdb94009530 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bb40 tx=0x7fdb8400d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8442c3afe571b270 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb917fa640 1 -- 172.21.15.73:0/851633817 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84017420 con 0x55e82bb20350 2024-06-06T07:09:36.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb917fa640 1 -- 172.21.15.73:0/851633817 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb84017a70 con 0x55e82bb20350 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.212+0000 7fdb917fa640 1 -- 172.21.15.73:0/851633817 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400fa60 con 0x55e82bb20350 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- 172.21.15.73:0/851633817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 msgr2=0x55e82bb22740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fdb8400bb40 tx=0x7fdb8400d340 comp rx=0 tx=0).stop 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- 172.21.15.73:0/851633817 shutdown_connections 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb22740 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15a90 0x55e82bb1fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 --2- 172.21.15.73:0/851633817 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb15550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- 172.21.15.73:0/851633817 >> 172.21.15.73:0/851633817 conn(0x55e82bb11680 msgr2=0x55e82bb11a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- 172.21.15.73:0/851633817 shutdown_connections 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- 172.21.15.73:0/851633817 wait complete. 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 -- start start 2024-06-06T07:09:36.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.213+0000 7fdba45e4740 1 Processor -- start 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb2a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e82bb22f80 con 0x55e82bb15170 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e82ba02ae0 con 0x55e82bb15a90 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e82bb23100 con 0x55e82bb20350 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 10 client.0 _ll_register_callbacks cb 0x55e82ba3b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42418/0 (socket says 172.21.15.73:42418) 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 -- 172.21.15.73:0/901202254 learned_addr learned my addr 172.21.15.73:0/901202254 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb2a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 -- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 msgr2=0x55e82bb246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 -- 172.21.15.73:0/901202254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 msgr2=0x55e82bb2a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb2a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 -- 172.21.15.73:0/901202254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e82b78d210 con 0x55e82bb15170 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb9affd640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005940 tx=0x7fdb8c0bbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ca67f9a1b96ecb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb827fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8c0be0e0 con 0x55e82bb15170 2024-06-06T07:09:36.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fdb8c0ba050 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb84017a70 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdb8c082050 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 10 client.16530 fetch_fsmap learned FSMap version 43 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.214+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e82b859620 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb84017a70 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fdb8c0bcab0 con 0x55e82bb15170 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 10 client.16530 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 20 client.16530 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb34840 0x55e82bb36ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 4 client.16530 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e82ba8cf10 con 0x55e82bb34840 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.215+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb34840 0x55e82bb36ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.216+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb34840 0x55e82bb36ce0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e2837e64c3d89e7 server_cookie=6b926d7306587b2 in_seq=0 out_seq=0 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.216+0000 7fdb827fc640 10 client.16530 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdb827fc640 1 -- 172.21.15.73:0/901202254 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e82ba8cf10 con 0x55e82bb34840 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdb827fc640 10 client.16530 handle_command_reply: tid=0 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 client.16530 shutdown 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 msgr2=0x55e82bb2a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c005940 tx=0x7fdb8c0bbef0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 shutdown_connections 2024-06-06T07:09:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb34840 0x55e82bb36ce0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdb827fc640 0 client.16530 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb2a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 --2- 172.21.15.73:0/901202254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15170 0x55e82bb2a330 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 >> 172.21.15.73:0/901202254 conn(0x55e82bb11680 msgr2=0x55e82bb173b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 shutdown_connections 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 1 -- 172.21.15.73:0/901202254 wait complete. 2024-06-06T07:09:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.217+0000 7fdba45e4740 20 client.16530 trim_cache size 0 max 16384 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.232+0000 7fdba45e4740 1 Processor -- start 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.232+0000 7fdba45e4740 1 -- start start 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15170 0x55e82bb3c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb3cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1acca0 con 0x55e82bb20350 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1ae870 con 0x55e82bb15a90 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdba45e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1ad370 con 0x55e82bb15170 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42428/0 (socket says 172.21.15.73:42428) 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 -- 172.21.15.73:0/4215356297 learned_addr learned my addr 172.21.15.73:0/4215356297 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9affd640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15170 0x55e82bb3c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb3cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 -- 172.21.15.73:0/4215356297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15170 msgr2=0x55e82bb3c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15170 0x55e82bb3c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 -- 172.21.15.73:0/4215356297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 msgr2=0x55e82bb3cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb3cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 -- 172.21.15.73:0/4215356297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e82bb15880 con 0x55e82bb20350 2024-06-06T07:09:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb3cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.233+0000 7fdb9b7fe640 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdb940108a0 tx=0x7fdb94005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4cd89dbf11115 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdb90ff9640 1 -- 172.21.15.73:0/4215356297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb94010e20 con 0x55e82bb20350 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdb90ff9640 1 -- 172.21.15.73:0/4215356297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9401d8f0 con 0x55e82bb20350 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdb90ff9640 1 -- 172.21.15.73:0/4215356297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb940052f0 con 0x55e82bb20350 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 -- 172.21.15.73:0/4215356297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 msgr2=0x55e82bb30080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fdb940108a0 tx=0x7fdb94005bd0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 -- 172.21.15.73:0/4215356297 shutdown_connections 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb20350 0x55e82bb30080 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15a90 0x55e82bb3cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 --2- 172.21.15.73:0/4215356297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb15170 0x55e82bb3c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 -- 172.21.15.73:0/4215356297 >> 172.21.15.73:0/4215356297 conn(0x55e82bb11680 msgr2=0x55e82bb29ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 -- 172.21.15.73:0/4215356297 shutdown_connections 2024-06-06T07:09:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.234+0000 7fdba45e4740 1 -- 172.21.15.73:0/4215356297 wait complete. 2024-06-06T07:09:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 -- start start 2024-06-06T07:09:36.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 Processor -- start 2024-06-06T07:09:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb30700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb2e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1ae140 con 0x55e82bb15a90 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1ada70 con 0x55e82bb15170 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c1aaca0 con 0x55e82bb20350 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdba45e4740 10 client.0 _ll_register_callbacks cb 0x7fdb8c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42432/0 (socket says 172.21.15.73:42432) 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/24704505 learned_addr learned my addr 172.21.15.73:0/24704505 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9affd640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb30700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 msgr2=0x55e82bb2e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb2e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/24704505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 msgr2=0x55e82bb30700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb30700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9a7fc640 1 -- 172.21.15.73:0/24704505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e82bb2e710 con 0x55e82bb15a90 2024-06-06T07:09:36.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.235+0000 7fdb9affd640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb30700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb9a7fc640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdb84019040 tx=0x7fdb84004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19263740d352973d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84031020 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb9401d8f0 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fdb8400f750 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fdb8400f990 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 10 client.16536 fetch_fsmap learned FSMap version 43 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e82bb9f5c0 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb9401d8f0 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 43) ==== 8374+0+0 (secure 0 0 0) 0x7fdb84017420 con 0x55e82bb15a90 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 10 client.16536 fetch_fsmap finished waiting for FSMap version 43 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:36.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 20 client.16536 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb9fc80 0x55e82bba20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 4 client.16536 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.236+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e82ba86c20 con 0x55e82bb9fc80 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.237+0000 7fdb9affd640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb9fc80 0x55e82bba20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.237+0000 7fdb827fc640 10 client.16536 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:36.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.237+0000 7fdb9affd640 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb9fc80 0x55e82bba20a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfc1dfb1a3859157 server_cookie=759e3f127a0673bd in_seq=0 out_seq=0 2024-06-06T07:09:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.266+0000 7fdb827fc640 1 -- 172.21.15.73:0/24704505 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e82ba86c20 con 0x55e82bb9fc80 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.266+0000 7fdb827fc640 10 client.16536 handle_command_reply: tid=0 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.266+0000 7fdba45e4740 1 client.16536 shutdown 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 msgr2=0x55e82bb2dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fdb84019040 tx=0x7fdb84004660 comp rx=0 tx=0).stop 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 shutdown_connections 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e82bb9fc80 0x55e82bba20a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e82bb20350 0x55e82bb2e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdb827fc640 0 client.16536 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e82bb15a90 0x55e82bb2dae0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 --2- 172.21.15.73:0/24704505 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e82bb15170 0x55e82bb30700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 >> 172.21.15.73:0/24704505 conn(0x55e82bb11680 msgr2=0x55e82bb23c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 shutdown_connections 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 1 -- 172.21.15.73:0/24704505 wait complete. 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba45e4740 20 client.16536 trim_cache size 0 max 16384 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:09:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:09:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 msgr2=0x7fdb880a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 0x7fdb880a5260 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdb94005c40 tx=0x7fdb94009530 comp rx=0 tx=0).stop 2024-06-06T07:09:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 msgr2=0x7fdb9c1970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c004770 tx=0x7fdb8c004270 comp rx=0 tx=0).stop 2024-06-06T07:09:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 shutdown_connections 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb9c1068b0 0x7fdb9c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb880a2da0 0x7fdb880a5260 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb9c101260 0x7fdb9c197610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 --2- 172.21.15.73:0/1824791310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb9c1008a0 0x7fdb9c1970d0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.267+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 >> 172.21.15.73:0/1824791310 conn(0x7fdb9c0779b0 msgr2=0x7fdb9c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.268+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 shutdown_connections 2024-06-06T07:09:36.271 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:36.268+0000 7fdba17f9640 1 -- 172.21.15.73:0/1824791310 wait complete. 2024-06-06T07:09:36.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188140 2.48 MB/sec execute 283 sec latency 1470.361 ms 2024-06-06T07:09:36.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:36 smithi170 ceph-mon[36108]: pgmap v257: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 1013 KiB/s wr, 111 op/s 2024-06-06T07:09:37.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:36 smithi073 ceph-mon[32303]: pgmap v257: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 1013 KiB/s wr, 111 op/s 2024-06-06T07:09:37.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:36 smithi136 ceph-mon[35901]: pgmap v257: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 811 KiB/s rd, 1013 KiB/s wr, 111 op/s 2024-06-06T07:09:37.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188140 2.47 MB/sec execute 284 sec latency 2470.436 ms 2024-06-06T07:09:38.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188140 2.46 MB/sec execute 285 sec latency 3470.522 ms 2024-06-06T07:09:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:38 smithi073 ceph-mon[32303]: pgmap v258: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:09:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:38 smithi136 ceph-mon[35901]: pgmap v258: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:09:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:38 smithi170 ceph-mon[36108]: pgmap v258: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 862 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:09:39.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188140 2.45 MB/sec execute 286 sec latency 4470.562 ms 2024-06-06T07:09:40.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:39 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:09:40.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:39 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:09:40.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:39 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:09:40.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188346 2.45 MB/sec execute 287 sec latency 4757.802 ms 2024-06-06T07:09:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:40 smithi073 ceph-mon[32303]: pgmap v259: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:40 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:09:41.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:40 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:09:41.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:40 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:09:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:40 smithi136 ceph-mon[35901]: pgmap v259: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:40 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:09:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:40 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:09:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:40 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:09:41.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:40 smithi170 ceph-mon[36108]: pgmap v259: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:40 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:09:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:40 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:09:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:40 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:09:41.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 188662 2.44 MB/sec execute 288 sec latency 15.988 ms 2024-06-06T07:09:42.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 189131 2.44 MB/sec execute 289 sec latency 14.536 ms 2024-06-06T07:09:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:42 smithi073 ceph-mon[32303]: pgmap v260: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:42 smithi136 ceph-mon[35901]: pgmap v260: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:43.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:42 smithi170 ceph-mon[36108]: pgmap v260: 129 pgs: 129 active+clean; 119 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 689 KiB/s rd, 623 KiB/s wr, 80 op/s 2024-06-06T07:09:43.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 189658 2.44 MB/sec execute 290 sec latency 25.529 ms 2024-06-06T07:09:44.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 190291 2.44 MB/sec execute 291 sec latency 20.894 ms 2024-06-06T07:09:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:44 smithi073 ceph-mon[32303]: pgmap v261: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-06-06T07:09:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:44 smithi136 ceph-mon[35901]: pgmap v261: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-06-06T07:09:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:44 smithi170 ceph-mon[36108]: pgmap v261: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 126 op/s 2024-06-06T07:09:45.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 190783 2.44 MB/sec execute 292 sec latency 12.383 ms 2024-06-06T07:09:46.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191338 2.45 MB/sec execute 293 sec latency 15.278 ms 2024-06-06T07:09:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:46 smithi073 ceph-mon[32303]: pgmap v262: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 486 KiB/s wr, 65 op/s 2024-06-06T07:09:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:46 smithi136 ceph-mon[35901]: pgmap v262: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 486 KiB/s wr, 65 op/s 2024-06-06T07:09:47.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:46 smithi170 ceph-mon[36108]: pgmap v262: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 486 KiB/s wr, 65 op/s 2024-06-06T07:09:47.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191620 2.45 MB/sec execute 294 sec latency 494.973 ms 2024-06-06T07:09:48.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191620 2.44 MB/sec execute 295 sec latency 1495.075 ms 2024-06-06T07:09:49.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:48 smithi073 ceph-mon[32303]: pgmap v263: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:49.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:48 smithi136 ceph-mon[35901]: pgmap v263: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:48 smithi170 ceph-mon[36108]: pgmap v263: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 1.0 MiB/s wr, 125 op/s 2024-06-06T07:09:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.363+0000 7fbcf3e64640 1 -- 172.21.15.73:0/2502302053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1068e0 msgr2=0x7fbcec06a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.363+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/2502302053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1068e0 0x7fbcec06a180 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009900 tx=0x7fbce802a410 comp rx=0 tx=0).stop 2024-06-06T07:09:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 -- 172.21.15.73:0/2502302053 shutdown_connections 2024-06-06T07:09:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/2502302053 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec06ada0 0x7fbcec104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/2502302053 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec106d50 0x7fbcec06a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/2502302053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1068e0 0x7fbcec06a180 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 -- 172.21.15.73:0/2502302053 >> 172.21.15.73:0/2502302053 conn(0x7fbcec077680 msgr2=0x7fbcec077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 -- 172.21.15.73:0/2502302053 shutdown_connections 2024-06-06T07:09:49.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 -- 172.21.15.73:0/2502302053 wait complete. 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 Processor -- start 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.364+0000 7fbcf3e64640 1 -- start start 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec1068e0 0x7fbcec19e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec106d50 0x7fbcec19ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec118260 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1180e0 con 0x7fbcec1068e0 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1183e0 con 0x7fbcec106d50 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53964/0 (socket says 172.21.15.73:53964) 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 -- 172.21.15.73:0/3596389935 learned_addr learned my addr 172.21.15.73:0/3596389935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:49.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec1068e0 0x7fbcec19e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec106d50 0x7fbcec19ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 -- 172.21.15.73:0/3596389935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec106d50 msgr2=0x7fbcec19ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec106d50 0x7fbcec19ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 -- 172.21.15.73:0/3596389935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec1068e0 msgr2=0x7fbcec19e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec1068e0 0x7fbcec19e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 -- 172.21.15.73:0/3596389935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcec1a3a90 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf2401640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cad0 tx=0x7fbce400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbce2ffd640 1 -- 172.21.15.73:0/3596389935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce400ae20 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcec1a3d80 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbce2ffd640 1 -- 172.21.15.73:0/3596389935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce4005020 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbce2ffd640 1 -- 172.21.15.73:0/3596389935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce400f6b0 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.365+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcec1ab5c0 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.368+0000 7fbce2ffd640 1 -- 172.21.15.73:0/3596389935 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbce40051d0 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.368+0000 7fbcf6b73740 1 Processor -- start 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 -- start start 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbce2ffd640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 0x7fbcd80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe8c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe96e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fdf04f0 con 0x55fd0fe97350 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fd8e870 con 0x55fd0fe8ca90 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fdec4b0 con 0x55fd0fe8c170 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 0x7fbcd80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53972/0 (socket says 172.21.15.73:53972) 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 -- 172.21.15.73:0/546275317 learned_addr learned my addr 172.21.15.73:0/546275317 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe8c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbce2ffd640 1 -- 172.21.15.73:0/3596389935 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbce40cf950 con 0x7fbcec1a2fa0 2024-06-06T07:09:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe96e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 -- 172.21.15.73:0/546275317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 msgr2=0x55fd0fe8c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe8c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 -- 172.21.15.73:0/546275317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 msgr2=0x55fd0fe96e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe96e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.369+0000 7fbcf2401640 1 -- 172.21.15.73:0/546275317 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0fdaa9b0 con 0x55fd0fe97350 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 0x7fbcd80a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005e10 tx=0x7fbcdc0127c0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcf2401640 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbce40a41a0 tx=0x7fbce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebf090244d65a6b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcd7fff640 1 -- 172.21.15.73:0/546275317 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce401d070 con 0x55fd0fe97350 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcd7fff640 1 -- 172.21.15.73:0/546275317 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce40b5e50 con 0x55fd0fe97350 2024-06-06T07:09:49.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcd7fff640 1 -- 172.21.15.73:0/546275317 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce40ccf10 con 0x55fd0fe97350 2024-06-06T07:09:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcf6b73740 1 -- 172.21.15.73:0/546275317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 msgr2=0x55fd0fe99740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.370+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbce40a41a0 tx=0x7fbce4005830 comp rx=0 tx=0).stop 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 -- 172.21.15.73:0/546275317 shutdown_connections 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe99740 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe96e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/546275317 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe8c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 -- 172.21.15.73:0/546275317 >> 172.21.15.73:0/546275317 conn(0x55fd0fe88680 msgr2=0x55fd0fe88a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 -- 172.21.15.73:0/546275317 shutdown_connections 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 -- 172.21.15.73:0/546275317 wait complete. 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 -- start start 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 Processor -- start 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.371+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe94940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 0x55fd0fe94e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fe99f80 con 0x55fd0fe8c170 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fd79ae0 con 0x55fd0fe97350 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55fd0fe9a100 con 0x55fd0fe8ca90 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 10 client.0 _ll_register_callbacks cb 0x55fd0fdb29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53978/0 (socket says 172.21.15.73:53978) 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 -- 172.21.15.73:0/2905385806 learned_addr learned my addr 172.21.15.73:0/2905385806 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe94940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf2401640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 0x55fd0fe94e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 -- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8ca90 msgr2=0x55fd0fe94940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe94940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 -- 172.21.15.73:0/2905385806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 msgr2=0x55fd0fe94e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 0x55fd0fe94e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 -- 172.21.15.73:0/2905385806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0fb04210 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf2401640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 0x55fd0fe94e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbce802a930 tx=0x7fbce8031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=722d837a4b716e46 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8035040 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbce8004240 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce40b5e50 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbce8004440 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 10 client.16548 fetch_fsmap learned FSMap version 44 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.372+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fd0fbd0620 con 0x55fd0fe8c170 2024-06-06T07:09:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce40b5e50 con 0x55fd0fe8c170 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fbce80456e0 con 0x55fd0fe8c170 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 10 client.16548 fetch_fsmap finished waiting for FSMap version 44 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 20 client.16548 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0fea7650 0x55fd0fea9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 4 client.16548 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd0fe03f10 con 0x55fd0fea7650 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.373+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0fea7650 0x55fd0fea9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.374+0000 7fbcd57fa640 10 client.16548 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.374+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0fea7650 0x55fd0fea9aa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=451e95a8a16dc9b4 server_cookie=d6680b28b7cca145 in_seq=0 out_seq=0 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.374+0000 7fbcd57fa640 1 -- 172.21.15.73:0/2905385806 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fd0fe03f10 con 0x55fd0fea7650 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.374+0000 7fbcd57fa640 10 client.16548 handle_command_reply: tid=0 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.374+0000 7fbcf6b73740 1 client.16548 shutdown 2024-06-06T07:09:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 msgr2=0x55fd0fe94400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fbce802a930 tx=0x7fbce8031810 comp rx=0 tx=0).stop 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 shutdown_connections 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0fea7650 0x55fd0fea9aa0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe97350 0x55fd0fe94e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe94940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/2905385806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8c170 0x55fd0fe94400 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcd57fa640 0 client.16548 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 >> 172.21.15.73:0/2905385806 conn(0x55fd0fe88680 msgr2=0x55fd0fe8e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 shutdown_connections 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 1 -- 172.21.15.73:0/2905385806 wait complete. 2024-06-06T07:09:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.375+0000 7fbcf6b73740 20 client.16548 trim_cache size 0 max 16384 2024-06-06T07:09:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 Processor -- start 2024-06-06T07:09:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 -- start start 2024-06-06T07:09:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8c170 0x55fd0fe96340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 0x55fd0feaf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b5540 con 0x55fd0fe8ca90 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b7140 con 0x55fd0fe8c170 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b5c10 con 0x55fd0fe97350 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf13ff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf13ff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53992/0 (socket says 172.21.15.73:53992) 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf13ff640 1 -- 172.21.15.73:0/1328667504 learned_addr learned my addr 172.21.15.73:0/1328667504 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.390+0000 7fbcf2401640 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 0x55fd0feaf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 -- 172.21.15.73:0/1328667504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 msgr2=0x55fd0feaf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 0x55fd0feaf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 -- 172.21.15.73:0/1328667504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8c170 msgr2=0x55fd0fe96340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8c170 0x55fd0fe96340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 -- 172.21.15.73:0/1328667504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd0fe8d540 con 0x55fd0fe8ca90 2024-06-06T07:09:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf2401640 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 0x55fd0feaf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbcec100640 tx=0x7fbcdc005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279828464e98a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcd77fe640 1 -- 172.21.15.73:0/1328667504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc02d070 con 0x55fd0fe8ca90 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcd77fe640 1 -- 172.21.15.73:0/1328667504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcdc020d80 con 0x55fd0fe8ca90 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcd77fe640 1 -- 172.21.15.73:0/1328667504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcdc028ba0 con 0x55fd0fe8ca90 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 -- 172.21.15.73:0/1328667504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 msgr2=0x55fd0feaf370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fbcec100640 tx=0x7fbcdc005da0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 -- 172.21.15.73:0/1328667504 shutdown_connections 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe97350 0x55fd0feaf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe8ca90 0x55fd0feaf370 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/1328667504 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8c170 0x55fd0fe96340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.391+0000 7fbcf6b73740 1 -- 172.21.15.73:0/1328667504 >> 172.21.15.73:0/1328667504 conn(0x55fd0fe88680 msgr2=0x55fd0fe95c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- 172.21.15.73:0/1328667504 shutdown_connections 2024-06-06T07:09:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- 172.21.15.73:0/1328667504 wait complete. 2024-06-06T07:09:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- start start 2024-06-06T07:09:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 Processor -- start 2024-06-06T07:09:49.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe9f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe9fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b6a10 con 0x55fd0fe97350 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b6310 con 0x55fd0fe8ca90 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbcec1b3540 con 0x55fd0fe8c170 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf6b73740 10 client.0 _ll_register_callbacks cb 0x7fbce80314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf2401640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53994/0 (socket says 172.21.15.73:53994) 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf2401640 1 -- 172.21.15.73:0/248916488 learned_addr learned my addr 172.21.15.73:0/248916488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.392+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe9f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe9fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 -- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 msgr2=0x55fd0fe9f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe9f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 -- 172.21.15.73:0/248916488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 msgr2=0x55fd0fe9fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe9fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 -- 172.21.15.73:0/248916488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd0fea31e0 con 0x55fd0fe97350 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf13ff640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe9fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf2401640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbce40ccd40 tx=0x7fbce409fe80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d17f4b33324a3d1f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce401d040 con 0x55fd0fe97350 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7fbce4086230 con 0x55fd0fe97350 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcdc020d80 con 0x55fd0fe97350 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) ==== 24+0+0 (secure 0 0 0) 0x7fbce4086430 con 0x55fd0fe97350 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf6b73740 10 client.16554 fetch_fsmap learned FSMap version 44 2024-06-06T07:09:49.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) -- 0x55fd0ff12b30 con 0x55fd0fe97350 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcdc020d80 con 0x55fd0fe97350 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 44) ==== 8378+0+0 (secure 0 0 0) 0x7fbce4096140 con 0x55fd0fe97350 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf6b73740 10 client.16554 fetch_fsmap finished waiting for FSMap version 44 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.393+0000 7fbcf6b73740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf6b73740 20 client.16554 populate_metadata read hostname 'smithi073' 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0ff12f20 0x55fd0ff15310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf6b73740 4 client.16554 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fd0fdfdc20 con 0x55fd0ff12f20 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0ff12f20 0x55fd0ff15310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcf1c00640 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0ff12f20 0x55fd0ff15310 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f753db0f677c6dc server_cookie=edc1c836e37d968 in_seq=0 out_seq=0 2024-06-06T07:09:49.400 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.394+0000 7fbcd57fa640 10 client.16554 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcd57fa640 1 -- 172.21.15.73:0/248916488 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fd0fdfdc20 con 0x55fd0ff12f20 2024-06-06T07:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcd57fa640 10 client.16554 handle_command_reply: tid=0 2024-06-06T07:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 client.16554 shutdown 2024-06-06T07:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 msgr2=0x55fd0fe9c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbce40ccd40 tx=0x7fbce409fe80 comp rx=0 tx=0).stop 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 shutdown_connections 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55fd0ff12f20 0x55fd0ff15310 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fd0fe97350 0x55fd0fe9c320 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55fd0fe8ca90 0x55fd0fe9fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcd57fa640 0 client.16554 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 --2- 172.21.15.73:0/248916488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55fd0fe8c170 0x55fd0fe9f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 >> 172.21.15.73:0/248916488 conn(0x55fd0fe88680 msgr2=0x55fd0fea2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 shutdown_connections 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 1 -- 172.21.15.73:0/248916488 wait complete. 2024-06-06T07:09:49.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf6b73740 20 client.16554 trim_cache size 0 max 16384 2024-06-06T07:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:09:49.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 msgr2=0x7fbcd80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 0x7fbcd80a5260 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc005e10 tx=0x7fbcdc0127c0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 msgr2=0x7fbcec1a3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fbce400cad0 tx=0x7fbce400cf90 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 shutdown_connections 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbcec1a2fa0 0x7fbcec1a3450 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcd80a2da0 0x7fbcd80a5260 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbcec106d50 0x7fbcec19ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 --2- 172.21.15.73:0/3596389935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbcec1068e0 0x7fbcec19e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:09:49.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 >> 172.21.15.73:0/3596389935 conn(0x7fbcec077680 msgr2=0x7fbcec0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 shutdown_connections 2024-06-06T07:09:49.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:09:49.424+0000 7fbcf3e64640 1 -- 172.21.15.73:0/3596389935 wait complete. 2024-06-06T07:09:49.444 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191620 2.43 MB/sec execute 296 sec latency 2495.115 ms 2024-06-06T07:09:50.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:49 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:50.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:49 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:50.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:49 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:50.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191687 2.43 MB/sec execute 297 sec latency 2784.881 ms 2024-06-06T07:09:51.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:50 smithi073 ceph-mon[32303]: pgmap v264: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:51.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:50 smithi136 ceph-mon[35901]: pgmap v264: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:50 smithi170 ceph-mon[36108]: pgmap v264: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:51.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191687 2.42 MB/sec execute 298 sec latency 1458.542 ms 2024-06-06T07:09:52.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191687 2.41 MB/sec execute 299 sec latency 2458.601 ms 2024-06-06T07:09:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:52 smithi073 ceph-mon[32303]: pgmap v265: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:52 smithi136 ceph-mon[35901]: pgmap v265: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:52 smithi170 ceph-mon[36108]: pgmap v265: 129 pgs: 129 active+clean; 123 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 793 KiB/s rd, 992 KiB/s wr, 105 op/s 2024-06-06T07:09:53.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191687 2.40 MB/sec execute 300 sec latency 3458.653 ms 2024-06-06T07:09:54.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191687 2.39 MB/sec execute 301 sec latency 4458.749 ms 2024-06-06T07:09:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:54 smithi073 ceph-mon[32303]: pgmap v266: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:09:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:54 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:54 smithi136 ceph-mon[35901]: pgmap v266: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:09:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:54 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:54 smithi170 ceph-mon[36108]: pgmap v266: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 841 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:09:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:54 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:09:55.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 191887 2.39 MB/sec execute 302 sec latency 4753.249 ms 2024-06-06T07:09:56.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 192160 2.38 MB/sec execute 303 sec latency 14.748 ms 2024-06-06T07:09:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:56 smithi073 ceph-mon[32303]: pgmap v267: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 622 KiB/s wr, 79 op/s 2024-06-06T07:09:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:56 smithi136 ceph-mon[35901]: pgmap v267: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 622 KiB/s wr, 79 op/s 2024-06-06T07:09:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:56 smithi170 ceph-mon[36108]: pgmap v267: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 622 KiB/s wr, 79 op/s 2024-06-06T07:09:57.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 192606 2.38 MB/sec execute 304 sec latency 16.796 ms 2024-06-06T07:09:58.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193108 2.38 MB/sec execute 305 sec latency 25.661 ms 2024-06-06T07:09:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:09:58 smithi073 ceph-mon[32303]: pgmap v268: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:09:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:09:58 smithi136 ceph-mon[35901]: pgmap v268: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:09:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:09:58 smithi170 ceph-mon[36108]: pgmap v268: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 843 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:09:59.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193239 2.38 MB/sec execute 306 sec latency 762.531 ms 2024-06-06T07:10:00.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:09:59.999+0000 7f12df37d700 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:10:00.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:09:59.999+0000 7f12df37d700 -1 log_channel(cluster) log [ERR] : [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T07:10:00.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:09:59.999+0000 7f12df37d700 -1 log_channel(cluster) log [ERR] : mds.c(mds.1): Metadata damage detected 2024-06-06T07:10:00.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193239 2.37 MB/sec execute 307 sec latency 1762.578 ms 2024-06-06T07:10:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-mon[32303]: pgmap v269: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-mon[32303]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:10:01.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-mon[32303]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T07:10:01.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:00 smithi073 ceph-mon[32303]: mds.c(mds.1): Metadata damage detected 2024-06-06T07:10:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:00 smithi136 ceph-mon[35901]: pgmap v269: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:00 smithi136 ceph-mon[35901]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:10:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:00 smithi136 ceph-mon[35901]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T07:10:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:00 smithi136 ceph-mon[35901]: mds.c(mds.1): Metadata damage detected 2024-06-06T07:10:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:00 smithi170 ceph-mon[36108]: pgmap v269: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:00 smithi170 ceph-mon[36108]: Health detail: HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:10:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:00 smithi170 ceph-mon[36108]: [ERR] MDS_DAMAGE: 1 MDSs report damaged metadata 2024-06-06T07:10:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:00 smithi170 ceph-mon[36108]: mds.c(mds.1): Metadata damage detected 2024-06-06T07:10:01.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193239 2.36 MB/sec execute 308 sec latency 2762.614 ms 2024-06-06T07:10:02.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193239 2.35 MB/sec execute 309 sec latency 3762.647 ms 2024-06-06T07:10:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 -- 172.21.15.73:0/2804689874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0274100900 msgr2=0x7f0274100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 --2- 172.21.15.73:0/2804689874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0274100900 0x7f0274100d80 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f0268009930 tx=0x7f026802a420 comp rx=0 tx=0).stop 2024-06-06T07:10:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 -- 172.21.15.73:0/2804689874 shutdown_connections 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 --2- 172.21.15.73:0/2804689874 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f02741012c0 0x7f027410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 --2- 172.21.15.73:0/2804689874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0274100900 0x7f0274100d80 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 --2- 172.21.15.73:0/2804689874 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 0x7f0274106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 -- 172.21.15.73:0/2804689874 >> 172.21.15.73:0/2804689874 conn(0x7f0274077ab0 msgr2=0x7f0274077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 -- 172.21.15.73:0/2804689874 shutdown_connections 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.524+0000 7f02794ca640 1 -- 172.21.15.73:0/2804689874 wait complete. 2024-06-06T07:10:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 Processor -- start 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 -- start start 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0274100900 0x7f0274198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 0x7f02741a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0274110b80 con 0x7f02741012c0 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0274110a00 con 0x7f0274100900 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.525+0000 7f02794ca640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0274110d00 con 0x7f02741068c0 2024-06-06T07:10:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53812/0 (socket says 172.21.15.73:53812) 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 -- 172.21.15.73:0/4034039148 learned_addr learned my addr 172.21.15.73:0/4034039148 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02737fe640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 0x7f02741a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f0272ffd640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0274100900 0x7f0274198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 -- 172.21.15.73:0/4034039148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 msgr2=0x7f02741a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 0x7f02741a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 -- 172.21.15.73:0/4034039148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0274100900 msgr2=0x7f0274198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0274100900 0x7f0274198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 -- 172.21.15.73:0/4034039148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02741a44e0 con 0x7f02741012c0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02727fc640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0268005940 tx=0x7f02680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f025bfff640 1 -- 172.21.15.73:0/4034039148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026803f030 con 0x7f02741012c0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02741a46b0 con 0x7f02741012c0 2024-06-06T07:10:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f025bfff640 1 -- 172.21.15.73:0/4034039148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02680318a0 con 0x7f02741012c0 2024-06-06T07:10:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02741a4b20 con 0x7f02741012c0 2024-06-06T07:10:02.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.526+0000 7f025bfff640 1 -- 172.21.15.73:0/4034039148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0268049060 con 0x7f02741012c0 2024-06-06T07:10:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f025bfff640 1 -- 172.21.15.73:0/4034039148 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0268042030 con 0x7f02741012c0 2024-06-06T07:10:02.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 Processor -- start 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 -- start start 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d111740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55787d0684f0 con 0x55787d104170 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55787d006870 con 0x55787d10f350 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.529+0000 7f027c1d7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55787d0644b0 con 0x55787d104a90 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f025bfff640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 0x7f025c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53822/0 (socket says 172.21.15.73:53822) 2024-06-06T07:10:02.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 -- 172.21.15.73:0/3207301533 learned_addr learned my addr 172.21.15.73:0/3207301533 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 0x7f025c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f025bfff640 1 -- 172.21.15.73:0/4034039148 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f02680effa0 con 0x7f02741012c0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f0272ffd640 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02727fc640 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d111740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 -- 172.21.15.73:0/3207301533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 msgr2=0x55787d10ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 -- 172.21.15.73:0/3207301533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 msgr2=0x55787d111740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d111740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 -- 172.21.15.73:0/3207301533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787d0229b0 con 0x55787d104170 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f026c014f00 tx=0x7f026c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d980e00bd329de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02737fe640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 0x7f025c0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f026c005c40 tx=0x7f026c016000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02597fa640 1 -- 172.21.15.73:0/3207301533 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c024070 con 0x55787d104170 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02597fa640 1 -- 172.21.15.73:0/3207301533 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c007cb0 con 0x55787d104170 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.530+0000 7f02597fa640 1 -- 172.21.15.73:0/3207301533 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c0057e0 con 0x55787d104170 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- 172.21.15.73:0/3207301533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 msgr2=0x55787d104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f026c014f00 tx=0x7f026c0117b0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- 172.21.15.73:0/3207301533 shutdown_connections 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d111740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 --2- 172.21.15.73:0/3207301533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d104550 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- 172.21.15.73:0/3207301533 >> 172.21.15.73:0/3207301533 conn(0x55787d100680 msgr2=0x55787d100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- 172.21.15.73:0/3207301533 shutdown_connections 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- 172.21.15.73:0/3207301533 wait complete. 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.531+0000 7f027c1d7740 1 -- start start 2024-06-06T07:10:02.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 Processor -- start 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104a90 0x55787d10c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d10f350 0x55787d10ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55787d111f20 con 0x55787d104170 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55787cff1ae0 con 0x55787d104a90 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55787d1120a0 con 0x55787d10f350 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f027c1d7740 10 client.0 _ll_register_callbacks cb 0x55787d02a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53824/0 (socket says 172.21.15.73:53824) 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 -- 172.21.15.73:0/4192474405 learned_addr learned my addr 172.21.15.73:0/4192474405 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f02727fc640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104a90 0x55787d10c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f02737fe640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d10f350 0x55787d10ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 -- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d10f350 msgr2=0x55787d10ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d10f350 0x55787d10ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 -- 172.21.15.73:0/4192474405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104a90 msgr2=0x55787d10c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104a90 0x55787d10c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.532+0000 7f0272ffd640 1 -- 172.21.15.73:0/4192474405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787cd7c210 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0272ffd640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f02640126e0 tx=0x7f0264012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3865e2c8f0812640 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264004030 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f02640041d0 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026c007cb0 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f02640043d0 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 10 client.16566 fetch_fsmap learned FSMap version 45 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55787ce48620 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c007cb0 con 0x55787d104170 2024-06-06T07:10:02.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7f02640037a0 con 0x55787d104170 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 10 client.16566 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 10 client.16566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 20 client.16566 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d11f630 0x55787d121a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 4 client.16566 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.533+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55787d07bf10 con 0x55787d11f630 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.534+0000 7f02727fc640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d11f630 0x55787d121a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.534+0000 7f02727fc640 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d11f630 0x55787d121a80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4efb813f8813f29 server_cookie=71578bbb8fd11334 in_seq=0 out_seq=0 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.534+0000 7f0256ff5640 10 client.16566 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f0256ff5640 1 -- 172.21.15.73:0/4192474405 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55787d07bf10 con 0x55787d11f630 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f0256ff5640 10 client.16566 handle_command_reply: tid=0 2024-06-06T07:10:02.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 client.16566 shutdown 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 msgr2=0x55787d10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f02640126e0 tx=0x7f0264012ae0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 shutdown_connections 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d11f630 0x55787d121a80 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d10f350 0x55787d10ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104a90 0x55787d10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 --2- 172.21.15.73:0/4192474405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d10c3e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f0256ff5640 0 client.16566 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 >> 172.21.15.73:0/4192474405 conn(0x55787d100680 msgr2=0x55787d1063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 shutdown_connections 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 1 -- 172.21.15.73:0/4192474405 wait complete. 2024-06-06T07:10:02.543 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.535+0000 7f027c1d7740 20 client.16566 trim_cache size 0 max 16384 2024-06-06T07:10:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.550+0000 7f027c1d7740 1 Processor -- start 2024-06-06T07:10:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 -- start start 2024-06-06T07:10:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104170 0x55787d0fff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02741aed90 con 0x55787d10f350 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02741b09c0 con 0x55787d104170 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f027c1d7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02741af460 con 0x55787d104a90 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53828/0 (socket says 172.21.15.73:53828) 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 -- 172.21.15.73:0/682125119 learned_addr learned my addr 172.21.15.73:0/682125119 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02727fc640 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f0272ffd640 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104170 0x55787d0fff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 -- 172.21.15.73:0/682125119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 msgr2=0x55787d10e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 -- 172.21.15.73:0/682125119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104170 msgr2=0x55787d0fff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104170 0x55787d0fff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 -- 172.21.15.73:0/682125119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787d111ab0 con 0x55787d10f350 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f02737fe640 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f026c014f00 tx=0x7f026c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df85fefdae3b2531 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.551+0000 7f0258ff9640 1 -- 172.21.15.73:0/682125119 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c036070 con 0x55787d10f350 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f0258ff9640 1 -- 172.21.15.73:0/682125119 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c003e20 con 0x55787d10f350 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f0258ff9640 1 -- 172.21.15.73:0/682125119 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c024070 con 0x55787d10f350 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- 172.21.15.73:0/682125119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 msgr2=0x55787d127390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f026c014f00 tx=0x7f026c005040 comp rx=0 tx=0).stop 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- 172.21.15.73:0/682125119 shutdown_connections 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d10f350 0x55787d127390 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 --2- 172.21.15.73:0/682125119 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d104170 0x55787d0fff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- 172.21.15.73:0/682125119 >> 172.21.15.73:0/682125119 conn(0x55787d100680 msgr2=0x55787d10dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- 172.21.15.73:0/682125119 shutdown_connections 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- 172.21.15.73:0/682125119 wait complete. 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.552+0000 7f027c1d7740 1 -- start start 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 Processor -- start 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d113d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d189360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02741b0290 con 0x55787d104170 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f02741afb60 con 0x55787d10f350 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f02741acd90 con 0x55787d104a90 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f027c1d7740 10 client.0 _ll_register_callbacks cb 0x7f0264009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53834/0 (socket says 172.21.15.73:53834) 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 -- 172.21.15.73:0/1639103753 learned_addr learned my addr 172.21.15.73:0/1639103753 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f02737fe640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d189360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 -- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 msgr2=0x55787d113d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d113d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 -- 172.21.15.73:0/1639103753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 msgr2=0x55787d189360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d189360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 -- 172.21.15.73:0/1639103753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55787d1898d0 con 0x55787d104170 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f02737fe640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d189360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0272ffd640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0264009de0 tx=0x7f0264003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ce8091ed7f3da5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.553+0000 7f0256ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026402d070 con 0x55787d104170 2024-06-06T07:10:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9554+0+0 (secure 0 0 0) 0x7f0264004040 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026c003e20 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f0264005c50 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 10 client.16572 fetch_fsmap learned FSMap version 45 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55787d18abe0 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c003e20 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7f026401c460 con 0x55787d104170 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 10 client.16572 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 10 client.16572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 20 client.16572 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d18b3e0 0x55787d18d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 4 client.16572 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:10:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.554+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55787d075c20 con 0x55787d18b3e0 2024-06-06T07:10:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.555+0000 7f02727fc640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d18b3e0 0x55787d18d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.555+0000 7f02727fc640 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d18b3e0 0x55787d18d840 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48191505ad4f5f server_cookie=1f8f69138239fdcf in_seq=0 out_seq=0 2024-06-06T07:10:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.555+0000 7f0256ff5640 10 client.16572 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f0256ff5640 1 -- 172.21.15.73:0/1639103753 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55787d075c20 con 0x55787d18b3e0 2024-06-06T07:10:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f0256ff5640 10 client.16572 handle_command_reply: tid=0 2024-06-06T07:10:02.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 client.16572 shutdown 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 msgr2=0x55787d113810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f0264009de0 tx=0x7f0264003820 comp rx=0 tx=0).stop 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 shutdown_connections 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55787d18b3e0 0x55787d18d840 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55787d10f350 0x55787d189360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55787d104a90 0x55787d113d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f0256ff5640 0 client.16572 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 --2- 172.21.15.73:0/1639103753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55787d104170 0x55787d113810 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 >> 172.21.15.73:0/1639103753 conn(0x55787d100680 msgr2=0x55787d10cc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 shutdown_connections 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.583+0000 7f027c1d7740 1 -- 172.21.15.73:0/1639103753 wait complete. 2024-06-06T07:10:02.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f027c1d7740 20 client.16572 trim_cache size 0 max 16384 2024-06-06T07:10:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:10:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:10:02.586 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 msgr2=0x7f025c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 0x7f025c0a5260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f026c005c40 tx=0x7f026c016000 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 msgr2=0x7f0274199460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f0268005940 tx=0x7f02680042f0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 shutdown_connections 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f025c0a2da0 0x7f025c0a5260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f02741068c0 0x7f02741a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02741012c0 0x7f0274199460 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 --2- 172.21.15.73:0/4034039148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0274100900 0x7f0274198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:02.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 >> 172.21.15.73:0/4034039148 conn(0x7f0274077ab0 msgr2=0x7f0274103420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 shutdown_connections 2024-06-06T07:10:02.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:02.584+0000 7f02794ca640 1 -- 172.21.15.73:0/4034039148 wait complete. 2024-06-06T07:10:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:02 smithi073 ceph-mon[32303]: pgmap v270: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:02 smithi136 ceph-mon[35901]: pgmap v270: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:03.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:02 smithi170 ceph-mon[36108]: pgmap v270: 129 pgs: 129 active+clean; 122 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 478 KiB/s wr, 63 op/s 2024-06-06T07:10:03.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193394 2.35 MB/sec execute 310 sec latency 4467.032 ms 2024-06-06T07:10:04.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 193808 2.34 MB/sec execute 311 sec latency 325.697 ms 2024-06-06T07:10:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:04 smithi073 ceph-mon[32303]: pgmap v271: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 559 KiB/s wr, 94 op/s 2024-06-06T07:10:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:04 smithi073 ceph-mon[32303]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:10:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:04 smithi136 ceph-mon[35901]: pgmap v271: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 559 KiB/s wr, 94 op/s 2024-06-06T07:10:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:04 smithi136 ceph-mon[35901]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:10:05.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:04 smithi170 ceph-mon[36108]: pgmap v271: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 559 KiB/s wr, 94 op/s 2024-06-06T07:10:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:04 smithi170 ceph-mon[36108]: osdmap e109: 12 total, 12 up, 12 in 2024-06-06T07:10:05.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 194101 2.34 MB/sec execute 312 sec latency 614.847 ms 2024-06-06T07:10:06.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 194827 2.35 MB/sec execute 313 sec latency 15.638 ms 2024-06-06T07:10:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:06 smithi073 ceph-mon[32303]: pgmap v273: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 599 KiB/s wr, 90 op/s 2024-06-06T07:10:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:06 smithi136 ceph-mon[35901]: pgmap v273: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 599 KiB/s wr, 90 op/s 2024-06-06T07:10:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:06 smithi170 ceph-mon[36108]: pgmap v273: 129 pgs: 129 active+clean; 120 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 444 KiB/s rd, 599 KiB/s wr, 90 op/s 2024-06-06T07:10:07.445 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195165 2.35 MB/sec execute 314 sec latency 96.628 ms 2024-06-06T07:10:08.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195165 2.35 MB/sec execute 315 sec latency 1096.669 ms 2024-06-06T07:10:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:08 smithi073 ceph-mon[32303]: pgmap v274: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:08 smithi136 ceph-mon[35901]: pgmap v274: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:08 smithi170 ceph-mon[36108]: pgmap v274: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:09.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195165 2.34 MB/sec execute 316 sec latency 2096.727 ms 2024-06-06T07:10:10.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195232 2.33 MB/sec execute 317 sec latency 2389.014 ms 2024-06-06T07:10:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:10 smithi073 ceph-mon[32303]: pgmap v275: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:10 smithi136 ceph-mon[35901]: pgmap v275: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:11.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:10 smithi170 ceph-mon[36108]: pgmap v275: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:11.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195232 2.33 MB/sec execute 318 sec latency 1427.995 ms 2024-06-06T07:10:12.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195232 2.32 MB/sec execute 319 sec latency 2428.108 ms 2024-06-06T07:10:13.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195232 2.31 MB/sec execute 320 sec latency 3428.200 ms 2024-06-06T07:10:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:13 smithi136 ceph-mon[35901]: pgmap v276: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:13 smithi136 ceph-mon[35901]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:10:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:13 smithi170 ceph-mon[36108]: pgmap v276: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:13 smithi170 ceph-mon[36108]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:10:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:13 smithi073 ceph-mon[32303]: pgmap v276: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 798 KiB/s wr, 110 op/s 2024-06-06T07:10:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:13 smithi073 ceph-mon[32303]: osdmap e110: 12 total, 12 up, 12 in 2024-06-06T07:10:14.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195232 2.30 MB/sec execute 321 sec latency 4428.336 ms 2024-06-06T07:10:15.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195415 2.30 MB/sec execute 322 sec latency 4721.344 ms 2024-06-06T07:10:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 -- 172.21.15.73:0/2463055448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 msgr2=0x7f71f4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2463055448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4100d50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009930 tx=0x7f71e002a420 comp rx=0 tx=0).stop 2024-06-06T07:10:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 -- 172.21.15.73:0/2463055448 shutdown_connections 2024-06-06T07:10:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2463055448 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2463055448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4100d50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2463055448 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 0x7f71f4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 -- 172.21.15.73:0/2463055448 >> 172.21.15.73:0/2463055448 conn(0x7f71f4077a20 msgr2=0x7f71f4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 -- 172.21.15.73:0/2463055448 shutdown_connections 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 -- 172.21.15.73:0/2463055448 wait complete. 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.675+0000 7f71fab5a640 1 Processor -- start 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 -- start start 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f4197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 0x7f71f419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4110b50 con 0x7f71f41008d0 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41109d0 con 0x7f71f4101290 2024-06-06T07:10:15.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71fab5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71f4110cd0 con 0x7f71f4106890 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39412/0 (socket says 172.21.15.73:39412) 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 -- 172.21.15.73:0/2202568993 learned_addr learned my addr 172.21.15.73:0/2202568993 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f90f7640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 0x7f71f419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f3fff640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f4197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 -- 172.21.15.73:0/2202568993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 msgr2=0x7f71f419b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 0x7f71f419b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 -- 172.21.15.73:0/2202568993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 msgr2=0x7f71f4197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f4197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 -- 172.21.15.73:0/2202568993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f419c020 con 0x7f71f41008d0 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f3fff640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f88f6640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f1ffb640 1 -- 172.21.15.73:0/2202568993 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800cd80 con 0x7f71f41008d0 2024-06-06T07:10:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.676+0000 7f71f1ffb640 1 -- 172.21.15.73:0/2202568993 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e80054d0 con 0x7f71f41008d0 2024-06-06T07:10:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.677+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f419c310 con 0x7f71f41008d0 2024-06-06T07:10:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.677+0000 7f71f1ffb640 1 -- 172.21.15.73:0/2202568993 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8010490 con 0x7f71f41008d0 2024-06-06T07:10:15.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.677+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f41a3ba0 con 0x7f71f41008d0 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.679+0000 7f71f1ffb640 1 -- 172.21.15.73:0/2202568993 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f71e8005020 con 0x7f71f41008d0 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 Processor -- start 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 -- start start 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab69211e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f1ffb640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 0x7f71e40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab69214740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.687 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ab6916b4f0 con 0x55ab69207170 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab69109870 con 0x55ab69207a90 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71fd861740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ab691674b0 con 0x55ab69212350 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f3fff640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 0x7f71e40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39416/0 (socket says 172.21.15.73:39416) 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 -- 172.21.15.73:0/942484935 learned_addr learned my addr 172.21.15.73:0/942484935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab69214740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f88f6640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab69211e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f1ffb640 1 -- 172.21.15.73:0/2202568993 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f71e80cfb60 con 0x7f71f41008d0 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 -- 172.21.15.73:0/942484935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 msgr2=0x55ab69214740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab69214740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 -- 172.21.15.73:0/942484935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 msgr2=0x55ab69211e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab69211e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f90f7640 1 -- 172.21.15.73:0/942484935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab691259b0 con 0x55ab69207170 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.680+0000 7f71f88f6640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab69211e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.681+0000 7f71f90f7640 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55ab69204810 tx=0x7f71ec009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288a1d2864e357b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.681+0000 7f71f3fff640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 0x7f71e40a52e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000c00 tx=0x7f71e0009610 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.681+0000 7f71deffd640 1 -- 172.21.15.73:0/942484935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec004b20 con 0x55ab69207170 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.681+0000 7f71deffd640 1 -- 172.21.15.73:0/942484935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ec00fd70 con 0x55ab69207170 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.681+0000 7f71deffd640 1 -- 172.21.15.73:0/942484935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec005570 con 0x55ab69207170 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- 172.21.15.73:0/942484935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 msgr2=0x55ab69207550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x55ab69204810 tx=0x7f71ec009b30 comp rx=0 tx=0).stop 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- 172.21.15.73:0/942484935 shutdown_connections 2024-06-06T07:10:15.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab69214740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab69211e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- 172.21.15.73:0/942484935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab69207550 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- 172.21.15.73:0/942484935 >> 172.21.15.73:0/942484935 conn(0x55ab69203680 msgr2=0x55ab69203a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- 172.21.15.73:0/942484935 shutdown_connections 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- 172.21.15.73:0/942484935 wait complete. 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- start start 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 Processor -- start 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207170 0x55ab6920f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ab69214f20 con 0x55ab69207a90 2024-06-06T07:10:15.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ab690f4ae0 con 0x55ab69212350 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.682+0000 7f71fd861740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ab692150a0 con 0x55ab69207170 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71fd861740 10 client.0 _ll_register_callbacks cb 0x55ab6912d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39428/0 (socket says 172.21.15.73:39428) 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 -- 172.21.15.73:0/422223941 learned_addr learned my addr 172.21.15.73:0/422223941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f90f7640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f88f6640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207170 0x55ab6920f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 -- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207170 msgr2=0x55ab6920f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207170 0x55ab6920f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 -- 172.21.15.73:0/422223941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 msgr2=0x55ab6920fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 -- 172.21.15.73:0/422223941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab68e7f210 con 0x55ab69207a90 2024-06-06T07:10:15.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f90f7640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71f3fff640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f71e0033000 tx=0x7f71e0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d4818cfbbbae65 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e004b020 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f71e0005260 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec00fd70 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f71e0056050 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71fd861740 10 client.16584 fetch_fsmap learned FSMap version 45 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.683+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ab68f4b620 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71ec00fd70 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7f71e00035f0 con 0x55ab69207a90 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 10 client.16584 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T07:10:15.692 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 10 client.16584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 20 client.16584 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab692225e0 0x55ab692249f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 4 client.16584 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab6917ef10 con 0x55ab692225e0 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71f88f6640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab692225e0 0x55ab692249f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71d3fff640 10 client.16584 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.684+0000 7f71f88f6640 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab692225e0 0x55ab692249f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c35058529287f9 server_cookie=40aa30d4482fe235 in_seq=0 out_seq=0 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71d3fff640 1 -- 172.21.15.73:0/422223941 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab6917ef10 con 0x55ab692225e0 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71d3fff640 10 client.16584 handle_command_reply: tid=0 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 client.16584 shutdown 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 msgr2=0x55ab6920f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f71e0033000 tx=0x7f71e0009260 comp rx=0 tx=0).stop 2024-06-06T07:10:15.693 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 shutdown_connections 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab692225e0 0x55ab692249f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71d3fff640 0 client.16584 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207a90 0x55ab6920f840 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 --2- 172.21.15.73:0/422223941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207170 0x55ab6920f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 >> 172.21.15.73:0/422223941 conn(0x55ab69203680 msgr2=0x55ab69209400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 shutdown_connections 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.685+0000 7f71fd861740 1 -- 172.21.15.73:0/422223941 wait complete. 2024-06-06T07:10:15.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.686+0000 7f71fd861740 20 client.16584 trim_cache size 0 max 16384 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.702+0000 7f71fd861740 1 Processor -- start 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.702+0000 7f71fd861740 1 -- start start 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.702+0000 7f71fd861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.702+0000 7f71fd861740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab6922a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.702+0000 7f71fd861740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab6922a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71fd861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41acce0 con 0x55ab69207170 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71fd861740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41ae910 con 0x55ab69207a90 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71fd861740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41ad3b0 con 0x55ab69212350 2024-06-06T07:10:15.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39440/0 (socket says 172.21.15.73:39440) 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 -- 172.21.15.73:0/337385595 learned_addr learned my addr 172.21.15.73:0/337385595 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f90f7640 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab6922a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f3fff640 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab6922a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 -- 172.21.15.73:0/337385595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 msgr2=0x55ab6922a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab6922a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 -- 172.21.15.73:0/337385595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 msgr2=0x55ab6922a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab6922a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 -- 172.21.15.73:0/337385595 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6921e180 con 0x55ab69207170 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71f88f6640 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f71e809bc00 tx=0x7f71e8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc201cfc102108c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71de7fc640 1 -- 172.21.15.73:0/337385595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8005310 con 0x55ab69207170 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71de7fc640 1 -- 172.21.15.73:0/337385595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e8005680 con 0x55ab69207170 2024-06-06T07:10:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.703+0000 7f71de7fc640 1 -- 172.21.15.73:0/337385595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80a48e0 con 0x55ab69207170 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- 172.21.15.73:0/337385595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 msgr2=0x55ab6920fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f71e809bc00 tx=0x7f71e8009310 comp rx=0 tx=0).stop 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- 172.21.15.73:0/337385595 shutdown_connections 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69212350 0x55ab6922a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69207a90 0x55ab6922a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 --2- 172.21.15.73:0/337385595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6920fba0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- 172.21.15.73:0/337385595 >> 172.21.15.73:0/337385595 conn(0x55ab69203680 msgr2=0x55ab6920a670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- 172.21.15.73:0/337385595 shutdown_connections 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- 172.21.15.73:0/337385595 wait complete. 2024-06-06T07:10:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.704+0000 7f71fd861740 1 -- start start 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 Processor -- start 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207a90 0x55ab6921aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41ae1e0 con 0x55ab69207170 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41adab0 con 0x55ab69212350 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71f41aace0 con 0x55ab69207a90 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71fd861740 10 client.0 _ll_register_callbacks cb 0x7f71e003f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39450/0 (socket says 172.21.15.73:39450) 2024-06-06T07:10:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 -- 172.21.15.73:0/3560280442 learned_addr learned my addr 172.21.15.73:0/3560280442 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f90f7640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f3fff640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207a90 0x55ab6921aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 -- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207a90 msgr2=0x55ab6921aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207a90 0x55ab6921aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 -- 172.21.15.73:0/3560280442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 msgr2=0x55ab6920bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.705+0000 7f71f88f6640 1 -- 172.21.15.73:0/3560280442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6920c180 con 0x55ab69207170 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71f88f6640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f71e8098080 tx=0x7f71e8005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a8cb177abb22d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e80b8070 con 0x55ab69207170 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f71e80cd120 con 0x55ab69207170 2024-06-06T07:10:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71e8005680 con 0x55ab69207170 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f71e809bc70 con 0x55ab69207170 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 10 client.16590 fetch_fsmap learned FSMap version 45 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55ab6928cbc0 con 0x55ab69207170 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71e8005680 con 0x55ab69207170 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 45) ==== 8382+0+0 (secure 0 0 0) 0x7f71e809bed0 con 0x55ab69207170 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 10 client.16590 fetch_fsmap finished waiting for FSMap version 45 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 10 client.16590 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 20 client.16590 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab6928d390 0x55ab6928f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 4 client.16590 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ab69178c20 con 0x55ab6928d390 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.706+0000 7f71f3fff640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab6928d390 0x55ab6928f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.707+0000 7f71d3fff640 10 client.16590 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.707+0000 7f71f3fff640 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab6928d390 0x55ab6928f7b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9fcf089b237f874 server_cookie=c2e99ca4e2b2cd78 in_seq=0 out_seq=0 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71d3fff640 1 -- 172.21.15.73:0/3560280442 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ab69178c20 con 0x55ab6928d390 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71d3fff640 10 client.16590 handle_command_reply: tid=0 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 client.16590 shutdown 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 msgr2=0x55ab6921a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f71e8098080 tx=0x7f71e8005ca0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 shutdown_connections 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ab6928d390 0x55ab6928f7b0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ab69212350 0x55ab6920bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71d3fff640 0 client.16590 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ab69207a90 0x55ab6921aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 --2- 172.21.15.73:0/3560280442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ab69207170 0x55ab6921a4e0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.731+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 >> 172.21.15.73:0/3560280442 conn(0x55ab69203680 msgr2=0x55ab69201f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 shutdown_connections 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fd861740 1 -- 172.21.15.73:0/3560280442 wait complete. 2024-06-06T07:10:15.734 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fd861740 20 client.16590 trim_cache size 0 max 16384 2024-06-06T07:10:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:10:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:10:15.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:10:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 msgr2=0x7f71e40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 0x7f71e40a52e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000c00 tx=0x7f71e0009610 comp rx=0 tx=0).stop 2024-06-06T07:10:15.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 msgr2=0x7f71f4197030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f71e800e9f0 tx=0x7f71e800edf0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 shutdown_connections 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f71e40a2e20 0x7f71e40a52e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f71f4106890 0x7f71f419b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f71f4101290 0x7f71f4197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 --2- 172.21.15.73:0/2202568993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f41008d0 0x7f71f4197030 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 >> 172.21.15.73:0/2202568993 conn(0x7f71f4077a20 msgr2=0x7f71f4103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 shutdown_connections 2024-06-06T07:10:15.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:15.732+0000 7f71fab5a640 1 -- 172.21.15.73:0/2202568993 wait complete. 2024-06-06T07:10:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:15 smithi136 ceph-mon[35901]: pgmap v278: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 749 KiB/s wr, 91 op/s 2024-06-06T07:10:15.825 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:15 smithi073 ceph-mon[32303]: pgmap v278: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 749 KiB/s wr, 91 op/s 2024-06-06T07:10:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:15 smithi170 ceph-mon[36108]: pgmap v278: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 749 KiB/s wr, 91 op/s 2024-06-06T07:10:16.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 195697 2.29 MB/sec execute 323 sec latency 16.516 ms 2024-06-06T07:10:17.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196151 2.29 MB/sec execute 324 sec latency 15.842 ms 2024-06-06T07:10:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:17 smithi136 ceph-mon[35901]: pgmap v279: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 742 KiB/s wr, 90 op/s 2024-06-06T07:10:17.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:17 smithi170 ceph-mon[36108]: pgmap v279: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 742 KiB/s wr, 90 op/s 2024-06-06T07:10:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:17 smithi073 ceph-mon[32303]: pgmap v279: 129 pgs: 129 active+clean; 121 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 623 KiB/s rd, 742 KiB/s wr, 90 op/s 2024-06-06T07:10:18.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196655 2.29 MB/sec execute 325 sec latency 19.474 ms 2024-06-06T07:10:19.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196788 2.29 MB/sec execute 326 sec latency 746.485 ms 2024-06-06T07:10:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:19 smithi136 ceph-mon[35901]: pgmap v280: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:19 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:19 smithi170 ceph-mon[36108]: pgmap v280: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:19 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:19 smithi073 ceph-mon[32303]: pgmap v280: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:19 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:20.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196788 2.28 MB/sec execute 327 sec latency 1746.545 ms 2024-06-06T07:10:21.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196788 2.27 MB/sec execute 328 sec latency 2746.653 ms 2024-06-06T07:10:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:21 smithi136 ceph-mon[35901]: pgmap v281: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:21 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:21 smithi170 ceph-mon[36108]: pgmap v281: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:21 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:21 smithi073 ceph-mon[32303]: pgmap v281: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:21 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:10:22.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196788 2.27 MB/sec execute 329 sec latency 3746.700 ms 2024-06-06T07:10:23.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 196957 2.26 MB/sec execute 330 sec latency 4452.178 ms 2024-06-06T07:10:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:23 smithi136 ceph-mon[35901]: pgmap v282: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:23 smithi170 ceph-mon[36108]: pgmap v282: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:23 smithi073 ceph-mon[32303]: pgmap v282: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 270 KiB/s rd, 586 KiB/s wr, 76 op/s 2024-06-06T07:10:24.446 INFO:tasks.workunit.client.0.smithi073.stdout: 1 197497 2.26 MB/sec execute 331 sec latency 18.729 ms 2024-06-06T07:10:25.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198105 2.27 MB/sec execute 332 sec latency 18.170 ms 2024-06-06T07:10:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:25 smithi136 ceph-mon[35901]: pgmap v283: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 581 KiB/s wr, 92 op/s 2024-06-06T07:10:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:25 smithi170 ceph-mon[36108]: pgmap v283: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 581 KiB/s wr, 92 op/s 2024-06-06T07:10:26.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:25 smithi073 ceph-mon[32303]: pgmap v283: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 462 KiB/s rd, 581 KiB/s wr, 92 op/s 2024-06-06T07:10:26.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198605 2.28 MB/sec execute 333 sec latency 15.579 ms 2024-06-06T07:10:27.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198710 2.27 MB/sec execute 334 sec latency 800.950 ms 2024-06-06T07:10:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:27 smithi073 ceph-mon[32303]: pgmap v284: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 74 op/s 2024-06-06T07:10:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:27 smithi136 ceph-mon[35901]: pgmap v284: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 74 op/s 2024-06-06T07:10:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:27 smithi170 ceph-mon[36108]: pgmap v284: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 397 KiB/s rd, 514 KiB/s wr, 74 op/s 2024-06-06T07:10:28.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198710 2.26 MB/sec execute 335 sec latency 1801.085 ms 2024-06-06T07:10:28.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 -- 172.21.15.73:0/514664863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100ac0 msgr2=0x7f2984106650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 --2- 172.21.15.73:0/514664863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100ac0 0x7f2984106650 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2974009900 tx=0x7f297402a410 comp rx=0 tx=0).stop 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 -- 172.21.15.73:0/514664863 shutdown_connections 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 --2- 172.21.15.73:0/514664863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2984106d80 0x7f298410e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 --2- 172.21.15.73:0/514664863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984100ac0 0x7f2984106650 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 --2- 172.21.15.73:0/514664863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 0x7f29841004f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 -- 172.21.15.73:0/514664863 >> 172.21.15.73:0/514664863 conn(0x7f2984076e00 msgr2=0x7f29840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 -- 172.21.15.73:0/514664863 shutdown_connections 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.829+0000 7f298c4ee640 1 -- 172.21.15.73:0/514664863 wait complete. 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 Processor -- start 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 -- start start 2024-06-06T07:10:28.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 0x7f2984198190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f298419ca00 0x7f298419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111c90 con 0x7f2984106d80 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111b10 con 0x7f2984100110 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298c4ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2984111e10 con 0x7f298419ca00 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37254/0 (socket says 172.21.15.73:37254) 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 -- 172.21.15.73:0/2926508401 learned_addr learned my addr 172.21.15.73:0/2926508401 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298a28a640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 0x7f2984198190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f298aa8b640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f298419ca00 0x7f298419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 -- 172.21.15.73:0/2926508401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f298419ca00 msgr2=0x7f298419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f298419ca00 0x7f298419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 -- 172.21.15.73:0/2926508401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 msgr2=0x7f2984198190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 0x7f2984198190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.830+0000 7f2989a89640 1 -- 172.21.15.73:0/2926508401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298419d4f0 con 0x7f2984106d80 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f2989a89640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f297402a930 tx=0x7f2974004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f297b7fe640 1 -- 172.21.15.73:0/2926508401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297403f030 con 0x7f2984106d80 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f297b7fe640 1 -- 172.21.15.73:0/2926508401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2974003aa0 con 0x7f2984106d80 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298419d780 con 0x7f2984106d80 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f297b7fe640 1 -- 172.21.15.73:0/2926508401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974035070 con 0x7f2984106d80 2024-06-06T07:10:28.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.831+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a4f40 con 0x7f2984106d80 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.833+0000 7f297b7fe640 1 -- 172.21.15.73:0/2926508401 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2974042030 con 0x7f2984106d80 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 Processor -- start 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 -- start start 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38151470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c38160220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c380b54f0 con 0x562c38160760 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c38053870 con 0x562c38151090 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298f1fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562c380b14b0 con 0x562c381519b0 2024-06-06T07:10:28.840 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f297b7fe640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 0x7f29700a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298aa8b640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 0x7f29700a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37256/0 (socket says 172.21.15.73:37256) 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f2989a89640 1 -- 172.21.15.73:0/2201497217 learned_addr learned my addr 172.21.15.73:0/2201497217 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298a28a640 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c38160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.834+0000 7f298aa8b640 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f297b7fe640 1 -- 172.21.15.73:0/2926508401 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f29740efc80 con 0x7f2984106d80 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 -- 172.21.15.73:0/2201497217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 msgr2=0x562c38160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c38160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 -- 172.21.15.73:0/2201497217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 msgr2=0x562c38151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 -- 172.21.15.73:0/2201497217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3806f9b0 con 0x562c38160760 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f298aa8b640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 0x7f29700a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f297c009270 tx=0x7f297c009db0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2989a89640 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f29740b81a0 tx=0x7f297408ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9271c4a48c8566cf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2978bfe640 1 -- 172.21.15.73:0/2201497217 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974049070 con 0x562c38160760 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2978bfe640 1 -- 172.21.15.73:0/2201497217 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29740bc490 con 0x562c38160760 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.835+0000 7f2978bfe640 1 -- 172.21.15.73:0/2201497217 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974094cf0 con 0x562c38160760 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- 172.21.15.73:0/2201497217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 msgr2=0x562c38162b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f29740b81a0 tx=0x7f297408ee00 comp rx=0 tx=0).stop 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- 172.21.15.73:0/2201497217 shutdown_connections 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c38162b50 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c38160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 --2- 172.21.15.73:0/2201497217 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- 172.21.15.73:0/2201497217 >> 172.21.15.73:0/2201497217 conn(0x562c3814d680 msgr2=0x562c3814da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- 172.21.15.73:0/2201497217 shutdown_connections 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- 172.21.15.73:0/2201497217 wait complete. 2024-06-06T07:10:28.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 -- start start 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.836+0000 7f298f1fa740 1 Processor -- start 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c3815dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c380b54f0 con 0x562c38160760 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562c3803eae0 con 0x562c38151090 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562c38163150 con 0x562c381519b0 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 10 client.0 _ll_register_callbacks cb 0x562c380779b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37270/0 (socket says 172.21.15.73:37270) 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 -- 172.21.15.73:0/3178619968 learned_addr learned my addr 172.21.15.73:0/3178619968 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298a28a640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f2989a89640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c3815dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 -- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 msgr2=0x562c3815dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c3815dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 -- 172.21.15.73:0/3178619968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 msgr2=0x562c3815d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 -- 172.21.15.73:0/3178619968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c37dc9210 con 0x562c38160760 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298a28a640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f297c010d80 tx=0x7f297c009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8120616c6538bd36 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f296dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c010db0 con 0x562c38160760 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f297c004540 con 0x562c38160760 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.837+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29740bc490 con 0x562c38160760 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f297c004740 con 0x562c38160760 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 10 client.16602 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:28.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562c37e95620 con 0x562c38160760 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29740bc490 con 0x562c38160760 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7f297c01f050 con 0x562c38160760 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 10 client.16602 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 20 client.16602 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c38170720 0x562c38172b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 4 client.16602 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c380c8f10 con 0x562c38170720 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.838+0000 7f298a28a640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c38170720 0x562c38172b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.839+0000 7f296dffb640 10 client.16602 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.839+0000 7f298a28a640 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c38170720 0x562c38172b90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1528057e186167c6 server_cookie=4db9145daced11d9 in_seq=0 out_seq=0 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.839+0000 7f296dffb640 1 -- 172.21.15.73:0/3178619968 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c380c8f10 con 0x562c38170720 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.839+0000 7f296dffb640 10 client.16602 handle_command_reply: tid=0 2024-06-06T07:10:28.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.839+0000 7f298f1fa740 1 client.16602 shutdown 2024-06-06T07:10:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 msgr2=0x562c3815e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f297c010d80 tx=0x7f297c009500 comp rx=0 tx=0).stop 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 shutdown_connections 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c38170720 0x562c38172b90 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c38160760 0x562c3815e1d0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c381519b0 0x562c3815dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f296dffb640 0 client.16602 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3178619968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 >> 172.21.15.73:0/3178619968 conn(0x562c3814d680 msgr2=0x562c381534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 shutdown_connections 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 1 -- 172.21.15.73:0/3178619968 wait complete. 2024-06-06T07:10:28.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.840+0000 7f298f1fa740 20 client.16602 trim_cache size 0 max 16384 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 Processor -- start 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 -- start start 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c381789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29841acc50 con 0x562c381519b0 2024-06-06T07:10:28.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ae8e0 con 0x562c38151090 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f298f1fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ad320 con 0x562c38160760 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.855+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37280/0 (socket says 172.21.15.73:37280) 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 -- 172.21.15.73:0/3698238228 learned_addr learned my addr 172.21.15.73:0/3698238228 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f298a28a640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 -- 172.21.15.73:0/3698238228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 msgr2=0x562c381789a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c381789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c381789a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 -- 172.21.15.73:0/3698238228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 msgr2=0x562c3815de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 -- 172.21.15.73:0/3698238228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c3815bca0 con 0x562c381519b0 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f298a28a640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:28.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c381789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f2989a89640 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f29740b81f0 tx=0x7f2974000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26596de7d917e60d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f296ffff640 1 -- 172.21.15.73:0/3698238228 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974049040 con 0x562c381519b0 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f296ffff640 1 -- 172.21.15.73:0/3698238228 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29740bc490 con 0x562c381519b0 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.856+0000 7f296ffff640 1 -- 172.21.15.73:0/3698238228 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740c4a00 con 0x562c381519b0 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- 172.21.15.73:0/3698238228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 msgr2=0x562c38178460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f29740b81f0 tx=0x7f2974000bc0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- 172.21.15.73:0/3698238228 shutdown_connections 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c381789a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c38178460 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3698238228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c3815de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- 172.21.15.73:0/3698238228 >> 172.21.15.73:0/3698238228 conn(0x562c3814d680 msgr2=0x562c3815d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- 172.21.15.73:0/3698238228 shutdown_connections 2024-06-06T07:10:28.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- 172.21.15.73:0/3698238228 wait complete. 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- start start 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 Processor -- start 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38168b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c3815a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ae1b0 con 0x562c381519b0 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ada50 con 0x562c38151090 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f29841aac50 con 0x562c38160760 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298f1fa740 10 client.0 _ll_register_callbacks cb 0x7f297c0125a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f2989a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37290/0 (socket says 172.21.15.73:37290) 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f2989a89640 1 -- 172.21.15.73:0/3339579284 learned_addr learned my addr 172.21.15.73:0/3339579284 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298a28a640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38168b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.857+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c3815a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 -- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 msgr2=0x562c3815a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c3815a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 -- 172.21.15.73:0/3339579284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 msgr2=0x562c38168b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38168b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 -- 172.21.15.73:0/3339579284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c3815b190 con 0x562c381519b0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298a28a640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38168b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298aa8b640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c3815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f2989a89640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f297403d040 tx=0x7f29740b80b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274e786b044e3b8c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29740d8020 con 0x562c381519b0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7f29740b8400 con 0x562c381519b0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29740bc490 con 0x562c381519b0 2024-06-06T07:10:28.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f297408f070 con 0x562c381519b0 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 10 client.16608 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562c381db320 con 0x562c381519b0 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29740bc490 con 0x562c381519b0 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7f29740bbff0 con 0x562c381519b0 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 10 client.16608 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 20 client.16608 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c381db700 0x562c381ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 4 client.16608 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.858+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c380c2c20 con 0x562c381db700 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.859+0000 7f298a28a640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c381db700 0x562c381ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.859+0000 7f296dffb640 10 client.16608 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:28.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.859+0000 7f298a28a640 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c381db700 0x562c381ddaf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccbc32487c2504f4 server_cookie=969145c65cb524de in_seq=0 out_seq=0 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f296dffb640 1 -- 172.21.15.73:0/3339579284 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c380c2c20 con 0x562c381db700 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f296dffb640 10 client.16608 handle_command_reply: tid=0 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 client.16608 shutdown 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 msgr2=0x562c3815a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f297403d040 tx=0x7f29740b80b0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 shutdown_connections 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562c381db700 0x562c381ddaf0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562c38160760 0x562c3815a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f296dffb640 0 client.16608 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c381519b0 0x562c3815a420 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 --2- 172.21.15.73:0/3339579284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562c38151090 0x562c38168b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 >> 172.21.15.73:0/3339579284 conn(0x562c3814d680 msgr2=0x562c3815dfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 shutdown_connections 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 1 -- 172.21.15.73:0/3339579284 wait complete. 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.890+0000 7f298f1fa740 20 client.16608 trim_cache size 0 max 16384 2024-06-06T07:10:28.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:10:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:10:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:10:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 msgr2=0x7f29700a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 0x7f29700a5260 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f297c009270 tx=0x7f297c009db0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 msgr2=0x7f29841986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f297402a930 tx=0x7f2974004040 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 shutdown_connections 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f29700a2da0 0x7f29700a5260 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f298419ca00 0x7f298419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2984106d80 0x7f29841986d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 --2- 172.21.15.73:0/2926508401 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2984100110 0x7f2984198190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 >> 172.21.15.73:0/2926508401 conn(0x7f2984076e00 msgr2=0x7f29840fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 shutdown_connections 2024-06-06T07:10:28.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:28.891+0000 7f298c4ee640 1 -- 172.21.15.73:0/2926508401 wait complete. 2024-06-06T07:10:29.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198710 2.26 MB/sec execute 336 sec latency 2801.175 ms 2024-06-06T07:10:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:29 smithi170 ceph-mon[36108]: pgmap v285: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:30.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:29 smithi073 ceph-mon[32303]: pgmap v285: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:29 smithi136 ceph-mon[35901]: pgmap v285: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 845 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:30.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198777 2.25 MB/sec execute 337 sec latency 3092.304 ms 2024-06-06T07:10:31.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198777 2.24 MB/sec execute 338 sec latency 1449.253 ms 2024-06-06T07:10:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:31 smithi170 ceph-mon[36108]: pgmap v286: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 588 KiB/s wr, 72 op/s 2024-06-06T07:10:32.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:31 smithi073 ceph-mon[32303]: pgmap v286: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 588 KiB/s wr, 72 op/s 2024-06-06T07:10:32.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:31 smithi136 ceph-mon[35901]: pgmap v286: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 588 KiB/s wr, 72 op/s 2024-06-06T07:10:32.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198777 2.24 MB/sec execute 339 sec latency 2449.337 ms 2024-06-06T07:10:33.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:32 smithi073 ceph-mon[32303]: pgmap v287: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 589 KiB/s wr, 72 op/s 2024-06-06T07:10:33.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:32 smithi136 ceph-mon[35901]: pgmap v287: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 589 KiB/s wr, 72 op/s 2024-06-06T07:10:33.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:32 smithi170 ceph-mon[36108]: pgmap v287: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 659 KiB/s rd, 589 KiB/s wr, 72 op/s 2024-06-06T07:10:33.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198777 2.23 MB/sec execute 340 sec latency 3449.402 ms 2024-06-06T07:10:34.447 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198777 2.22 MB/sec execute 341 sec latency 4449.506 ms 2024-06-06T07:10:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:34 smithi073 ceph-mon[32303]: pgmap v288: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 635 KiB/s wr, 91 op/s 2024-06-06T07:10:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:34 smithi136 ceph-mon[35901]: pgmap v288: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 635 KiB/s wr, 91 op/s 2024-06-06T07:10:35.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:34 smithi170 ceph-mon[36108]: pgmap v288: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 635 KiB/s wr, 91 op/s 2024-06-06T07:10:35.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 198966 2.22 MB/sec execute 342 sec latency 4741.244 ms 2024-06-06T07:10:36.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 199249 2.22 MB/sec execute 343 sec latency 15.867 ms 2024-06-06T07:10:37.148 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:36 smithi170 ceph-mon[36108]: pgmap v289: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 576 KiB/s wr, 67 op/s 2024-06-06T07:10:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:36 smithi073 ceph-mon[32303]: pgmap v289: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 576 KiB/s wr, 67 op/s 2024-06-06T07:10:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:36 smithi136 ceph-mon[35901]: pgmap v289: 129 pgs: 129 active+clean; 122 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 576 KiB/s wr, 67 op/s 2024-06-06T07:10:37.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 199748 2.21 MB/sec execute 344 sec latency 12.440 ms 2024-06-06T07:10:38.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200261 2.22 MB/sec execute 345 sec latency 25.675 ms 2024-06-06T07:10:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:38 smithi073 ceph-mon[32303]: pgmap v290: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1004 KiB/s wr, 114 op/s 2024-06-06T07:10:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:38 smithi136 ceph-mon[35901]: pgmap v290: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1004 KiB/s wr, 114 op/s 2024-06-06T07:10:39.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:38 smithi170 ceph-mon[36108]: pgmap v290: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 1004 KiB/s wr, 114 op/s 2024-06-06T07:10:39.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200331 2.21 MB/sec execute 346 sec latency 867.776 ms 2024-06-06T07:10:40.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200331 2.20 MB/sec execute 347 sec latency 1867.844 ms 2024-06-06T07:10:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:40 smithi073 ceph-mon[32303]: pgmap v291: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:40 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:10:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:40 smithi136 ceph-mon[35901]: pgmap v291: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:40 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:10:41.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:40 smithi170 ceph-mon[36108]: pgmap v291: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:40 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:10:41.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200331 2.20 MB/sec execute 348 sec latency 2867.931 ms 2024-06-06T07:10:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 -- 172.21.15.73:0/2027717801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c1015b0 msgr2=0x7fb49c10d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/2027717801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c1015b0 0x7fb49c10d6c0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fb494009ca0 tx=0x7fb49402a8c0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 -- 172.21.15.73:0/2027717801 shutdown_connections 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/2027717801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c1015b0 0x7fb49c10d6c0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/2027717801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c100bf0 0x7fb49c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/2027717801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c105840 0x7fb49c105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 -- 172.21.15.73:0/2027717801 >> 172.21.15.73:0/2027717801 conn(0x7fb49c0766a0 msgr2=0x7fb49c076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 -- 172.21.15.73:0/2027717801 shutdown_connections 2024-06-06T07:10:41.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.985+0000 7fb4a0a38640 1 -- 172.21.15.73:0/2027717801 wait complete. 2024-06-06T07:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.986+0000 7fb4a0a38640 1 Processor -- start 2024-06-06T07:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.986+0000 7fb4a0a38640 1 -- start start 2024-06-06T07:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.986+0000 7fb4a0a38640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c100bf0 0x7fb49c196ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb4a0a38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c1015b0 0x7fb49c1973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb4a0a38640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb4a0a38640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c110b20 con 0x7fb49c105840 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb4a0a38640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1109a0 con 0x7fb49c100bf0 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb4a0a38640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c110ca0 con 0x7fb49c1015b0 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41282/0 (socket says 172.21.15.73:41282) 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 -- 172.21.15.73:0/3842183363 learned_addr learned my addr 172.21.15.73:0/3842183363 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb493fff640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c1015b0 0x7fb49c1973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c100bf0 0x7fb49c196ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 -- 172.21.15.73:0/3842183363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c1015b0 msgr2=0x7fb49c1973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c1015b0 0x7fb49c1973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 -- 172.21.15.73:0/3842183363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c100bf0 msgr2=0x7fb49c196ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:41.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c100bf0 0x7fb49c196ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 -- 172.21.15.73:0/3842183363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb49c19be40 con 0x7fb49c105840 2024-06-06T07:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.987+0000 7fb49ad9d640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb494009dc0 tx=0x7fb494032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.990+0000 7fb4937fe640 1 -- 172.21.15.73:0/3842183363 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494003cf0 con 0x7fb49c105840 2024-06-06T07:10:41.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.990+0000 7fb4937fe640 1 -- 172.21.15.73:0/3842183363 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4940044f0 con 0x7fb49c105840 2024-06-06T07:10:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.990+0000 7fb4937fe640 1 -- 172.21.15.73:0/3842183363 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb494030b80 con 0x7fb49c105840 2024-06-06T07:10:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.990+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb49c19c0d0 con 0x7fb49c105840 2024-06-06T07:10:41.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.990+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb49c1a39c0 con 0x7fb49c105840 2024-06-06T07:10:41.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.992+0000 7fb4937fe640 1 -- 172.21.15.73:0/3842183363 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb4940046a0 con 0x7fb49c105840 2024-06-06T07:10:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.992+0000 7fb4a3821740 1 Processor -- start 2024-06-06T07:10:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 -- start start 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d20844e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d20845b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d2083b24f0 con 0x55d20844ea90 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d208350870 con 0x55d20844e170 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4a3821740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2083ae4b0 con 0x55d208459350 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4937fe640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 0x7fb4880a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41288/0 (socket says 172.21.15.73:41288) 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49a59c640 1 -- 172.21.15.73:0/3874564232 learned_addr learned my addr 172.21.15.73:0/3874564232 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb493fff640 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d20844e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49ad9d640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 0x7fb4880a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb4937fe640 1 -- 172.21.15.73:0/3842183363 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fb4940f0be0 con 0x7fb49c105840 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb493fff640 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d20845b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49a59c640 1 -- 172.21.15.73:0/3874564232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 msgr2=0x55d20845b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.993+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d20845b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb49a59c640 1 -- 172.21.15.73:0/3874564232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 msgr2=0x55d20844e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d20844e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb49a59c640 1 -- 172.21.15.73:0/3874564232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d20836c9b0 con 0x55d20844ea90 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00ad00 tx=0x7fb48c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e210f529865ed485 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb49ad9d640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 0x7fb4880a52e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4940ee0e0 tx=0x7fb4940b2290 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb490ff9640 1 -- 172.21.15.73:0/3874564232 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c01d070 con 0x55d20844ea90 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb490ff9640 1 -- 172.21.15.73:0/3874564232 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48c018af0 con 0x55d20844ea90 2024-06-06T07:10:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.994+0000 7fb490ff9640 1 -- 172.21.15.73:0/3874564232 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c010b40 con 0x55d20844ea90 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- 172.21.15.73:0/3874564232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 msgr2=0x55d208458e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00ad00 tx=0x7fb48c00f790 comp rx=0 tx=0).stop 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- 172.21.15.73:0/3874564232 shutdown_connections 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d20845b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208458e10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3874564232 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d20844e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- 172.21.15.73:0/3874564232 >> 172.21.15.73:0/3874564232 conn(0x55d20844a680 msgr2=0x55d20844aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- 172.21.15.73:0/3874564232 shutdown_connections 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- 172.21.15.73:0/3874564232 wait complete. 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- start start 2024-06-06T07:10:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 Processor -- start 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844ea90 0x55d208456850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208456d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d20845bf80 con 0x55d20844e170 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d20833bae0 con 0x55d20844ea90 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d20845c100 con 0x55d208459350 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.995+0000 7fb4a3821740 10 client.0 _ll_register_callbacks cb 0x55d2083749b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41290/0 (socket says 172.21.15.73:41290) 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 -- 172.21.15.73:0/3476551632 learned_addr learned my addr 172.21.15.73:0/3476551632 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844ea90 0x55d208456850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 -- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 msgr2=0x55d208456d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208456d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 -- 172.21.15.73:0/3476551632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844ea90 msgr2=0x55d208456850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844ea90 0x55d208456850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 -- 172.21.15.73:0/3476551632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2080c6210 con 0x55d20844e170 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb493fff640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb4840042e0 tx=0x7fb4840039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f051a1753d55bd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484005220 con 0x55d20844e170 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fb4840053c0 con 0x55d20844e170 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb48c018af0 con 0x55d20844e170 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb4840055c0 con 0x55d20844e170 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb4a3821740 10 client.16620 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:42.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.996+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d208192620 con 0x55d20844e170 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb48c018af0 con 0x55d20844e170 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fb48401c030 con 0x55d20844e170 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 10 client.16620 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 10 client.16620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 20 client.16620 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084695a0 0x55d20846b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 4 client.16620 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2083c5f10 con 0x55d2084695a0 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084695a0 0x55d20846b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb481ffb640 10 client.16620 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.997+0000 7fb49a59c640 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084695a0 0x55d20846b990 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62a363891f2b8e06 server_cookie=a28a7f1c5b16c602 in_seq=0 out_seq=0 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb481ffb640 1 -- 172.21.15.73:0/3476551632 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2083c5f10 con 0x55d2084695a0 2024-06-06T07:10:42.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb481ffb640 10 client.16620 handle_command_reply: tid=0 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 client.16620 shutdown 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 msgr2=0x55d208456310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fb4840042e0 tx=0x7fb4840039c0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 shutdown_connections 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084695a0 0x55d20846b990 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208456d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844ea90 0x55d208456850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb481ffb640 0 client.16620 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 --2- 172.21.15.73:0/3476551632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208456310 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 >> 172.21.15.73:0/3476551632 conn(0x55d20844a680 msgr2=0x55d208450400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 shutdown_connections 2024-06-06T07:10:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.998+0000 7fb4a3821740 1 -- 172.21.15.73:0/3476551632 wait complete. 2024-06-06T07:10:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:41.999+0000 7fb4a3821740 20 client.16620 trim_cache size 0 max 16384 2024-06-06T07:10:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 Processor -- start 2024-06-06T07:10:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 -- start start 2024-06-06T07:10:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d20844ea90 0x55d208461540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d208459350 0x55d208461a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1adc70 con 0x55d20844e170 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1af900 con 0x55d208459350 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb4a3821740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1ae340 con 0x55d20844ea90 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41294/0 (socket says 172.21.15.73:41294) 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 -- 172.21.15.73:0/4204267732 learned_addr learned my addr 172.21.15.73:0/4204267732 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb49a59c640 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d20844ea90 0x55d208461540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d208459350 0x55d208461a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 -- 172.21.15.73:0/4204267732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d20844ea90 msgr2=0x55d208461540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d20844ea90 0x55d208461540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 -- 172.21.15.73:0/4204267732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d208459350 msgr2=0x55d208461a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d208459350 0x55d208461a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 -- 172.21.15.73:0/4204267732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2084510c0 con 0x55d20844e170 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb493fff640 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb484001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a077865c0c509c62 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.014+0000 7fb483fff640 1 -- 172.21.15.73:0/4204267732 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb484033070 con 0x55d20844e170 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb483fff640 1 -- 172.21.15.73:0/4204267732 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb484012a70 con 0x55d20844e170 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb483fff640 1 -- 172.21.15.73:0/4204267732 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48402b430 con 0x55d20844e170 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- 172.21.15.73:0/4204267732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 msgr2=0x55d208450b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fb484000c00 tx=0x7fb484001e70 comp rx=0 tx=0).stop 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- 172.21.15.73:0/4204267732 shutdown_connections 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d208459350 0x55d208461a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d20844ea90 0x55d208461540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4204267732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844e170 0x55d208450b80 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- 172.21.15.73:0/4204267732 >> 172.21.15.73:0/4204267732 conn(0x55d20844a680 msgr2=0x55d20844aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- 172.21.15.73:0/4204267732 shutdown_connections 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- 172.21.15.73:0/4204267732 wait complete. 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 -- start start 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.015+0000 7fb4a3821740 1 Processor -- start 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d208453f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208471300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1af1d0 con 0x55d20844ea90 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1aea70 con 0x55d20844e170 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb49c1abc70 con 0x55d208459350 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 10 client.0 _ll_register_callbacks cb 0x7fb48400bd60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41300/0 (socket says 172.21.15.73:41300) 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 -- 172.21.15.73:0/4172140587 learned_addr learned my addr 172.21.15.73:0/4172140587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb493fff640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d208453f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb493fff640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208471300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 -- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 msgr2=0x55d208471300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208471300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 -- 172.21.15.73:0/4172140587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 msgr2=0x55d208453f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d208453f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 -- 172.21.15.73:0/4172140587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d208471b00 con 0x55d20844ea90 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb493fff640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208471300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb49a59c640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00fc90 tx=0x7fb48c00a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6ee4946d09e478 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48c01d040 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fb48c0187c0 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb484012a70 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb48c03b050 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 10 client.16626 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.016+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d2084d6300 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb484012a70 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fb48c010810 con 0x55d20844ea90 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 10 client.16626 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 20 client.16626 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084d67f0 0x55d2084d8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 4 client.16626 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:10:42.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2083bfc20 con 0x55d2084d67f0 2024-06-06T07:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb493fff640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084d67f0 0x55d2084d8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb481ffb640 10 client.16626 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:42.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.017+0000 7fb493fff640 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084d67f0 0x55d2084d8c10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b794fb904de1ae7 server_cookie=76ed23d1b9e292f0 in_seq=0 out_seq=0 2024-06-06T07:10:42.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb481ffb640 1 -- 172.21.15.73:0/4172140587 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d2083bfc20 con 0x55d2084d67f0 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb481ffb640 10 client.16626 handle_command_reply: tid=0 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 client.16626 shutdown 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 msgr2=0x55d208470dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb48c00fc90 tx=0x7fb48c00a820 comp rx=0 tx=0).stop 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 shutdown_connections 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2084d67f0 0x55d2084d8c10 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d208459350 0x55d208471300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d20844ea90 0x55d208470dc0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 --2- 172.21.15.73:0/4172140587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d20844e170 0x55d208453f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb481ffb640 0 client.16626 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.048+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 >> 172.21.15.73:0/4172140587 conn(0x55d20844a680 msgr2=0x55d208457b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 shutdown_connections 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a3821740 1 -- 172.21.15.73:0/4172140587 wait complete. 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a3821740 20 client.16626 trim_cache size 0 max 16384 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:10:42.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 msgr2=0x7fb4880a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 0x7fb4880a52e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4940ee0e0 tx=0x7fb4940b2290 comp rx=0 tx=0).stop 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 msgr2=0x7fb49c19b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fb494009dc0 tx=0x7fb494032ff0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 shutdown_connections 2024-06-06T07:10:42.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb49c105840 0x7fb49c19b710 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb4880a2e20 0x7fb4880a52e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb49c1015b0 0x7fb49c1973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 --2- 172.21.15.73:0/3842183363 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb49c100bf0 0x7fb49c196ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 >> 172.21.15.73:0/3842183363 conn(0x7fb49c0766a0 msgr2=0x7fb49c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 shutdown_connections 2024-06-06T07:10:42.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:42.049+0000 7fb4a0a38640 1 -- 172.21.15.73:0/3842183363 wait complete. 2024-06-06T07:10:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:10:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:10:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:10:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:10:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:10:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:10:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:10:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:10:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:10:42.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200331 2.19 MB/sec execute 349 sec latency 3868.017 ms 2024-06-06T07:10:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:42 smithi073 ceph-mon[32303]: pgmap v292: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:42 smithi136 ceph-mon[35901]: pgmap v292: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:43.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:42 smithi170 ceph-mon[36108]: pgmap v292: 129 pgs: 129 active+clean; 125 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 475 KiB/s wr, 65 op/s 2024-06-06T07:10:43.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 200501 2.19 MB/sec execute 350 sec latency 4573.016 ms 2024-06-06T07:10:44.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 201039 2.18 MB/sec execute 351 sec latency 19.268 ms 2024-06-06T07:10:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:44 smithi073 ceph-mon[32303]: pgmap v293: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 547 KiB/s wr, 92 op/s 2024-06-06T07:10:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:44 smithi136 ceph-mon[35901]: pgmap v293: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 547 KiB/s wr, 92 op/s 2024-06-06T07:10:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:44 smithi170 ceph-mon[36108]: pgmap v293: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 547 KiB/s wr, 92 op/s 2024-06-06T07:10:45.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 201649 2.19 MB/sec execute 352 sec latency 19.321 ms 2024-06-06T07:10:46.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202166 2.20 MB/sec execute 353 sec latency 14.297 ms 2024-06-06T07:10:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:46 smithi073 ceph-mon[32303]: pgmap v294: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 500 KiB/s wr, 73 op/s 2024-06-06T07:10:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:46 smithi136 ceph-mon[35901]: pgmap v294: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 500 KiB/s wr, 73 op/s 2024-06-06T07:10:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:46 smithi170 ceph-mon[36108]: pgmap v294: 129 pgs: 129 active+clean; 123 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 500 KiB/s wr, 73 op/s 2024-06-06T07:10:47.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202255 2.20 MB/sec execute 354 sec latency 837.780 ms 2024-06-06T07:10:48.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202255 2.19 MB/sec execute 355 sec latency 1837.828 ms 2024-06-06T07:10:49.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202255 2.18 MB/sec execute 356 sec latency 2837.862 ms 2024-06-06T07:10:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:49 smithi136 ceph-mon[35901]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:49 smithi170 ceph-mon[36108]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:49 smithi073 ceph-mon[32303]: pgmap v295: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 829 KiB/s rd, 1.0 MiB/s wr, 123 op/s 2024-06-06T07:10:50.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202322 2.18 MB/sec execute 357 sec latency 3126.779 ms 2024-06-06T07:10:51.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202322 2.17 MB/sec execute 358 sec latency 1441.704 ms 2024-06-06T07:10:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:51 smithi136 ceph-mon[35901]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:51 smithi170 ceph-mon[36108]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:52.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:51 smithi073 ceph-mon[32303]: pgmap v296: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:52.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202322 2.17 MB/sec execute 359 sec latency 2441.776 ms 2024-06-06T07:10:53.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202322 2.16 MB/sec execute 360 sec latency 3441.807 ms 2024-06-06T07:10:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:53 smithi136 ceph-mon[35901]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:53 smithi170 ceph-mon[36108]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:54.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:53 smithi073 ceph-mon[32303]: pgmap v297: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 600 KiB/s wr, 76 op/s 2024-06-06T07:10:54.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202322 2.15 MB/sec execute 361 sec latency 4441.845 ms 2024-06-06T07:10:55.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 -- 172.21.15.73:0/2134169937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c101980 msgr2=0x7fd61c101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2134169937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c101980 0x7fd61c101e00 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd60c009930 tx=0x7fd60c02a420 comp rx=0 tx=0).stop 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 -- 172.21.15.73:0/2134169937 shutdown_connections 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2134169937 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c102340 0x7fd61c10e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2134169937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c101980 0x7fd61c101e00 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2134169937 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 0x7fd61c107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 -- 172.21.15.73:0/2134169937 >> 172.21.15.73:0/2134169937 conn(0x7fd61c0fd450 msgr2=0x7fd61c0ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 -- 172.21.15.73:0/2134169937 shutdown_connections 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.139+0000 7fd6237e7640 1 -- 172.21.15.73:0/2134169937 wait complete. 2024-06-06T07:10:55.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 Processor -- start 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 -- start start 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 0x7fd61c073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 0x7fd61c074720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c111c50 con 0x7fd61c102340 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c111ad0 con 0x7fd61c101980 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd6237e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c111dd0 con 0x7fd61c107990 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd620d82640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 0x7fd61c073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd620d82640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37144/0 (socket says 172.21.15.73:37144) 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd620d82640 1 -- 172.21.15.73:0/2764593690 learned_addr learned my addr 172.21.15.73:0/2764593690 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:55.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621d84640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 0x7fd61c074720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 -- 172.21.15.73:0/2764593690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 msgr2=0x7fd61c074720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 0x7fd61c074720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 -- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 msgr2=0x7fd61c0741e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd621583640 1 -- 172.21.15.73:0/2764593690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1a6650 con 0x7fd61c101980 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.140+0000 7fd620d82640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd621583640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 0x7fd61c073ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd6127fc640 1 -- 172.21.15.73:0/2764593690 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800ccc0 con 0x7fd61c101980 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a6940 con 0x7fd61c101980 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd6127fc640 1 -- 172.21.15.73:0/2764593690 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd618005020 con 0x7fd61c101980 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd6127fc640 1 -- 172.21.15.73:0/2764593690 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618004a00 con 0x7fd61c101980 2024-06-06T07:10:55.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.141+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a6e10 con 0x7fd61c101980 2024-06-06T07:10:55.150 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 Processor -- start 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 -- start start 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6127fc640 1 -- 172.21.15.73:0/2764593690 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd618004ba0 con 0x7fd61c101980 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97a90 0x55ed55ca1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ed55bfb4f0 con 0x55ed55ca2350 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ed55b99870 con 0x55ed55c97170 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd6265d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ed55bf74b0 con 0x55ed55c97a90 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd620d82640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd621d84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd621d84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47274/0 (socket says 172.21.15.73:47274) 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd621583640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97a90 0x55ed55ca1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.144+0000 7fd621d84640 1 -- 172.21.15.73:0/2592749968 learned_addr learned my addr 172.21.15.73:0/2592749968 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 -- 172.21.15.73:0/2592749968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97a90 msgr2=0x55ed55ca1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97a90 0x55ed55ca1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 -- 172.21.15.73:0/2592749968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 msgr2=0x55ed55c97550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 -- 172.21.15.73:0/2592749968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55bb59b0 con 0x55ed55ca2350 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd621d84640 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd6127fc640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 0x7fd6080a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd620d82640 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca4740 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd60c004770 tx=0x7fd60c032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6339ebfb55f4c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd621d84640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 0x7fd6080a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd6127fc640 1 -- 172.21.15.73:0/2764593690 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fd6180cf750 con 0x7fd61c101980 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd6077fe640 1 -- 172.21.15.73:0/2592749968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c035040 con 0x55ed55ca2350 2024-06-06T07:10:55.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd6077fe640 1 -- 172.21.15.73:0/2592749968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd60c031820 con 0x55ed55ca2350 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.145+0000 7fd6077fe640 1 -- 172.21.15.73:0/2592749968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0049a0 con 0x55ed55ca2350 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- 172.21.15.73:0/2592749968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 msgr2=0x55ed55ca4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca4740 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd60c004770 tx=0x7fd60c032f60 comp rx=0 tx=0).stop 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd621d84640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 0x7fd6080a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd614005f40 tx=0x7fd614009600 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- 172.21.15.73:0/2592749968 shutdown_connections 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca4740 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97a90 0x55ed55ca1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 --2- 172.21.15.73:0/2592749968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97170 0x55ed55c97550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- 172.21.15.73:0/2592749968 >> 172.21.15.73:0/2592749968 conn(0x55ed55c93680 msgr2=0x55ed55c93a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- 172.21.15.73:0/2592749968 shutdown_connections 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- 172.21.15.73:0/2592749968 wait complete. 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 -- start start 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.146+0000 7fd6265d5740 1 Processor -- start 2024-06-06T07:10:55.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55c9f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55c9fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ca4f20 con 0x55ed55c97170 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ed55b84ae0 con 0x55ed55c97a90 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ed55ca50a0 con 0x55ed55ca2350 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd6265d5740 10 client.0 _ll_register_callbacks cb 0x55ed55bbd9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37162/0 (socket says 172.21.15.73:37162) 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 -- 172.21.15.73:0/1829907184 learned_addr learned my addr 172.21.15.73:0/1829907184 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621d84640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55c9fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd620d82640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55c9f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 -- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 msgr2=0x55ed55c9fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55c9fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 -- 172.21.15.73:0/1829907184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 msgr2=0x55ed55c9f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55c9f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 -- 172.21.15.73:0/1829907184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed5590f210 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd621583640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c6a0 tx=0x7fd618097500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa3cf9e389af115 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.147+0000 7fd604ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61801c070 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fd618097ed0 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd60c031820 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd61809b630 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 10 client.16638 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ed559db620 con 0x55ed55c97170 2024-06-06T07:10:55.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd60c031820 con 0x55ed55c97170 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fd618086070 con 0x55ed55c97170 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 10 client.16638 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 20 client.16638 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55cb24c0 0x55ed55cb4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 4 client.16638 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed55c0ef10 con 0x55ed55cb24c0 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.148+0000 7fd620d82640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55cb24c0 0x55ed55cb4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd620d82640 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55cb24c0 0x55ed55cb4910 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4c400464ccf929d server_cookie=7611c5b639514b9c in_seq=0 out_seq=0 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd604ff9640 10 client.16638 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd604ff9640 1 -- 172.21.15.73:0/1829907184 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ed55c0ef10 con 0x55ed55cb24c0 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd604ff9640 10 client.16638 handle_command_reply: tid=0 2024-06-06T07:10:55.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 client.16638 shutdown 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 msgr2=0x55ed55c9f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c6a0 tx=0x7fd618097500 comp rx=0 tx=0).stop 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 shutdown_connections 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55cb24c0 0x55ed55cb4910 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.149+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55c9fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55c9f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1829907184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c9f270 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd604ff9640 0 client.16638 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 >> 172.21.15.73:0/1829907184 conn(0x55ed55c93680 msgr2=0x55ed55c993b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 shutdown_connections 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 1 -- 172.21.15.73:0/1829907184 wait complete. 2024-06-06T07:10:55.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.150+0000 7fd6265d5740 20 client.16638 trim_cache size 0 max 16384 2024-06-06T07:10:55.170 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 Processor -- start 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 -- start start 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55cba220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55cba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b10d0 con 0x55ed55c97170 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b2d60 con 0x55ed55c97a90 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd6265d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b17a0 con 0x55ed55ca2350 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37176/0 (socket says 172.21.15.73:37176) 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 -- 172.21.15.73:0/1808428858 learned_addr learned my addr 172.21.15.73:0/1808428858 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621d84640 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55cba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 -- 172.21.15.73:0/1808428858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 msgr2=0x55ed55cba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55cba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 -- 172.21.15.73:0/1808428858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 msgr2=0x55ed55cba220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55cba220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 -- 172.21.15.73:0/1808428858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed55c98100 con 0x55ed55c97170 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621d84640 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55cba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.166+0000 7fd621583640 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cd090 tx=0x7fd618099d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f43dd935c00b3847 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd606ffd640 1 -- 172.21.15.73:0/1808428858 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6180b8070 con 0x55ed55c97170 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd606ffd640 1 -- 172.21.15.73:0/1808428858 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6180d5240 con 0x55ed55c97170 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd606ffd640 1 -- 172.21.15.73:0/1808428858 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61801c040 con 0x55ed55c97170 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 -- 172.21.15.73:0/1808428858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 msgr2=0x55ed55c99230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd6180cd090 tx=0x7fd618099d70 comp rx=0 tx=0).stop 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 -- 172.21.15.73:0/1808428858 shutdown_connections 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55ca2350 0x55ed55cba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55cba220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 --2- 172.21.15.73:0/1808428858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55c97170 0x55ed55c99230 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 -- 172.21.15.73:0/1808428858 >> 172.21.15.73:0/1808428858 conn(0x55ed55c93680 msgr2=0x55ed55c98b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 -- 172.21.15.73:0/1808428858 shutdown_connections 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.167+0000 7fd6265d5740 1 -- 172.21.15.73:0/1808428858 wait complete. 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 -- start start 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 Processor -- start 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 0x55ed55ca6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55ca65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b2630 con 0x55ed55ca2350 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1b1ed0 con 0x55ed55c97a90 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c1af0d0 con 0x55ed55c97170 2024-06-06T07:10:55.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd6265d5740 10 client.0 _ll_register_callbacks cb 0x7fd61800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd621d84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd621583640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 0x55ed55ca6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd621d84640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37188/0 (socket says 172.21.15.73:37188) 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd621d84640 1 -- 172.21.15.73:0/3493201570 learned_addr learned my addr 172.21.15.73:0/3493201570 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.168+0000 7fd620d82640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55ca65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 -- 172.21.15.73:0/3493201570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 msgr2=0x55ed55ca65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55ca65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 -- 172.21.15.73:0/3493201570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 msgr2=0x55ed55ca6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 -- 172.21.15.73:0/3493201570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed55c8ef70 con 0x55ed55c97170 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621d84640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd621583640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 0x55ed55ca6080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c6a0 tx=0x7fd618097500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a2dfd59f9f2a34c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:10:55.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd604ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61801c070 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6180d5240 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 9554+0+0 (secure 0 0 0) 0x7fd61809cb90 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd6180b8070 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd6265d5740 10 client.26137 fetch_fsmap learned FSMap version 47 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55ed55c8ef70 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.169+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6180d5240 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fd6180cf2d0 con 0x55ed55c97170 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 10 client.26137 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 10 client.26137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 20 client.26137 populate_metadata read hostname 'smithi073' 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55d1d620 0x55ed55d1fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:10:55.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 4 client.26137 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed55c08c20 con 0x55ed55d1d620 2024-06-06T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd620d82640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55d1d620 0x55ed55d1fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd620d82640 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55d1d620 0x55ed55d1fa40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f74d9de03454ebe server_cookie=565ea6f266cbb8ca in_seq=0 out_seq=0 2024-06-06T07:10:55.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.170+0000 7fd604ff9640 10 client.26137 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd604ff9640 1 -- 172.21.15.73:0/3493201570 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ed55c08c20 con 0x55ed55d1d620 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd604ff9640 10 client.26137 handle_command_reply: tid=0 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 client.26137 shutdown 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 msgr2=0x55ed55ca6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 0x55ed55ca6080 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd61800c6a0 tx=0x7fd618097500 comp rx=0 tx=0).stop 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 shutdown_connections 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ed55d1d620 0x55ed55d1fa40 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ed55ca2350 0x55ed55ca6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd604ff9640 0 client.26137 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ed55c97a90 0x55ed55ca65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 --2- 172.21.15.73:0/3493201570 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ed55c97170 0x55ed55ca6080 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 >> 172.21.15.73:0/3493201570 conn(0x55ed55c93680 msgr2=0x55ed55c993b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 shutdown_connections 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 1 -- 172.21.15.73:0/3493201570 wait complete. 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.193+0000 7fd6265d5740 20 client.26137 trim_cache size 0 max 16384 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:10:55.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 msgr2=0x7fd6080a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 0x7fd6080a5260 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fd614005f40 tx=0x7fd614009600 comp rx=0 tx=0).stop 2024-06-06T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 msgr2=0x7fd61c073ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 0x7fd61c073ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9f0 tx=0x7fd61800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 shutdown_connections 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd6080a2da0 0x7fd6080a5260 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd61c107990 0x7fd61c074720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd61c102340 0x7fd61c0741e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 --2- 172.21.15.73:0/2764593690 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd61c101980 0x7fd61c073ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 >> 172.21.15.73:0/2764593690 conn(0x7fd61c0fd450 msgr2=0x7fd61c0fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 shutdown_connections 2024-06-06T07:10:55.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:10:55.194+0000 7fd6237e7640 1 -- 172.21.15.73:0/2764593690 wait complete. 2024-06-06T07:10:55.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202505 2.15 MB/sec execute 362 sec latency 4732.547 ms 2024-06-06T07:10:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:55 smithi136 ceph-mon[35901]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 634 KiB/s wr, 90 op/s 2024-06-06T07:10:55.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:55 smithi170 ceph-mon[36108]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 634 KiB/s wr, 90 op/s 2024-06-06T07:10:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:55 smithi073 ceph-mon[32303]: pgmap v298: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 694 KiB/s rd, 634 KiB/s wr, 90 op/s 2024-06-06T07:10:56.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 202797 2.15 MB/sec execute 363 sec latency 15.771 ms 2024-06-06T07:10:57.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203294 2.15 MB/sec execute 364 sec latency 14.993 ms 2024-06-06T07:10:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:57 smithi136 ceph-mon[35901]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 562 KiB/s wr, 62 op/s 2024-06-06T07:10:57.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:57 smithi170 ceph-mon[36108]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 562 KiB/s wr, 62 op/s 2024-06-06T07:10:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:57 smithi073 ceph-mon[32303]: pgmap v299: 129 pgs: 129 active+clean; 127 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 562 KiB/s wr, 62 op/s 2024-06-06T07:10:58.448 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203806 2.15 MB/sec execute 365 sec latency 22.965 ms 2024-06-06T07:10:59.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203878 2.14 MB/sec execute 366 sec latency 868.954 ms 2024-06-06T07:10:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:10:59 smithi136 ceph-mon[35901]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1016 KiB/s wr, 114 op/s 2024-06-06T07:10:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:10:59 smithi170 ceph-mon[36108]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1016 KiB/s wr, 114 op/s 2024-06-06T07:11:00.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:10:59 smithi073 ceph-mon[32303]: pgmap v300: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 657 KiB/s rd, 1016 KiB/s wr, 114 op/s 2024-06-06T07:11:00.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203878 2.14 MB/sec execute 367 sec latency 1869.080 ms 2024-06-06T07:11:01.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203878 2.13 MB/sec execute 368 sec latency 2869.132 ms 2024-06-06T07:11:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:01 smithi136 ceph-mon[35901]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:01 smithi170 ceph-mon[36108]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:02.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:01 smithi073 ceph-mon[32303]: pgmap v301: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:02.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 203878 2.13 MB/sec execute 369 sec latency 3869.178 ms 2024-06-06T07:11:03.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 204035 2.12 MB/sec execute 370 sec latency 4572.388 ms 2024-06-06T07:11:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:03 smithi136 ceph-mon[35901]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:03.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:03 smithi170 ceph-mon[36108]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:04.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:03 smithi073 ceph-mon[32303]: pgmap v302: 129 pgs: 129 active+clean; 126 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 488 KiB/s wr, 64 op/s 2024-06-06T07:11:04.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 204572 2.12 MB/sec execute 371 sec latency 58.349 ms 2024-06-06T07:11:05.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205178 2.13 MB/sec execute 372 sec latency 14.693 ms 2024-06-06T07:11:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:05 smithi170 ceph-mon[36108]: pgmap v303: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 536 KiB/s wr, 88 op/s 2024-06-06T07:11:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:05 smithi170 ceph-mon[36108]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:11:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:05 smithi073 ceph-mon[32303]: pgmap v303: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 536 KiB/s wr, 88 op/s 2024-06-06T07:11:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:05 smithi073 ceph-mon[32303]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:11:06.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:05 smithi136 ceph-mon[35901]: pgmap v303: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 536 KiB/s wr, 88 op/s 2024-06-06T07:11:06.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:05 smithi136 ceph-mon[35901]: osdmap e111: 12 total, 12 up, 12 in 2024-06-06T07:11:06.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205676 2.13 MB/sec execute 373 sec latency 17.674 ms 2024-06-06T07:11:07.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205802 2.13 MB/sec execute 374 sec latency 735.314 ms 2024-06-06T07:11:07.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:07 smithi170 ceph-mon[36108]: pgmap v305: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 602 KiB/s wr, 89 op/s 2024-06-06T07:11:08.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:07 smithi073 ceph-mon[32303]: pgmap v305: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 602 KiB/s wr, 89 op/s 2024-06-06T07:11:08.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:07 smithi136 ceph-mon[35901]: pgmap v305: 129 pgs: 129 active+clean; 123 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 602 KiB/s wr, 89 op/s 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 -- 172.21.15.73:0/784202849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c1019b0 msgr2=0x7f374c101e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 --2- 172.21.15.73:0/784202849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c1019b0 0x7f374c101e30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f3740009930 tx=0x7f374002a420 comp rx=0 tx=0).stop 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 -- 172.21.15.73:0/784202849 shutdown_connections 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 --2- 172.21.15.73:0/784202849 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c102370 0x7f374c10e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 --2- 172.21.15.73:0/784202849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c1019b0 0x7f374c101e30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 --2- 172.21.15.73:0/784202849 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1079c0 0x7f374c107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 -- 172.21.15.73:0/784202849 >> 172.21.15.73:0/784202849 conn(0x7f374c078e30 msgr2=0x7f374c079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 -- 172.21.15.73:0/784202849 shutdown_connections 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.292+0000 7f3752d77640 1 -- 172.21.15.73:0/784202849 wait complete. 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 Processor -- start 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 -- start start 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1019b0 0x7f374c198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c1079c0 0x7f374c19c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111c50 con 0x7f374c102370 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111ad0 con 0x7f374c1079c0 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3752d77640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f374c111dd0 con 0x7f374c1019b0 2024-06-06T07:11:08.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45562/0 (socket says 172.21.15.73:45562) 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 -- 172.21.15.73:0/2304785649 learned_addr learned my addr 172.21.15.73:0/2304785649 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3750b13640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1019b0 0x7f374c198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f3751314640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c1079c0 0x7f374c19c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 -- 172.21.15.73:0/2304785649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1019b0 msgr2=0x7f374c198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1019b0 0x7f374c198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 -- 172.21.15.73:0/2304785649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c1079c0 msgr2=0x7f374c19c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c1079c0 0x7f374c19c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.293+0000 7f374bfff640 1 -- 172.21.15.73:0/2304785649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374c19d120 con 0x7f374c102370 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f374bfff640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3740004770 tx=0x7f37400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f3749ffb640 1 -- 172.21.15.73:0/2304785649 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f374003f070 con 0x7f374c102370 2024-06-06T07:11:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f3749ffb640 1 -- 172.21.15.73:0/2304785649 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f374003b460 con 0x7f374c102370 2024-06-06T07:11:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374c19d3b0 con 0x7f374c102370 2024-06-06T07:11:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f3749ffb640 1 -- 172.21.15.73:0/2304785649 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740035070 con 0x7f374c102370 2024-06-06T07:11:08.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.294+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f374c1a4ca0 con 0x7f374c102370 2024-06-06T07:11:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.296+0000 7f3749ffb640 1 -- 172.21.15.73:0/2304785649 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f374003b610 con 0x7f374c102370 2024-06-06T07:11:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 Processor -- start 2024-06-06T07:11:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 -- start start 2024-06-06T07:11:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e657550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e664740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f0e5bb4f0 con 0x556f0e657a90 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556f0e559870 con 0x556f0e662350 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3755b63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556f0e5b74b0 con 0x556f0e657170 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3749ffb640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 0x7f37340a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45578/0 (socket says 172.21.15.73:45578) 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 -- 172.21.15.73:0/3198455739 learned_addr learned my addr 172.21.15.73:0/3198455739 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3751314640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 0x7f37340a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3751314640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e657550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3749ffb640 1 -- 172.21.15.73:0/2304785649 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 9642+0+0 (secure 0 0 0) 0x7f37400efcd0 con 0x7f374c102370 2024-06-06T07:11:08.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f374bfff640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e664740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 -- 172.21.15.73:0/3198455739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 msgr2=0x556f0e657550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e657550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 -- 172.21.15.73:0/3198455739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 msgr2=0x556f0e664740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e664740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f3750b13640 1 -- 172.21.15.73:0/3198455739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0e5759b0 con 0x556f0e657a90 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.297+0000 7f374bfff640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e664740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.298+0000 7f3750b13640 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f373c00cb80 tx=0x7f373c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ed0fcf6f9e19a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.298+0000 7f3751314640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 0x7f37340a5260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3744009fd0 tx=0x7f3744009570 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.298+0000 7f3732ffd640 1 -- 172.21.15.73:0/3198455739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c010630 con 0x556f0e657a90 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.298+0000 7f3732ffd640 1 -- 172.21.15.73:0/3198455739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373c010c80 con 0x556f0e657a90 2024-06-06T07:11:08.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.298+0000 7f3732ffd640 1 -- 172.21.15.73:0/3198455739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c00fb20 con 0x556f0e657a90 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- 172.21.15.73:0/3198455739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 msgr2=0x556f0e661e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f373c00cb80 tx=0x7f373c00e720 comp rx=0 tx=0).stop 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- 172.21.15.73:0/3198455739 shutdown_connections 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e664740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657a90 0x556f0e661e10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 --2- 172.21.15.73:0/3198455739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e657550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- 172.21.15.73:0/3198455739 >> 172.21.15.73:0/3198455739 conn(0x556f0e653680 msgr2=0x556f0e653a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- 172.21.15.73:0/3198455739 shutdown_connections 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- 172.21.15.73:0/3198455739 wait complete. 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 -- start start 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.299+0000 7f3755b63740 1 Processor -- start 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e65f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e657a90 0x556f0e65f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556f0e664f80 con 0x556f0e662350 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556f0e544ae0 con 0x556f0e657a90 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556f0e665100 con 0x556f0e657170 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3755b63740 10 client.0 _ll_register_callbacks cb 0x556f0e57d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f374bfff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e657a90 0x556f0e65f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45580/0 (socket says 172.21.15.73:45580) 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 -- 172.21.15.73:0/1990356857 learned_addr learned my addr 172.21.15.73:0/1990356857 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3750b13640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e65f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 -- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 msgr2=0x556f0e65f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e65f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 -- 172.21.15.73:0/1990356857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e657a90 msgr2=0x556f0e65f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e657a90 0x556f0e65f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 -- 172.21.15.73:0/1990356857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0e2cf210 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f3751314640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f374400f770 tx=0x7f3744004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87dfbcac443ae4a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f372bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.300+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3744024030 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9642+0+0 (secure 0 0 0) 0x7f3744012870 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c010c80 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3744031050 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 10 client.16656 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f0e39b620 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373c010c80 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7f374401d050 con 0x556f0e662350 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 10 client.16656 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 10 client.16656 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:08.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 20 client.16656 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6725c0 0x556f0e674a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 4 client.16656 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.301+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f0e5cef10 con 0x556f0e6725c0 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.302+0000 7f3750b13640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6725c0 0x556f0e674a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.302+0000 7f3750b13640 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6725c0 0x556f0e674a10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec90903a5b9b6534 server_cookie=9872386803e3edfa in_seq=0 out_seq=0 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.302+0000 7f372bfff640 10 client.16656 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f372bfff640 1 -- 172.21.15.73:0/1990356857 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556f0e5cef10 con 0x556f0e6725c0 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f372bfff640 10 client.16656 handle_command_reply: tid=0 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 client.16656 shutdown 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 msgr2=0x556f0e65fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f374400f770 tx=0x7f3744004a40 comp rx=0 tx=0).stop 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 shutdown_connections 2024-06-06T07:11:08.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6725c0 0x556f0e674a10 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e662350 0x556f0e65fd70 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e657a90 0x556f0e65f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f372bfff640 0 client.16656 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 --2- 172.21.15.73:0/1990356857 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657170 0x556f0e65f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 >> 172.21.15.73:0/1990356857 conn(0x556f0e653680 msgr2=0x556f0e6593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 shutdown_connections 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 1 -- 172.21.15.73:0/1990356857 wait complete. 2024-06-06T07:11:08.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.303+0000 7f3755b63740 20 client.16656 trim_cache size 0 max 16384 2024-06-06T07:11:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 Processor -- start 2024-06-06T07:11:08.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 -- start start 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e652660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e67a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1af090 con 0x556f0e657170 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b0d20 con 0x556f0e662350 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3755b63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1af760 con 0x556f0e657a90 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45590/0 (socket says 172.21.15.73:45590) 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 -- 172.21.15.73:0/989932739 learned_addr learned my addr 172.21.15.73:0/989932739 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f374bfff640 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e652660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 -- 172.21.15.73:0/989932739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 msgr2=0x556f0e652660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e652660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 -- 172.21.15.73:0/989932739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 msgr2=0x556f0e67a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e67a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 -- 172.21.15.73:0/989932739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0e65d840 con 0x556f0e657170 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f374bfff640 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e652660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f3750b13640 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f373c005f90 tx=0x7f373c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41137caac86e9222 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.319+0000 7f37327fc640 1 -- 172.21.15.73:0/989932739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c005240 con 0x556f0e657170 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f37327fc640 1 -- 172.21.15.73:0/989932739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373c00fc60 con 0x556f0e657170 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f37327fc640 1 -- 172.21.15.73:0/989932739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c004950 con 0x556f0e657170 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- 172.21.15.73:0/989932739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 msgr2=0x556f0e659b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f373c005f90 tx=0x7f373c00f3c0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- 172.21.15.73:0/989932739 shutdown_connections 2024-06-06T07:11:08.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e67a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e652660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 --2- 172.21.15.73:0/989932739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e659b30 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- 172.21.15.73:0/989932739 >> 172.21.15.73:0/989932739 conn(0x556f0e653680 msgr2=0x556f0e651f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- 172.21.15.73:0/989932739 shutdown_connections 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- 172.21.15.73:0/989932739 wait complete. 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.320+0000 7f3755b63740 1 -- start start 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 Processor -- start 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e6668e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e666e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1b05f0 con 0x556f0e657170 2024-06-06T07:11:08.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1afe90 con 0x556f0e662350 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f374c1ad090 con 0x556f0e657a90 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3755b63740 10 client.0 _ll_register_callbacks cb 0x7f374c06b220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45598/0 (socket says 172.21.15.73:45598) 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 -- 172.21.15.73:0/1486694539 learned_addr learned my addr 172.21.15.73:0/1486694539 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 -- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 msgr2=0x556f0e6668e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e6668e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 -- 172.21.15.73:0/1486694539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 msgr2=0x556f0e666e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e666e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.321+0000 7f3750b13640 1 -- 172.21.15.73:0/1486694539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0e6dc590 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3750b13640 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f373c0058e0 tx=0x7f373c00fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ba4b2ef652dfe64 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373c034030 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(111..111 src has 1..111) ==== 9642+0+0 (secure 0 0 0) 0x7f373c00f940 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f373c00fc60 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f373c040050 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 10 client.16662 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x556f0e6dd810 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f373c00fc60 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7f373c004b90 con 0x556f0e657170 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 10 client.16662 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 10 client.16662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:08.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 20 client.16662 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6ddcc0 0x556f0e6e0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 4 client.16662 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.322+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f0e5c8c20 con 0x556f0e6ddcc0 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.323+0000 7f374bfff640 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6ddcc0 0x556f0e6e0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.323+0000 7f374bfff640 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6ddcc0 0x556f0e6e0180 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f802073b3a4d5a3b server_cookie=8ec2f8950992780d in_seq=0 out_seq=0 2024-06-06T07:11:08.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.323+0000 7f372bfff640 10 client.16662 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f372bfff640 1 -- 172.21.15.73:0/1486694539 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f0e5c8c20 con 0x556f0e6ddcc0 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f372bfff640 10 client.16662 handle_command_reply: tid=0 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 client.16662 shutdown 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 msgr2=0x556f0e6663a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f373c0058e0 tx=0x7f373c00fe30 comp rx=0 tx=0).stop 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 shutdown_connections 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556f0e6ddcc0 0x556f0e6e0180 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556f0e662350 0x556f0e666e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556f0e657a90 0x556f0e6668e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 --2- 172.21.15.73:0/1486694539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556f0e657170 0x556f0e6663a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f372bfff640 0 client.16662 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 >> 172.21.15.73:0/1486694539 conn(0x556f0e653680 msgr2=0x556f0e65fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 shutdown_connections 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 1 -- 172.21.15.73:0/1486694539 wait complete. 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.350+0000 7f3755b63740 20 client.16662 trim_cache size 0 max 16384 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:11:08.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:11:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 msgr2=0x7f37340a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 0x7f37340a5260 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3744009fd0 tx=0x7f3744009570 comp rx=0 tx=0).stop 2024-06-06T07:11:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 msgr2=0x7f374c198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f3740004770 tx=0x7f37400042f0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 shutdown_connections 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f374c1079c0 0x7f374c19c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f374c102370 0x7f374c198670 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37340a2da0 0x7f37340a5260 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 --2- 172.21.15.73:0/2304785649 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f374c1019b0 0x7f374c198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 >> 172.21.15.73:0/2304785649 conn(0x7f374c078e30 msgr2=0x7f374c0ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 shutdown_connections 2024-06-06T07:11:08.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:08.351+0000 7f3752d77640 1 -- 172.21.15.73:0/2304785649 wait complete. 2024-06-06T07:11:08.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205802 2.12 MB/sec execute 375 sec latency 1735.395 ms 2024-06-06T07:11:09.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:08 smithi073 ceph-mon[32303]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:09.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:08 smithi136 ceph-mon[35901]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:09.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:08 smithi170 ceph-mon[36108]: pgmap v306: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:09.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205802 2.12 MB/sec execute 376 sec latency 2735.465 ms 2024-06-06T07:11:10.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205869 2.11 MB/sec execute 377 sec latency 3024.632 ms 2024-06-06T07:11:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:10 smithi073 ceph-mon[32303]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:10 smithi136 ceph-mon[35901]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:10 smithi170 ceph-mon[36108]: pgmap v307: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:11.449 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205869 2.11 MB/sec execute 378 sec latency 1462.985 ms 2024-06-06T07:11:12.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205869 2.10 MB/sec execute 379 sec latency 2463.089 ms 2024-06-06T07:11:13.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205869 2.10 MB/sec execute 380 sec latency 3463.186 ms 2024-06-06T07:11:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:13 smithi136 ceph-mon[35901]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:13 smithi136 ceph-mon[35901]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:11:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:13 smithi170 ceph-mon[36108]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:13 smithi170 ceph-mon[36108]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:11:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:13 smithi073 ceph-mon[32303]: pgmap v308: 129 pgs: 129 active+clean; 127 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 840 KiB/s rd, 774 KiB/s wr, 105 op/s 2024-06-06T07:11:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:13 smithi073 ceph-mon[32303]: osdmap e112: 12 total, 12 up, 12 in 2024-06-06T07:11:14.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 205869 2.09 MB/sec execute 381 sec latency 4463.247 ms 2024-06-06T07:11:15.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 206070 2.09 MB/sec execute 382 sec latency 4752.131 ms 2024-06-06T07:11:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:15 smithi136 ceph-mon[35901]: pgmap v310: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 835 KiB/s wr, 108 op/s 2024-06-06T07:11:15.825 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:15 smithi073 ceph-mon[32303]: pgmap v310: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 835 KiB/s wr, 108 op/s 2024-06-06T07:11:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:15 smithi170 ceph-mon[36108]: pgmap v310: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 835 KiB/s wr, 108 op/s 2024-06-06T07:11:16.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 206351 2.08 MB/sec execute 383 sec latency 15.276 ms 2024-06-06T07:11:17.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 206826 2.08 MB/sec execute 384 sec latency 16.409 ms 2024-06-06T07:11:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:17 smithi136 ceph-mon[35901]: pgmap v311: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 775 KiB/s wr, 100 op/s 2024-06-06T07:11:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:17 smithi170 ceph-mon[36108]: pgmap v311: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 775 KiB/s wr, 100 op/s 2024-06-06T07:11:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:17 smithi073 ceph-mon[32303]: pgmap v311: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 624 KiB/s rd, 775 KiB/s wr, 100 op/s 2024-06-06T07:11:18.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207352 2.09 MB/sec execute 385 sec latency 22.068 ms 2024-06-06T07:11:19.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207421 2.08 MB/sec execute 386 sec latency 882.594 ms 2024-06-06T07:11:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:19 smithi136 ceph-mon[35901]: pgmap v312: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:19 smithi170 ceph-mon[36108]: pgmap v312: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:19 smithi073 ceph-mon[32303]: pgmap v312: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:20.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207421 2.08 MB/sec execute 387 sec latency 1882.719 ms 2024-06-06T07:11:21.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 -- 172.21.15.73:0/195891127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8401015b0 msgr2=0x7fc84010d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 --2- 172.21.15.73:0/195891127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8401015b0 0x7fc84010d6c0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fc838009ca0 tx=0x7fc83802a8c0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 -- 172.21.15.73:0/195891127 shutdown_connections 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 --2- 172.21.15.73:0/195891127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8401015b0 0x7fc84010d6c0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 --2- 172.21.15.73:0/195891127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 --2- 172.21.15.73:0/195891127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc840105840 0x7fc840105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 -- 172.21.15.73:0/195891127 >> 172.21.15.73:0/195891127 conn(0x7fc8400766a0 msgr2=0x7fc840076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 -- 172.21.15.73:0/195891127 shutdown_connections 2024-06-06T07:11:21.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.448+0000 7fc8453ce640 1 -- 172.21.15.73:0/195891127 wait complete. 2024-06-06T07:11:21.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207421 2.07 MB/sec execute 388 sec latency 2882.769 ms 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 Processor -- start 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 -- start start 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401015b0 0x7fc840197560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc840110b40 con 0x7fc840105840 2024-06-06T07:11:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401109c0 con 0x7fc840100bf0 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc8453ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc840110cc0 con 0x7fc8401015b0 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc83f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47152/0 (socket says 172.21.15.73:47152) 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc83f7fe640 1 -- 172.21.15.73:0/1935978434 learned_addr learned my addr 172.21.15.73:0/1935978434 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.449+0000 7fc837fff640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401015b0 0x7fc840197560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83effd640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 -- 172.21.15.73:0/1935978434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401015b0 msgr2=0x7fc840197560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401015b0 0x7fc840197560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 -- 172.21.15.73:0/1935978434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 msgr2=0x7fc840197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 -- 172.21.15.73:0/1935978434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc84019bfc0 con 0x7fc840105840 2024-06-06T07:11:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83effd640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83f7fe640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc838009dc0 tx=0x7fc838032ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83cff9640 1 -- 172.21.15.73:0/1935978434 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838003cf0 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83cff9640 1 -- 172.21.15.73:0/1935978434 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8380044f0 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc83cff9640 1 -- 172.21.15.73:0/1935978434 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc838030b80 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc84019c250 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.450+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8401a3b40 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.452+0000 7fc83cff9640 1 -- 172.21.15.73:0/1935978434 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc8380046a0 con 0x7fc840105840 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 Processor -- start 2024-06-06T07:11:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 -- start start 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5abd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5acc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5603f5a1f4f0 con 0x5603f5abd850 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603f59bd870 con 0x5603f5abea30 2024-06-06T07:11:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc8481ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603f5a1b4b0 con 0x5603f5aca4a0 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc83cff9640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 0x7fc82c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc83effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc83effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47162/0 (socket says 172.21.15.73:47162) 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc83effd640 1 -- 172.21.15.73:0/603603135 learned_addr learned my addr 172.21.15.73:0/603603135 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc83f7fe640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 0x7fc82c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc837fff640 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5acc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.453+0000 7fc837fff640 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5abd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83cff9640 1 -- 172.21.15.73:0/1935978434 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fc8380f0ce0 con 0x7fc840105840 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 -- 172.21.15.73:0/603603135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 msgr2=0x5603f5acc890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5acc890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 -- 172.21.15.73:0/603603135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 msgr2=0x5603f5abd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5abd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 -- 172.21.15.73:0/603603135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f59d99b0 con 0x5603f5abd850 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83effd640 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c50 tx=0x7fc83000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8a8cc1fa15c01f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc83f7fe640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 0x7fc82c0a52e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8380ee1e0 tx=0x7fc83808be30 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc8357fa640 1 -- 172.21.15.73:0/603603135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc830016070 con 0x5603f5abd850 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc8357fa640 1 -- 172.21.15.73:0/603603135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc830011920 con 0x5603f5abd850 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.454+0000 7fc8357fa640 1 -- 172.21.15.73:0/603603135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8300107f0 con 0x5603f5abd850 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- 172.21.15.73:0/603603135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 msgr2=0x5603f5abdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fc830009c50 tx=0x7fc83000e6c0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- 172.21.15.73:0/603603135 shutdown_connections 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5acc890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5abdcd0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 --2- 172.21.15.73:0/603603135 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5abd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- 172.21.15.73:0/603603135 >> 172.21.15.73:0/603603135 conn(0x5603f5ab9890 msgr2=0x5603f5ab9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- 172.21.15.73:0/603603135 shutdown_connections 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- 172.21.15.73:0/603603135 wait complete. 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 -- start start 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.455+0000 7fc8481ba740 1 Processor -- start 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5ac7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5ac7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5603f5acd330 con 0x5603f5abd850 2024-06-06T07:11:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5603f59a8ae0 con 0x5603f5abea30 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603f5acd4b0 con 0x5603f5aca4a0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 10 client.0 _ll_register_callbacks cb 0x5603f59e19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47176/0 (socket says 172.21.15.73:47176) 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 -- 172.21.15.73:0/3171270612 learned_addr learned my addr 172.21.15.73:0/3171270612 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc83effd640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5ac7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5ac7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 -- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 msgr2=0x5603f5ac7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5ac7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 -- 172.21.15.73:0/3171270612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 msgr2=0x5603f5ac7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5ac7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 -- 172.21.15.73:0/3171270612 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f5733210 con 0x5603f5abd850 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc837fff640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f10 tx=0x7fc828006d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6fc7191b319a528 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8267fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828014ad0 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fc828014c70 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc830011920 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.456+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc828014e70 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 10 client.16674 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5603f57ff620 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc830011920 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fc828016430 con 0x5603f5abd850 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 10 client.16674 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 20 client.16674 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5adaa00 0x5603f5adcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 4 client.16674 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603f5a32f10 con 0x5603f5adaa00 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.457+0000 7fc83effd640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5adaa00 0x5603f5adcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.458+0000 7fc83effd640 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5adaa00 0x5603f5adcec0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35885dbd4163cb7d server_cookie=81f6c84d59d79e1c in_seq=0 out_seq=0 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.458+0000 7fc8267fc640 10 client.16674 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.458+0000 7fc8267fc640 1 -- 172.21.15.73:0/3171270612 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603f5a32f10 con 0x5603f5adaa00 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.458+0000 7fc8267fc640 10 client.16674 handle_command_reply: tid=0 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 client.16674 shutdown 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 msgr2=0x5603f5ac7410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fc828009f10 tx=0x7fc828006d80 comp rx=0 tx=0).stop 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 shutdown_connections 2024-06-06T07:11:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5adaa00 0x5603f5adcec0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5aca4a0 0x5603f5ac7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8267fc640 0 client.16674 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abea30 0x5603f5ac7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3171270612 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5abd850 0x5603f5ac7410 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 >> 172.21.15.73:0/3171270612 conn(0x5603f5ab9890 msgr2=0x5603f5abf700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 shutdown_connections 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 1 -- 172.21.15.73:0/3171270612 wait complete. 2024-06-06T07:11:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.459+0000 7fc8481ba740 20 client.16674 trim_cache size 0 max 16384 2024-06-06T07:11:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.474+0000 7fc8481ba740 1 Processor -- start 2024-06-06T07:11:21.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.474+0000 7fc8481ba740 1 -- start start 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ab9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ac6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401acc40 con 0x5603f5aca4a0 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401ae8d0 con 0x5603f5abd850 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc8481ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401ad310 con 0x5603f5abea30 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47190/0 (socket says 172.21.15.73:47190) 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 -- 172.21.15.73:0/3258513059 learned_addr learned my addr 172.21.15.73:0/3258513059 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:21.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ab9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 -- 172.21.15.73:0/3258513059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 msgr2=0x5603f5ac6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc83effd640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ac6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ac6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 -- 172.21.15.73:0/3258513059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 msgr2=0x5603f5ab9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ab9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 -- 172.21.15.73:0/3258513059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603f5ab1ae0 con 0x5603f5aca4a0 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc83effd640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ac6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.475+0000 7fc837fff640 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc8280197a0 tx=0x7fc828008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b249dbc158bbc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc834ff9640 1 -- 172.21.15.73:0/3258513059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828016630 con 0x5603f5aca4a0 2024-06-06T07:11:21.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc834ff9640 1 -- 172.21.15.73:0/3258513059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82801f070 con 0x5603f5aca4a0 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc834ff9640 1 -- 172.21.15.73:0/3258513059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828014720 con 0x5603f5aca4a0 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 -- 172.21.15.73:0/3258513059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 msgr2=0x5603f5ab6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fc8280197a0 tx=0x7fc828008040 comp rx=0 tx=0).stop 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 -- 172.21.15.73:0/3258513059 shutdown_connections 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ab6830 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ac6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 --2- 172.21.15.73:0/3258513059 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ab9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 -- 172.21.15.73:0/3258513059 >> 172.21.15.73:0/3258513059 conn(0x5603f5ab9890 msgr2=0x5603f5ac9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 -- 172.21.15.73:0/3258513059 shutdown_connections 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.476+0000 7fc8481ba740 1 -- 172.21.15.73:0/3258513059 wait complete. 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 -- start start 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 Processor -- start 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ad2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ad2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401ae1a0 con 0x5603f5aca4a0 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401ada40 con 0x5603f5abd850 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc8401aac40 con 0x5603f5abea30 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc8481ba740 10 client.0 _ll_register_callbacks cb 0x7fc828019690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47200/0 (socket says 172.21.15.73:47200) 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 -- 172.21.15.73:0/2549566454 learned_addr learned my addr 172.21.15.73:0/2549566454 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc83effd640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ad2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ad2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 -- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 msgr2=0x5603f5ad2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ad2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 -- 172.21.15.73:0/2549566454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 msgr2=0x5603f5ad2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ad2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.477+0000 7fc837fff640 1 -- 172.21.15.73:0/2549566454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603f5ae1c70 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc837fff640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc828017ac0 tx=0x7fc828016b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2148e7a3ed9c4533 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828034040 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82801f070 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fc828017700 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc8280059c0 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 10 client.16680 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5603f5b44da0 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc82801f070 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fc82803a070 con 0x5603f5aca4a0 2024-06-06T07:11:21.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 10 client.16680 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 20 client.16680 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5b454e0 0x5603f5b47900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 4 client.16680 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.478+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603f5a2cc20 con 0x5603f5b454e0 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.479+0000 7fc83effd640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5b454e0 0x5603f5b47900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.479+0000 7fc83effd640 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5b454e0 0x5603f5b47900 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=468d2837a0635582 server_cookie=7c84f6c888b9b76d in_seq=0 out_seq=0 2024-06-06T07:11:21.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.479+0000 7fc8267fc640 10 client.16680 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.499+0000 7fc8267fc640 1 -- 172.21.15.73:0/2549566454 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603f5a2cc20 con 0x5603f5b454e0 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.499+0000 7fc8267fc640 10 client.16680 handle_command_reply: tid=0 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.499+0000 7fc8481ba740 1 client.16680 shutdown 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 msgr2=0x5603f5ae1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fc828017ac0 tx=0x7fc828016b60 comp rx=0 tx=0).stop 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 shutdown_connections 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5603f5b454e0 0x5603f5b47900 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5603f5aca4a0 0x5603f5ae1730 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5603f5abea30 0x5603f5ad2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8267fc640 0 client.16680 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 --2- 172.21.15.73:0/2549566454 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5603f5abd850 0x5603f5ad2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 >> 172.21.15.73:0/2549566454 conn(0x5603f5ab9890 msgr2=0x5603f5abee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 shutdown_connections 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 1 -- 172.21.15.73:0/2549566454 wait complete. 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8481ba740 20 client.16680 trim_cache size 0 max 16384 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:11:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 msgr2=0x7fc82c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 0x7fc82c0a52e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc8380ee1e0 tx=0x7fc83808be30 comp rx=0 tx=0).stop 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 msgr2=0x7fc84019b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fc838009dc0 tx=0x7fc838032ff0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 shutdown_connections 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc840105840 0x7fc84019b890 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc82c0a2e20 0x7fc82c0a52e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc8401015b0 0x7fc840197560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 --2- 172.21.15.73:0/1935978434 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc840100bf0 0x7fc840197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.500+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 >> 172.21.15.73:0/1935978434 conn(0x7fc8400766a0 msgr2=0x7fc8400fe870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.501+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 shutdown_connections 2024-06-06T07:11:21.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:21.501+0000 7fc8453ce640 1 -- 172.21.15.73:0/1935978434 wait complete. 2024-06-06T07:11:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:21 smithi136 ceph-mon[35901]: pgmap v313: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:21 smithi170 ceph-mon[36108]: pgmap v313: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:21 smithi073 ceph-mon[32303]: pgmap v313: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:22.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207421 2.07 MB/sec execute 389 sec latency 3882.800 ms 2024-06-06T07:11:23.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207578 2.06 MB/sec execute 390 sec latency 4584.532 ms 2024-06-06T07:11:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:23 smithi136 ceph-mon[35901]: pgmap v314: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:23 smithi170 ceph-mon[36108]: pgmap v314: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:23 smithi073 ceph-mon[32303]: pgmap v314: 129 pgs: 129 active+clean; 131 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 251 KiB/s rd, 584 KiB/s wr, 79 op/s 2024-06-06T07:11:24.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 207990 2.06 MB/sec execute 391 sec latency 392.608 ms 2024-06-06T07:11:25.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 208305 2.06 MB/sec execute 392 sec latency 680.811 ms 2024-06-06T07:11:25.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:25 smithi170 ceph-mon[36108]: pgmap v315: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 596 KiB/s wr, 99 op/s 2024-06-06T07:11:26.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:25 smithi073 ceph-mon[32303]: pgmap v315: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 596 KiB/s wr, 99 op/s 2024-06-06T07:11:26.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:25 smithi136 ceph-mon[35901]: pgmap v315: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 468 KiB/s rd, 596 KiB/s wr, 99 op/s 2024-06-06T07:11:26.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209012 2.07 MB/sec execute 393 sec latency 16.488 ms 2024-06-06T07:11:27.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209346 2.07 MB/sec execute 394 sec latency 138.061 ms 2024-06-06T07:11:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:27 smithi170 ceph-mon[36108]: pgmap v316: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:11:28.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:27 smithi073 ceph-mon[32303]: pgmap v316: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:11:28.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:27 smithi136 ceph-mon[35901]: pgmap v316: 129 pgs: 129 active+clean; 124 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:11:28.450 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209346 2.07 MB/sec execute 395 sec latency 1138.134 ms 2024-06-06T07:11:29.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209346 2.06 MB/sec execute 396 sec latency 2138.229 ms 2024-06-06T07:11:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:29 smithi170 ceph-mon[36108]: pgmap v317: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:11:30.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:29 smithi073 ceph-mon[32303]: pgmap v317: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:11:30.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:29 smithi136 ceph-mon[35901]: pgmap v317: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 852 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:11:30.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209358 2.05 MB/sec execute 397 sec latency 2424.223 ms 2024-06-06T07:11:31.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209358 2.05 MB/sec execute 398 sec latency 1650.636 ms 2024-06-06T07:11:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:31 smithi170 ceph-mon[36108]: pgmap v318: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:32.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:31 smithi073 ceph-mon[32303]: pgmap v318: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:32.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:31 smithi136 ceph-mon[35901]: pgmap v318: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:32.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209358 2.04 MB/sec execute 399 sec latency 2650.751 ms 2024-06-06T07:11:32.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:32 smithi170 ceph-mon[36108]: pgmap v319: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:33.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:32 smithi073 ceph-mon[32303]: pgmap v319: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:33.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:32 smithi136 ceph-mon[35901]: pgmap v319: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 680 KiB/s rd, 603 KiB/s wr, 74 op/s 2024-06-06T07:11:33.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209413 2.04 MB/sec execute 400 sec latency 3361.410 ms 2024-06-06T07:11:34.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209413 2.03 MB/sec execute 401 sec latency 1106.905 ms 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 -- 172.21.15.73:0/3062260632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8781068d0 msgr2=0x7fb878106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 --2- 172.21.15.73:0/3062260632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8781068d0 0x7fb878106cb0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c02a420 comp rx=0 tx=0).stop 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 -- 172.21.15.73:0/3062260632 shutdown_connections 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 --2- 172.21.15.73:0/3062260632 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb878101280 0x7fb87810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 --2- 172.21.15.73:0/3062260632 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781008c0 0x7fb878100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 --2- 172.21.15.73:0/3062260632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8781068d0 0x7fb878106cb0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 -- 172.21.15.73:0/3062260632 >> 172.21.15.73:0/3062260632 conn(0x7fb878076c20 msgr2=0x7fb878077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 -- 172.21.15.73:0/3062260632 shutdown_connections 2024-06-06T07:11:34.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.603+0000 7fb87f121640 1 -- 172.21.15.73:0/3062260632 wait complete. 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 Processor -- start 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 -- start start 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 0x7fb878102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 0x7fb878102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb878110b90 con 0x7fb878101280 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb878110a10 con 0x7fb8781068d0 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87f121640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb878110d10 con 0x7fb8781008c0 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47094/0 (socket says 172.21.15.73:47094) 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 -- 172.21.15.73:0/3049989884 learned_addr learned my addr 172.21.15.73:0/3049989884 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 0x7fb878102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 0x7fb878102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 -- 172.21.15.73:0/3049989884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 msgr2=0x7fb878102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 0x7fb878102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 -- 172.21.15.73:0/3049989884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 msgr2=0x7fb878102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 0x7fb878102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb877fff640 1 -- 172.21.15.73:0/3049989884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8781a65e0 con 0x7fb878101280 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 0x7fb878102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.604+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 0x7fb878102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb877fff640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb86400e9f0 tx=0x7fb86400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb875ffb640 1 -- 172.21.15.73:0/3049989884 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86400cd80 con 0x7fb878101280 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb875ffb640 1 -- 172.21.15.73:0/3049989884 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8640108e0 con 0x7fb878101280 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8781a68d0 con 0x7fb878101280 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb875ffb640 1 -- 172.21.15.73:0/3049989884 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864018920 con 0x7fb878101280 2024-06-06T07:11:34.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.605+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8781a6e70 con 0x7fb878101280 2024-06-06T07:11:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb875ffb640 1 -- 172.21.15.73:0/3049989884 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb864010a90 con 0x7fb878101280 2024-06-06T07:11:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 Processor -- start 2024-06-06T07:11:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 -- start start 2024-06-06T07:11:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4cae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4cd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb875ffb640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 0x7fb8680a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561ad4244f0 con 0x5561ad4c0170 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561ad3c2870 con 0x5561ad4c0a90 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.608+0000 7fb881e2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561ad4204b0 con 0x5561ad4cb350 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47104/0 (socket says 172.21.15.73:47104) 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 -- 172.21.15.73:0/2291618382 learned_addr learned my addr 172.21.15.73:0/2291618382 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 0x7fb8680a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb875ffb640 1 -- 172.21.15.73:0/3049989884 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fb8640cfc40 con 0x7fb878101280 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb877fff640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87cebd640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 -- 172.21.15.73:0/2291618382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 msgr2=0x5561ad4cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4cd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 -- 172.21.15.73:0/2291618382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 msgr2=0x5561ad4cae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4cae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 -- 172.21.15.73:0/2291618382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ad3de9b0 con 0x5561ad4c0170 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87cebd640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb8700092f0 tx=0x7fb870011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34985ef92ca59839 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 0x7fb8680a52e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb870011c50 tx=0x7fb870016000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb862ffd640 1 -- 172.21.15.73:0/2291618382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870022070 con 0x5561ad4c0170 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb862ffd640 1 -- 172.21.15.73:0/2291618382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb870009bc0 con 0x5561ad4c0170 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.609+0000 7fb862ffd640 1 -- 172.21.15.73:0/2291618382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8700052f0 con 0x5561ad4c0170 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 -- 172.21.15.73:0/2291618382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 msgr2=0x5561ad4c0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb8700092f0 tx=0x7fb870011750 comp rx=0 tx=0).stop 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 -- 172.21.15.73:0/2291618382 shutdown_connections 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4cae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 --2- 172.21.15.73:0/2291618382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c0550 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 -- 172.21.15.73:0/2291618382 >> 172.21.15.73:0/2291618382 conn(0x5561ad4bc680 msgr2=0x5561ad4bca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 -- 172.21.15.73:0/2291618382 shutdown_connections 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.610+0000 7fb881e2a740 1 -- 172.21.15.73:0/2291618382 wait complete. 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 -- start start 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 Processor -- start 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4c8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4c8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5561ad4cdf20 con 0x5561ad4c0170 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5561ad3adae0 con 0x5561ad4c0a90 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5561ad4ce0a0 con 0x5561ad4cb350 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb881e2a740 10 client.0 _ll_register_callbacks cb 0x5561ad3e69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47110/0 (socket says 172.21.15.73:47110) 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.611+0000 7fb87cebd640 1 -- 172.21.15.73:0/3462505508 learned_addr learned my addr 172.21.15.73:0/3462505508 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb877fff640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4c8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 -- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 msgr2=0x5561ad4c8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4c8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 -- 172.21.15.73:0/3462505508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 msgr2=0x5561ad4c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 -- 172.21.15.73:0/3462505508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ad138210 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb877fff640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b745c385ad3df3c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb857fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c035070 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fb86c004240 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb870009bc0 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb86c04d050 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb881e2a740 10 client.16692 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.612+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5561ad204620 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb870009bc0 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fb86c0048f0 con 0x5561ad4c0170 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 10 client.16692 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:34.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 10 client.16692 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 20 client.16692 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad4db580 0x5561ad4dda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 4 client.16692 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561ad437f10 con 0x5561ad4db580 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.613+0000 7fb877fff640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad4db580 0x5561ad4dda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.614+0000 7fb877fff640 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad4db580 0x5561ad4dda10 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866a69e74ef62731 server_cookie=cbb8fd640d26cc5d in_seq=0 out_seq=0 2024-06-06T07:11:34.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.614+0000 7fb857fff640 10 client.16692 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.614+0000 7fb857fff640 1 -- 172.21.15.73:0/3462505508 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561ad437f10 con 0x5561ad4db580 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.614+0000 7fb857fff640 10 client.16692 handle_command_reply: tid=0 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 client.16692 shutdown 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 msgr2=0x5561ad4c83e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009930 tx=0x7fb86c031750 comp rx=0 tx=0).stop 2024-06-06T07:11:34.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 shutdown_connections 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad4db580 0x5561ad4dda10 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4cb350 0x5561ad4c8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4c0a90 0x5561ad4c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb857fff640 0 client.16692 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3462505508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4c83e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 >> 172.21.15.73:0/3462505508 conn(0x5561ad4bc680 msgr2=0x5561ad4c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 shutdown_connections 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 1 -- 172.21.15.73:0/3462505508 wait complete. 2024-06-06T07:11:34.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.615+0000 7fb881e2a740 20 client.16692 trim_cache size 0 max 16384 2024-06-06T07:11:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.630+0000 7fb881e2a740 1 Processor -- start 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.630+0000 7fb881e2a740 1 -- start start 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4ca320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4e3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781b1080 con 0x5561ad4c0170 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781b2d10 con 0x5561ad4cb350 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb881e2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781b1750 con 0x5561ad4c0a90 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47116/0 (socket says 172.21.15.73:47116) 2024-06-06T07:11:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 -- 172.21.15.73:0/1375114024 learned_addr learned my addr 172.21.15.73:0/1375114024 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb877fff640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4ca320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 -- 172.21.15.73:0/1375114024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 msgr2=0x5561ad4ca320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87d6be640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4e3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4ca320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 -- 172.21.15.73:0/1375114024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 msgr2=0x5561ad4e3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4e3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 -- 172.21.15.73:0/1375114024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561ad4c6930 con 0x5561ad4c0170 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb877fff640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb87cebd640 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb86c031440 tx=0x7fb86c032f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7148a7b96cd62491 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.631+0000 7fb8627fc640 1 -- 172.21.15.73:0/1375114024 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c035030 con 0x5561ad4c0170 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb8627fc640 1 -- 172.21.15.73:0/1375114024 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb86c004e40 con 0x5561ad4c0170 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb8627fc640 1 -- 172.21.15.73:0/1375114024 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c044690 con 0x5561ad4c0170 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- 172.21.15.73:0/1375114024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 msgr2=0x5561ad4bbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fb86c031440 tx=0x7fb86c032f30 comp rx=0 tx=0).stop 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- 172.21.15.73:0/1375114024 shutdown_connections 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4e3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4ca320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 --2- 172.21.15.73:0/1375114024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4bbf60 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- 172.21.15.73:0/1375114024 >> 172.21.15.73:0/1375114024 conn(0x5561ad4bc680 msgr2=0x5561ad4c9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- 172.21.15.73:0/1375114024 shutdown_connections 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- 172.21.15.73:0/1375114024 wait complete. 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.632+0000 7fb881e2a740 1 -- start start 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 Processor -- start 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4cfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781b25e0 con 0x5561ad4c0170 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781b1e80 con 0x5561ad4cb350 2024-06-06T07:11:34.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8781af080 con 0x5561ad4c0a90 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb881e2a740 10 client.0 _ll_register_callbacks cb 0x7fb864005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47128/0 (socket says 172.21.15.73:47128) 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 -- 172.21.15.73:0/3014687650 learned_addr learned my addr 172.21.15.73:0/3014687650 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb877fff640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 -- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 msgr2=0x5561ad4cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 -- 172.21.15.73:0/3014687650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 msgr2=0x5561ad4cfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87d6be640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4cfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4cfac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.633+0000 7fb87cebd640 1 -- 172.21.15.73:0/3014687650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ad5452f0 con 0x5561ad4c0170 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb87cebd640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006010 tx=0x7fb86c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740f8e0b8230b11e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c035070 con 0x5561ad4c0170 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fb86c004b10 con 0x5561ad4c0170 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86c004e40 con 0x5561ad4c0170 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb86c069050 con 0x5561ad4c0170 2024-06-06T07:11:34.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 10 client.16698 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5561ad546540 con 0x5561ad4c0170 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb86c004e40 con 0x5561ad4c0170 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fb86c044a50 con 0x5561ad4c0170 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 10 client.16698 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 20 client.16698 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad546ba0 0x5561ad548fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 4 client.16698 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:11:34.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.634+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5561ad431c20 con 0x5561ad546ba0 2024-06-06T07:11:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.635+0000 7fb877fff640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad546ba0 0x5561ad548fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.635+0000 7fb877fff640 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad546ba0 0x5561ad548fc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8f86d5a9805009 server_cookie=da0f6d9e6d913b7b in_seq=0 out_seq=0 2024-06-06T07:11:34.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.635+0000 7fb857fff640 10 client.16698 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.659+0000 7fb857fff640 1 -- 172.21.15.73:0/3014687650 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5561ad431c20 con 0x5561ad546ba0 2024-06-06T07:11:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.659+0000 7fb857fff640 10 client.16698 handle_command_reply: tid=0 2024-06-06T07:11:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.659+0000 7fb881e2a740 1 client.16698 shutdown 2024-06-06T07:11:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 msgr2=0x5561ad4cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb86c006010 tx=0x7fb86c005bd0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 shutdown_connections 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5561ad546ba0 0x5561ad548fc0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5561ad4cb350 0x5561ad4cfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5561ad4c0a90 0x5561ad4cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb857fff640 0 client.16698 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 --2- 172.21.15.73:0/3014687650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5561ad4c0170 0x5561ad4cf040 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 >> 172.21.15.73:0/3014687650 conn(0x5561ad4bc680 msgr2=0x5561ad4c23b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 shutdown_connections 2024-06-06T07:11:34.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 1 -- 172.21.15.73:0/3014687650 wait complete. 2024-06-06T07:11:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.660+0000 7fb881e2a740 20 client.16698 trim_cache size 0 max 16384 2024-06-06T07:11:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:11:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:11:34.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:11:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 msgr2=0x7fb8680a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 0x7fb8680a52e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb870011c50 tx=0x7fb870016000 comp rx=0 tx=0).stop 2024-06-06T07:11:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 msgr2=0x7fb8781029d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:34.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb86400e9f0 tx=0x7fb86400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 shutdown_connections 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8781068d0 0x7fb878102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb878101280 0x7fb8781029d0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8680a2e20 0x7fb8680a52e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 --2- 172.21.15.73:0/3049989884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8781008c0 0x7fb878102490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 >> 172.21.15.73:0/3049989884 conn(0x7fb878076c20 msgr2=0x7fb87806a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 shutdown_connections 2024-06-06T07:11:34.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:34.661+0000 7fb87f121640 1 -- 172.21.15.73:0/3049989884 wait complete. 2024-06-06T07:11:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:34 smithi073 ceph-mon[32303]: pgmap v320: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 622 KiB/s wr, 84 op/s 2024-06-06T07:11:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:34 smithi136 ceph-mon[35901]: pgmap v320: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 622 KiB/s wr, 84 op/s 2024-06-06T07:11:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:34 smithi170 ceph-mon[36108]: pgmap v320: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 622 KiB/s wr, 84 op/s 2024-06-06T07:11:35.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209425 2.03 MB/sec execute 402 sec latency 1392.205 ms 2024-06-06T07:11:36.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209425 2.02 MB/sec execute 403 sec latency 1658.292 ms 2024-06-06T07:11:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:36 smithi073 ceph-mon[32303]: pgmap v321: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 547 KiB/s wr, 57 op/s 2024-06-06T07:11:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:36 smithi136 ceph-mon[35901]: pgmap v321: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 547 KiB/s wr, 57 op/s 2024-06-06T07:11:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:36 smithi170 ceph-mon[36108]: pgmap v321: 129 pgs: 129 active+clean; 128 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 466 KiB/s rd, 547 KiB/s wr, 57 op/s 2024-06-06T07:11:37.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209425 2.02 MB/sec execute 404 sec latency 2658.390 ms 2024-06-06T07:11:38.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209515 2.02 MB/sec execute 405 sec latency 3360.656 ms 2024-06-06T07:11:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:38 smithi073 ceph-mon[32303]: pgmap v322: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 602 KiB/s wr, 72 op/s 2024-06-06T07:11:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:38 smithi136 ceph-mon[35901]: pgmap v322: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 602 KiB/s wr, 72 op/s 2024-06-06T07:11:39.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:38 smithi170 ceph-mon[36108]: pgmap v322: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 602 KiB/s wr, 72 op/s 2024-06-06T07:11:39.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 209780 2.01 MB/sec execute 406 sec latency 13.052 ms 2024-06-06T07:11:40.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 210218 2.01 MB/sec execute 407 sec latency 16.209 ms 2024-06-06T07:11:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:40 smithi073 ceph-mon[32303]: pgmap v323: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:40 smithi136 ceph-mon[35901]: pgmap v323: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:40 smithi170 ceph-mon[36108]: pgmap v323: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:41.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 210723 2.02 MB/sec execute 408 sec latency 15.697 ms 2024-06-06T07:11:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:11:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:41 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:11:42.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:41 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:41 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:42.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 210970 2.01 MB/sec execute 409 sec latency 420.244 ms 2024-06-06T07:11:43.277 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:42 smithi073 ceph-mon[32303]: pgmap v324: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:42 smithi136 ceph-mon[35901]: pgmap v324: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:43.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:42 smithi170 ceph-mon[36108]: pgmap v324: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 73 KiB/s wr, 25 op/s 2024-06-06T07:11:43.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 211141 2.01 MB/sec execute 410 sec latency 1119.578 ms 2024-06-06T07:11:44.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 211651 2.01 MB/sec execute 411 sec latency 19.233 ms 2024-06-06T07:11:44.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:11:44.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:11:44.803 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:11:44.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:11:45.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212259 2.01 MB/sec execute 412 sec latency 17.995 ms 2024-06-06T07:11:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: pgmap v325: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 531 KiB/s wr, 84 op/s 2024-06-06T07:11:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:11:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:11:45.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:11:45.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:11:45.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:11:45.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:45 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: pgmap v325: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 531 KiB/s wr, 84 op/s 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:11:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:11:45.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:45 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: pgmap v325: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 531 KiB/s wr, 84 op/s 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:11:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:45 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:11:46.451 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212672 2.01 MB/sec execute 413 sec latency 16.199 ms 2024-06-06T07:11:47.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212892 2.02 MB/sec execute 414 sec latency 700.267 ms 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 -- 172.21.15.73:0/1090684991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 msgr2=0x7fd844100b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1090684991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd844100b90 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fd834009900 tx=0x7fd83402a410 comp rx=0 tx=0).stop 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 -- 172.21.15.73:0/1090684991 shutdown_connections 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1090684991 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd84410e360 0x7fd844110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1090684991 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd8441010d0 0x7fd84410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1090684991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd844100b90 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 -- 172.21.15.73:0/1090684991 >> 172.21.15.73:0/1090684991 conn(0x7fd844077a20 msgr2=0x7fd844077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.753 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 -- 172.21.15.73:0/1090684991 shutdown_connections 2024-06-06T07:11:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 -- 172.21.15.73:0/1090684991 wait complete. 2024-06-06T07:11:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.752+0000 7fd8492ee640 1 Processor -- start 2024-06-06T07:11:47.754 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 -- start start 2024-06-06T07:11:47.755 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8441010d0 0x7fd84419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 0x7fd84419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd844113e40 con 0x7fd8441007b0 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd844113cc0 con 0x7fd84410e360 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8492ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd844113fc0 con 0x7fd8441010d0 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50694/0 (socket says 172.21.15.73:50694) 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 -- 172.21.15.73:0/1082782706 learned_addr learned my addr 172.21.15.73:0/1082782706 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8427fc640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8441010d0 0x7fd84419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.756 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 0x7fd84419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 -- 172.21.15.73:0/1082782706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8441010d0 msgr2=0x7fd84419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8441010d0 0x7fd84419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 -- 172.21.15.73:0/1082782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 msgr2=0x7fd84419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 0x7fd84419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 -- 172.21.15.73:0/1082782706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd84419f300 con 0x7fd8441007b0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 0x7fd84419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.753+0000 7fd842ffd640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd834005940 tx=0x7fd8340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.754+0000 7fd82bfff640 1 -- 172.21.15.73:0/1082782706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83403f030 con 0x7fd8441007b0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.754+0000 7fd82bfff640 1 -- 172.21.15.73:0/1082782706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8340044c0 con 0x7fd8441007b0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.754+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd84419f590 con 0x7fd8441007b0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.754+0000 7fd82bfff640 1 -- 172.21.15.73:0/1082782706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834035070 con 0x7fd8441007b0 2024-06-06T07:11:47.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.754+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441a6e80 con 0x7fd8441007b0 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.756+0000 7fd82bfff640 1 -- 172.21.15.73:0/1082782706 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd834042030 con 0x7fd8441007b0 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.756+0000 7fd84bffc740 1 Processor -- start 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.756+0000 7fd84bffc740 1 -- start start 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 0x55a0d8a10550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0d89744f0 con 0x55a0d8a10a90 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0d8912870 con 0x55a0d8a1b350 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd84bffc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a0d89704b0 con 0x55a0d8a10170 2024-06-06T07:11:47.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50704/0 (socket says 172.21.15.73:50704) 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd842ffd640 1 -- 172.21.15.73:0/1089059107 learned_addr learned my addr 172.21.15.73:0/1089059107 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 0x55a0d8a10550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd82bfff640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 0x7fd82c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8427fc640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 0x7fd82c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd82bfff640 1 -- 172.21.15.73:0/1082782706 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fd8340efc30 con 0x7fd8441007b0 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 -- 172.21.15.73:0/1089059107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a1b350 msgr2=0x55a0d8a1d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 -- 172.21.15.73:0/1089059107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 msgr2=0x55a0d8a1ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd8437fe640 1 -- 172.21.15.73:0/1089059107 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d892e9b0 con 0x55a0d8a10170 2024-06-06T07:11:47.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.757+0000 7fd842ffd640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 0x55a0d8a10550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c012e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a64d5bd5285d4258 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd8437fe640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 0x7fd82c0a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01c000 tx=0x7fd83c020000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd8297fa640 1 -- 172.21.15.73:0/1089059107 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c012280 con 0x55a0d8a10170 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd8297fa640 1 -- 172.21.15.73:0/1089059107 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83c023070 con 0x55a0d8a10170 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd8297fa640 1 -- 172.21.15.73:0/1089059107 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83c010af0 con 0x55a0d8a10170 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd84bffc740 1 -- 172.21.15.73:0/1089059107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 msgr2=0x55a0d8a10550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.758+0000 7fd84bffc740 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 0x55a0d8a10550 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd83c005fd0 tx=0x7fd83c012e80 comp rx=0 tx=0).stop 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- 172.21.15.73:0/1089059107 shutdown_connections 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a1ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- 172.21.15.73:0/1089059107 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a10170 0x55a0d8a10550 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- 172.21.15.73:0/1089059107 >> 172.21.15.73:0/1089059107 conn(0x55a0d8a0c680 msgr2=0x55a0d8a0ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- 172.21.15.73:0/1089059107 shutdown_connections 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- 172.21.15.73:0/1089059107 wait complete. 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- start start 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 Processor -- start 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a182a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a18d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a0d8a1df20 con 0x55a0d8a10a90 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a0d88fdae0 con 0x55a0d8a10170 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a0d8a1e0a0 con 0x55a0d8a1b350 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd84bffc740 10 client.0 _ll_register_callbacks cb 0x55a0d89369b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:47.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50710/0 (socket says 172.21.15.73:50710) 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd8427fc640 1 -- 172.21.15.73:0/2506874057 learned_addr learned my addr 172.21.15.73:0/2506874057 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd8437fe640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a18d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.759+0000 7fd842ffd640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a182a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 -- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 msgr2=0x55a0d8a18d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a18d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 -- 172.21.15.73:0/2506874057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 msgr2=0x55a0d8a182a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a182a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 -- 172.21.15.73:0/2506874057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d8688210 con 0x55a0d8a10a90 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd8427fc640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd838012700 tx=0x7fd838012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9c453e5a9ed7e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:47.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83800f920 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fd83800fac0 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd83c023070 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd83800fcc0 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd84bffc740 10 client.16710 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a0d8754620 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd83c023070 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.760+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fd83801c420 con 0x55a0d8a10a90 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 10 client.16710 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 10 client.16710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 20 client.16710 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a2b530 0x55a0d8a2d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 4 client.16710 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0d8987f10 con 0x55a0d8a2b530 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd842ffd640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a2b530 0x55a0d8a2d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd826ff5640 10 client.16710 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.761+0000 7fd842ffd640 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a2b530 0x55a0d8a2d980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7039b38cf790c91e server_cookie=962c458410614b7 in_seq=0 out_seq=0 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd826ff5640 1 -- 172.21.15.73:0/2506874057 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a0d8987f10 con 0x55a0d8a2b530 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd826ff5640 10 client.16710 handle_command_reply: tid=0 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 client.16710 shutdown 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 msgr2=0x55a0d8a187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fd838012700 tx=0x7fd838012bc0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 shutdown_connections 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a2b530 0x55a0d8a2d980 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a18d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a187e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 --2- 172.21.15.73:0/2506874057 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a182a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd826ff5640 0 client.16710 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 >> 172.21.15.73:0/2506874057 conn(0x55a0d8a0c680 msgr2=0x55a0d8a123b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 shutdown_connections 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 1 -- 172.21.15.73:0/2506874057 wait complete. 2024-06-06T07:11:47.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.762+0000 7fd84bffc740 20 client.16710 trim_cache size 0 max 16384 2024-06-06T07:11:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:47 smithi073 ceph-mon[32303]: pgmap v326: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 512 KiB/s wr, 73 op/s 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 Processor -- start 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 -- start start 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 0x55a0d8a332e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a33820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b1070 con 0x55a0d8a10170 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b2d00 con 0x55a0d8a10a90 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd84bffc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b1740 con 0x55a0d8a1b350 2024-06-06T07:11:47.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd842ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50722/0 (socket says 172.21.15.73:50722) 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd842ffd640 1 -- 172.21.15.73:0/590642032 learned_addr learned my addr 172.21.15.73:0/590642032 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd8427fc640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 0x55a0d8a332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.778+0000 7fd8437fe640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a33820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 -- 172.21.15.73:0/590642032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 msgr2=0x55a0d8a33820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a33820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 -- 172.21.15.73:0/590642032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 msgr2=0x55a0d8a332e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 0x55a0d8a332e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 -- 172.21.15.73:0/590642032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0d8a167f0 con 0x55a0d8a10170 2024-06-06T07:11:47.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd8427fc640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 0x55a0d8a332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:11:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd842ffd640 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd834031440 tx=0x7fd8340b80b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411d4881ce928352 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd828ff9640 1 -- 172.21.15.73:0/590642032 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834049040 con 0x55a0d8a10170 2024-06-06T07:11:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd828ff9640 1 -- 172.21.15.73:0/590642032 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8340bc3e0 con 0x55a0d8a10170 2024-06-06T07:11:47.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.779+0000 7fd828ff9640 1 -- 172.21.15.73:0/590642032 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8340b8350 con 0x55a0d8a10170 2024-06-06T07:11:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- 172.21.15.73:0/590642032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 msgr2=0x55a0d8a1a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.785 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fd834031440 tx=0x7fd8340b80b0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- 172.21.15.73:0/590642032 shutdown_connections 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a33820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10a90 0x55a0d8a332e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 --2- 172.21.15.73:0/590642032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10170 0x55a0d8a1a1e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- 172.21.15.73:0/590642032 >> 172.21.15.73:0/590642032 conn(0x55a0d8a0c680 msgr2=0x55a0d8a19ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- 172.21.15.73:0/590642032 shutdown_connections 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- 172.21.15.73:0/590642032 wait complete. 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 -- start start 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.780+0000 7fd84bffc740 1 Processor -- start 2024-06-06T07:11:47.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a236b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b25d0 con 0x55a0d8a10a90 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441b1e70 con 0x55a0d8a10170 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd8441af070 con 0x55a0d8a1b350 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd84bffc740 10 client.0 _ll_register_callbacks cb 0x7fd838009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50730/0 (socket says 172.21.15.73:50730) 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 -- 172.21.15.73:0/195080056 learned_addr learned my addr 172.21.15.73:0/195080056 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8437fe640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd842ffd640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a236b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 -- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 msgr2=0x55a0d8a1ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 -- 172.21.15.73:0/195080056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 msgr2=0x55a0d8a236b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a236b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 -- 172.21.15.73:0/195080056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0d8a1f3f0 con 0x55a0d8a10a90 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd842ffd640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd8427fc640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd838001eb0 tx=0x7fd838009fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca1e47b7eb999ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd826ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83801c630 con 0x55a0d8a10a90 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.781+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fd83801cc90 con 0x55a0d8a10a90 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8340bc3e0 con 0x55a0d8a10a90 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fd838005110 con 0x55a0d8a10a90 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 10 client.16716 fetch_fsmap learned FSMap version 47 2024-06-06T07:11:47.788 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a0d8a95f20 con 0x55a0d8a10a90 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8340bc3e0 con 0x55a0d8a10a90 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fd838010430 con 0x55a0d8a10a90 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 10 client.16716 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 20 client.16716 populate_metadata read hostname 'smithi073' 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a965b0 0x55a0d8a989d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 4 client.16716 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0d8981c20 con 0x55a0d8a965b0 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.782+0000 7fd842ffd640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a965b0 0x55a0d8a989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.783+0000 7fd842ffd640 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a965b0 0x55a0d8a989d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75beb7a2a11afd7e server_cookie=a2fef02312960780 in_seq=0 out_seq=0 2024-06-06T07:11:47.789 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.783+0000 7fd826ff5640 10 client.16716 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:47 smithi136 ceph-mon[35901]: pgmap v326: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 512 KiB/s wr, 73 op/s 2024-06-06T07:11:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd826ff5640 1 -- 172.21.15.73:0/195080056 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a0d8981c20 con 0x55a0d8a965b0 2024-06-06T07:11:47.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd826ff5640 10 client.16716 handle_command_reply: tid=0 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 client.16716 shutdown 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 msgr2=0x55a0d8a23bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fd838001eb0 tx=0x7fd838009fb0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 shutdown_connections 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a0d8a965b0 0x55a0d8a989d0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a0d8a1b350 0x55a0d8a1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a0d8a10a90 0x55a0d8a23bf0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd826ff5640 0 client.16716 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 --2- 172.21.15.73:0/195080056 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a0d8a10170 0x55a0d8a236b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 >> 172.21.15.73:0/195080056 conn(0x55a0d8a0c680 msgr2=0x55a0d8a1a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 shutdown_connections 2024-06-06T07:11:47.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 1 -- 172.21.15.73:0/195080056 wait complete. 2024-06-06T07:11:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.810+0000 7fd84bffc740 20 client.16716 trim_cache size 0 max 16384 2024-06-06T07:11:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:11:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:11:47.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 msgr2=0x7fd82c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 0x7fd82c0a5260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd83c01c000 tx=0x7fd83c020000 comp rx=0 tx=0).stop 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 msgr2=0x7fd84419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fd834005940 tx=0x7fd8340040c0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 shutdown_connections 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd84410e360 0x7fd84419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd82c0a2da0 0x7fd82c0a5260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd8441010d0 0x7fd84419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 --2- 172.21.15.73:0/1082782706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8441007b0 0x7fd84419a360 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:11:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 >> 172.21.15.73:0/1082782706 conn(0x7fd844077a20 msgr2=0x7fd84410f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:11:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.811+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 shutdown_connections 2024-06-06T07:11:47.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:11:47.812+0000 7fd8492ee640 1 -- 172.21.15.73:0/1082782706 wait complete. 2024-06-06T07:11:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:47 smithi170 ceph-mon[36108]: pgmap v326: 129 pgs: 129 active+clean; 129 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 512 KiB/s wr, 73 op/s 2024-06-06T07:11:48.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212892 2.01 MB/sec execute 415 sec latency 1700.344 ms 2024-06-06T07:11:49.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212892 2.01 MB/sec execute 416 sec latency 2700.444 ms 2024-06-06T07:11:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:49 smithi136 ceph-mon[35901]: pgmap v327: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:11:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:49 smithi170 ceph-mon[36108]: pgmap v327: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:11:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:49 smithi073 ceph-mon[32303]: pgmap v327: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 842 KiB/s rd, 1.0 MiB/s wr, 124 op/s 2024-06-06T07:11:50.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212959 2.00 MB/sec execute 417 sec latency 2987.063 ms 2024-06-06T07:11:51.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212959 2.00 MB/sec execute 418 sec latency 1446.821 ms 2024-06-06T07:11:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:51 smithi136 ceph-mon[35901]: pgmap v328: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:51 smithi170 ceph-mon[36108]: pgmap v328: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:52.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:51 smithi073 ceph-mon[32303]: pgmap v328: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:52.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212959 1.99 MB/sec execute 419 sec latency 2446.908 ms 2024-06-06T07:11:53.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212959 1.99 MB/sec execute 420 sec latency 3446.989 ms 2024-06-06T07:11:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:53 smithi136 ceph-mon[35901]: pgmap v329: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:53.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:53 smithi170 ceph-mon[36108]: pgmap v329: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:54.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:53 smithi073 ceph-mon[32303]: pgmap v329: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 799 KiB/s rd, 987 KiB/s wr, 110 op/s 2024-06-06T07:11:54.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 212959 1.98 MB/sec execute 421 sec latency 4447.099 ms 2024-06-06T07:11:55.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 213147 1.98 MB/sec execute 422 sec latency 4736.373 ms 2024-06-06T07:11:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:55 smithi136 ceph-mon[35901]: pgmap v330: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-06-06T07:11:55.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:55 smithi170 ceph-mon[36108]: pgmap v330: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-06-06T07:11:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:55 smithi073 ceph-mon[32303]: pgmap v330: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 838 KiB/s rd, 1.0 MiB/s wr, 129 op/s 2024-06-06T07:11:56.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 213431 1.98 MB/sec execute 423 sec latency 16.310 ms 2024-06-06T07:11:57.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 213912 1.98 MB/sec execute 424 sec latency 16.019 ms 2024-06-06T07:11:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:57 smithi073 ceph-mon[32303]: pgmap v331: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 580 KiB/s wr, 70 op/s 2024-06-06T07:11:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:57 smithi136 ceph-mon[35901]: pgmap v331: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 580 KiB/s wr, 70 op/s 2024-06-06T07:11:57.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:57 smithi170 ceph-mon[36108]: pgmap v331: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 580 KiB/s wr, 70 op/s 2024-06-06T07:11:58.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214429 1.98 MB/sec execute 425 sec latency 20.260 ms 2024-06-06T07:11:59.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214512 1.98 MB/sec execute 426 sec latency 829.010 ms 2024-06-06T07:11:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:11:59 smithi136 ceph-mon[35901]: pgmap v332: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1010 KiB/s wr, 116 op/s 2024-06-06T07:11:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:11:59 smithi170 ceph-mon[36108]: pgmap v332: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1010 KiB/s wr, 116 op/s 2024-06-06T07:12:00.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:11:59 smithi073 ceph-mon[32303]: pgmap v332: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 660 KiB/s rd, 1010 KiB/s wr, 116 op/s 2024-06-06T07:12:00.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214512 1.97 MB/sec execute 427 sec latency 1829.104 ms 2024-06-06T07:12:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1853508988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f8102370 msgr2=0x7fe7f810e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1853508988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f8102370 0x7fe7f810e880 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0009930 tx=0x7fe7f002a420 comp rx=0 tx=0).stop 2024-06-06T07:12:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1853508988 shutdown_connections 2024-06-06T07:12:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1853508988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f8102370 0x7fe7f810e880 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1853508988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 0x7fe7f8101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1853508988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f81079c0 0x7fe7f8107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1853508988 >> 172.21.15.73:0/1853508988 conn(0x7fe7f8078e30 msgr2=0x7fe7f8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1853508988 shutdown_connections 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1853508988 wait complete. 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 Processor -- start 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.910+0000 7fe7ffdb6640 1 -- start start 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 0x7fe7f81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f8102370 0x7fe7f8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8111c80 con 0x7fe7f81079c0 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8111b00 con 0x7fe7f8102370 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f8111e00 con 0x7fe7f81019b0 2024-06-06T07:12:00.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53150/0 (socket says 172.21.15.73:53150) 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 -- 172.21.15.73:0/1101823948 learned_addr learned my addr 172.21.15.73:0/1101823948 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 0x7fe7f81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f8102370 0x7fe7f8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 -- 172.21.15.73:0/1101823948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 msgr2=0x7fe7f81980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 0x7fe7f81980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 -- 172.21.15.73:0/1101823948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f8102370 msgr2=0x7fe7f8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f8102370 0x7fe7f8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 -- 172.21.15.73:0/1101823948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7f819d0d0 con 0x7fe7f81079c0 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7fe353640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005940 tx=0x7fe7f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7eeffd640 1 -- 172.21.15.73:0/1101823948 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f003f030 con 0x7fe7f81079c0 2024-06-06T07:12:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7eeffd640 1 -- 172.21.15.73:0/1101823948 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f0003cb0 con 0x7fe7f81079c0 2024-06-06T07:12:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7eeffd640 1 -- 172.21.15.73:0/1101823948 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f0035070 con 0x7fe7f81079c0 2024-06-06T07:12:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.911+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7f819d360 con 0x7fe7f81079c0 2024-06-06T07:12:00.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.912+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7f81a4c50 con 0x7fe7f81079c0 2024-06-06T07:12:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.914+0000 7fe7eeffd640 1 -- 172.21.15.73:0/1101823948 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe7f0042030 con 0x7fe7f81079c0 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.914+0000 7fe802b9c740 1 Processor -- start 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.914+0000 7fe802b9c740 1 -- start start 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f69550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f76740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5595c6ecd4f0 con 0x5595c6f69a90 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595c6e6b870 con 0x5595c6f69170 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe802b9c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5595c6ec94b0 con 0x5595c6f74350 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53154/0 (socket says 172.21.15.73:53154) 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fd351640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f69550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 -- 172.21.15.73:0/3180582699 learned_addr learned my addr 172.21.15.73:0/3180582699 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:00.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7eeffd640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 0x7fe7e40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fd351640 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f76740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fe353640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 0x7fe7e40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 -- 172.21.15.73:0/3180582699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 msgr2=0x5595c6f76740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f76740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 -- 172.21.15.73:0/3180582699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 msgr2=0x5595c6f69550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f69550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7eeffd640 1 -- 172.21.15.73:0/1101823948 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fe7f00f0070 con 0x7fe7f81079c0 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 -- 172.21.15.73:0/3180582699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c6e879b0 con 0x5595c6f69a90 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fd351640 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.915+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400aec0 tx=0x7fe7f400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a659a0e1f33edf27 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe7fe353640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 0x7fe7e40a5260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed460 tx=0x7fe7f008bfe0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe7e3fff640 1 -- 172.21.15.73:0/3180582699 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4010840 con 0x5595c6f69a90 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe7e3fff640 1 -- 172.21.15.73:0/3180582699 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f40183f0 con 0x5595c6f69a90 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe7e3fff640 1 -- 172.21.15.73:0/3180582699 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4017600 con 0x5595c6f69a90 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 -- 172.21.15.73:0/3180582699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 msgr2=0x5595c6f73e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fe7f400aec0 tx=0x7fe7f400f790 comp rx=0 tx=0).stop 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 -- 172.21.15.73:0/3180582699 shutdown_connections 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f76740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f73e10 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.916+0000 7fe802b9c740 1 --2- 172.21.15.73:0/3180582699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f69550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- 172.21.15.73:0/3180582699 >> 172.21.15.73:0/3180582699 conn(0x5595c6f65680 msgr2=0x5595c6f65a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- 172.21.15.73:0/3180582699 shutdown_connections 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- 172.21.15.73:0/3180582699 wait complete. 2024-06-06T07:12:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- start start 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 Processor -- start 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f71240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f71cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5595c6f76f80 con 0x5595c6f69a90 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5595c6e56ae0 con 0x5595c6f69170 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.917+0000 7fe802b9c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5595c6f77100 con 0x5595c6f74350 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe802b9c740 10 client.0 _ll_register_callbacks cb 0x5595c6e8f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53168/0 (socket says 172.21.15.73:53168) 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 -- 172.21.15.73:0/1099965729 learned_addr learned my addr 172.21.15.73:0/1099965729 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:00.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f71cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 -- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 msgr2=0x5595c6f71cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f71cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 -- 172.21.15.73:0/1099965729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 msgr2=0x5595c6f71240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f71240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 -- 172.21.15.73:0/1099965729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c6be1210 con 0x5595c6f69a90 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4018d70 tx=0x7fe7f4004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ada1369daf937d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f4026070 con 0x5595c6f69a90 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fe7f4010a50 con 0x5595c6f69a90 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.918+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f40183f0 con 0x5595c6f69a90 2024-06-06T07:12:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe7f4032050 con 0x5595c6f69a90 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 10 client.16728 fetch_fsmap learned FSMap version 47 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5595c6cad620 con 0x5595c6f69a90 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f40183f0 con 0x5595c6f69a90 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fe7f40054f0 con 0x5595c6f69a90 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 10 client.16728 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 20 client.16728 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6f843c0 0x5595c6f867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 4 client.16728 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595c6ee0f10 con 0x5595c6f843c0 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.919+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6f843c0 0x5595c6f867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.920+0000 7fe7fd351640 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6f843c0 0x5595c6f867b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf22e4ba3b24b0b9 server_cookie=ad6f93a43ad0587f in_seq=0 out_seq=0 2024-06-06T07:12:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.920+0000 7fe7e17fa640 10 client.16728 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe7e17fa640 1 -- 172.21.15.73:0/1099965729 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5595c6ee0f10 con 0x5595c6f843c0 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe7e17fa640 10 client.16728 handle_command_reply: tid=0 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 client.16728 shutdown 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 msgr2=0x5595c6f71780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4018d70 tx=0x7fe7f4004c70 comp rx=0 tx=0).stop 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 shutdown_connections 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6f843c0 0x5595c6f867b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f74350 0x5595c6f71cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f71780 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe7e17fa640 0 client.16728 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 --2- 172.21.15.73:0/1099965729 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f69170 0x5595c6f71240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 >> 172.21.15.73:0/1099965729 conn(0x5595c6f65680 msgr2=0x5595c6f6b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 shutdown_connections 2024-06-06T07:12:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 1 -- 172.21.15.73:0/1099965729 wait complete. 2024-06-06T07:12:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.921+0000 7fe802b9c740 20 client.16728 trim_cache size 0 max 16384 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 Processor -- start 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 -- start start 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69a90 0x5595c6f8bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f8c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81aeea0 con 0x5595c6f69170 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81b0b30 con 0x5595c6f74350 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.937+0000 7fe802b9c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81af570 con 0x5595c6f69a90 2024-06-06T07:12:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53176/0 (socket says 172.21.15.73:53176) 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 -- 172.21.15.73:0/772493720 learned_addr learned my addr 172.21.15.73:0/772493720 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69a90 0x5595c6f8bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f8c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 -- 172.21.15.73:0/772493720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69a90 msgr2=0x5595c6f8bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69a90 0x5595c6f8bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 -- 172.21.15.73:0/772493720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 msgr2=0x5595c6f8c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f8c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 -- 172.21.15.73:0/772493720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c6f698c0 con 0x5595c6f69170 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7fd351640 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5595c6f66810 tx=0x7fe7e800c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e214ec5810d514 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7e37fe640 1 -- 172.21.15.73:0/772493720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e801c040 con 0x5595c6f69170 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7e37fe640 1 -- 172.21.15.73:0/772493720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7e80047a0 con 0x5595c6f69170 2024-06-06T07:12:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.938+0000 7fe7e37fe640 1 -- 172.21.15.73:0/772493720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7e8022620 con 0x5595c6f69170 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- 172.21.15.73:0/772493720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 msgr2=0x5595c6f73180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x5595c6f66810 tx=0x7fe7e800c610 comp rx=0 tx=0).stop 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- 172.21.15.73:0/772493720 shutdown_connections 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f8c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69a90 0x5595c6f8bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 --2- 172.21.15.73:0/772493720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69170 0x5595c6f73180 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- 172.21.15.73:0/772493720 >> 172.21.15.73:0/772493720 conn(0x5595c6f65680 msgr2=0x5595c6f72a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- 172.21.15.73:0/772493720 shutdown_connections 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- 172.21.15.73:0/772493720 wait complete. 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.939+0000 7fe802b9c740 1 -- start start 2024-06-06T07:12:00.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 Processor -- start 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69170 0x5595c6f78190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f78c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81b0400 con 0x5595c6f69a90 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81afca0 con 0x5595c6f74350 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7f81acea0 con 0x5595c6f69170 2024-06-06T07:12:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe802b9c740 10 client.0 _ll_register_callbacks cb 0x7fe7e800c4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53190/0 (socket says 172.21.15.73:53190) 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 -- 172.21.15.73:0/2993145284 learned_addr learned my addr 172.21.15.73:0/2993145284 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fd351640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f78c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 -- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69170 msgr2=0x5595c6f78190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69170 0x5595c6f78190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 -- 172.21.15.73:0/2993145284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 msgr2=0x5595c6f78c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f78c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.940+0000 7fe7fdb52640 1 -- 172.21.15.73:0/2993145284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c6fedea0 con 0x5595c6f69a90 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7fdb52640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000f80 tx=0x7fe7f4004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f8ab552d8ea175 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f403b060 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 9554+0+0 (secure 0 0 0) 0x7fe7f4010850 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7e80047a0 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe7f4042050 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 10 client.16734 fetch_fsmap learned FSMap version 47 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5595c6fef190 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7e80047a0 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 47) ==== 8390+0+0 (secure 0 0 0) 0x7fe7f4005370 con 0x5595c6f69a90 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 10 client.16734 fetch_fsmap finished waiting for FSMap version 47 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 20 client.16734 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6fef940 0x5595c6ff1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 4 client.16734 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:12:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.941+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5595c6edac20 con 0x5595c6fef940 2024-06-06T07:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.942+0000 7fe7fd351640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6fef940 0x5595c6ff1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.942+0000 7fe7e17fa640 10 client.16734 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.942+0000 7fe7fd351640 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6fef940 0x5595c6ff1d60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7622e9648387a5f9 server_cookie=4aed97ddbd6591fb in_seq=0 out_seq=0 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7e17fa640 1 -- 172.21.15.73:0/2993145284 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5595c6edac20 con 0x5595c6fef940 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7e17fa640 10 client.16734 handle_command_reply: tid=0 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 client.16734 shutdown 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 msgr2=0x5595c6f786d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe7f4000f80 tx=0x7fe7f4004f10 comp rx=0 tx=0).stop 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 shutdown_connections 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5595c6fef940 0x5595c6ff1d60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5595c6f74350 0x5595c6f78c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5595c6f69a90 0x5595c6f786d0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7e17fa640 0 client.16734 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 --2- 172.21.15.73:0/2993145284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5595c6f69170 0x5595c6f78190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 >> 172.21.15.73:0/2993145284 conn(0x5595c6f65680 msgr2=0x5595c6f73300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 shutdown_connections 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 1 -- 172.21.15.73:0/2993145284 wait complete. 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe802b9c740 20 client.16734 trim_cache size 0 max 16384 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:12:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:12:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 msgr2=0x7fe7e40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 0x7fe7e40a5260 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe7f00ed460 tx=0x7fe7f008bfe0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 msgr2=0x7fe7f819c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fe7f0005940 tx=0x7fe7f00042f0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 shutdown_connections 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7f81079c0 0x7fe7f819c9a0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7f8102370 0x7fe7f8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7e40a2da0 0x7fe7e40a5260 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 --2- 172.21.15.73:0/1101823948 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7f81019b0 0x7fe7f81980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 >> 172.21.15.73:0/1101823948 conn(0x7fe7f8078e30 msgr2=0x7fe7f80ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 shutdown_connections 2024-06-06T07:12:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:00.968+0000 7fe7ffdb6640 1 -- 172.21.15.73:0/1101823948 wait complete. 2024-06-06T07:12:01.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214512 1.97 MB/sec execute 428 sec latency 2829.150 ms 2024-06-06T07:12:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:01 smithi136 ceph-mon[35901]: pgmap v333: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:01 smithi170 ceph-mon[36108]: pgmap v333: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:02.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:01 smithi073 ceph-mon[32303]: pgmap v333: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:02.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214512 1.96 MB/sec execute 429 sec latency 3829.208 ms 2024-06-06T07:12:03.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 214669 1.96 MB/sec execute 430 sec latency 4530.558 ms 2024-06-06T07:12:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:03 smithi136 ceph-mon[35901]: pgmap v334: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:03.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:03 smithi170 ceph-mon[36108]: pgmap v334: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:04.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:03 smithi073 ceph-mon[32303]: pgmap v334: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 481 KiB/s wr, 65 op/s 2024-06-06T07:12:04.452 INFO:tasks.workunit.client.0.smithi073.stdout: 1 215275 1.96 MB/sec execute 431 sec latency 19.976 ms 2024-06-06T07:12:05.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 215872 1.97 MB/sec execute 432 sec latency 28.610 ms 2024-06-06T07:12:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:05 smithi136 ceph-mon[35901]: pgmap v335: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 555 KiB/s wr, 94 op/s 2024-06-06T07:12:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:05 smithi136 ceph-mon[35901]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:12:05.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:05 smithi170 ceph-mon[36108]: pgmap v335: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 555 KiB/s wr, 94 op/s 2024-06-06T07:12:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:05 smithi170 ceph-mon[36108]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:12:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:05 smithi073 ceph-mon[32303]: pgmap v335: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 426 KiB/s rd, 555 KiB/s wr, 94 op/s 2024-06-06T07:12:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:05 smithi073 ceph-mon[32303]: osdmap e113: 12 total, 12 up, 12 in 2024-06-06T07:12:06.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216362 1.97 MB/sec execute 433 sec latency 16.632 ms 2024-06-06T07:12:07.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216491 1.97 MB/sec execute 434 sec latency 746.617 ms 2024-06-06T07:12:07.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:07 smithi136 ceph-mon[35901]: pgmap v337: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 605 KiB/s wr, 90 op/s 2024-06-06T07:12:07.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:07 smithi170 ceph-mon[36108]: pgmap v337: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 605 KiB/s wr, 90 op/s 2024-06-06T07:12:08.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:07 smithi073 ceph-mon[32303]: pgmap v337: 129 pgs: 129 active+clean; 130 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 465 KiB/s rd, 605 KiB/s wr, 90 op/s 2024-06-06T07:12:08.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216491 1.96 MB/sec execute 435 sec latency 1746.707 ms 2024-06-06T07:12:09.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216491 1.96 MB/sec execute 436 sec latency 2746.815 ms 2024-06-06T07:12:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:09 smithi170 ceph-mon[36108]: pgmap v338: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:10.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:09 smithi073 ceph-mon[32303]: pgmap v338: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:10.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:09 smithi136 ceph-mon[35901]: pgmap v338: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:10.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216558 1.95 MB/sec execute 437 sec latency 3032.870 ms 2024-06-06T07:12:11.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216558 1.95 MB/sec execute 438 sec latency 1457.282 ms 2024-06-06T07:12:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:11 smithi170 ceph-mon[36108]: pgmap v339: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:12.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:11 smithi073 ceph-mon[32303]: pgmap v339: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:11 smithi136 ceph-mon[35901]: pgmap v339: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:12.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216558 1.95 MB/sec execute 439 sec latency 2457.344 ms 2024-06-06T07:12:13.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216558 1.94 MB/sec execute 440 sec latency 3457.419 ms 2024-06-06T07:12:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:13 smithi136 ceph-mon[35901]: pgmap v340: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:13 smithi136 ceph-mon[35901]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:12:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:13 smithi170 ceph-mon[36108]: pgmap v340: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:13 smithi170 ceph-mon[36108]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:12:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:13 smithi073 ceph-mon[32303]: pgmap v340: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 792 KiB/s wr, 113 op/s 2024-06-06T07:12:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:13 smithi073 ceph-mon[32303]: osdmap e114: 12 total, 12 up, 12 in 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 -- 172.21.15.73:0/825125925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c100900 msgr2=0x7fdc2c100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 --2- 172.21.15.73:0/825125925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c100900 0x7fdc2c100d80 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009930 tx=0x7fdc1402a420 comp rx=0 tx=0).stop 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 -- 172.21.15.73:0/825125925 shutdown_connections 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 --2- 172.21.15.73:0/825125925 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 --2- 172.21.15.73:0/825125925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c100900 0x7fdc2c100d80 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 --2- 172.21.15.73:0/825125925 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c1068c0 0x7fdc2c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 -- 172.21.15.73:0/825125925 >> 172.21.15.73:0/825125925 conn(0x7fdc2c077ab0 msgr2=0x7fdc2c077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 -- 172.21.15.73:0/825125925 shutdown_connections 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 -- 172.21.15.73:0/825125925 wait complete. 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.065+0000 7fdc31672640 1 Processor -- start 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 -- start start 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 0x7fdc2c199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1068c0 0x7fdc2c1a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c110b80 con 0x7fdc2c1012c0 2024-06-06T07:12:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c110a00 con 0x7fdc2c100900 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc31672640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c110d00 con 0x7fdc2c1068c0 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56408/0 (socket says 172.21.15.73:56408) 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 -- 172.21.15.73:0/2312882824 learned_addr learned my addr 172.21.15.73:0/2312882824 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc2affd640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 0x7fdc2c199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1068c0 0x7fdc2c1a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 -- 172.21.15.73:0/2312882824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1068c0 msgr2=0x7fdc2c1a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1068c0 0x7fdc2c1a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 -- 172.21.15.73:0/2312882824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 msgr2=0x7fdc2c199000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 0x7fdc2c199000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc227fc640 1 -- 172.21.15.73:0/2312882824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc2c1a45c0 con 0x7fdc2c1012c0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.066+0000 7fdc2affd640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 0x7fdc2c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdc14004770 tx=0x7fdc140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc28ff9640 1 -- 172.21.15.73:0/2312882824 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc1403f070 con 0x7fdc2c1012c0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc28ff9640 1 -- 172.21.15.73:0/2312882824 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc1403b460 con 0x7fdc2c1012c0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc28ff9640 1 -- 172.21.15.73:0/2312882824 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14035070 con 0x7fdc2c1012c0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc2c1a4790 con 0x7fdc2c1012c0 2024-06-06T07:12:14.071 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.067+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc2c1a4bf0 con 0x7fdc2c1012c0 2024-06-06T07:12:14.077 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.069+0000 7fdc28ff9640 1 -- 172.21.15.73:0/2312882824 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdc1403b610 con 0x7fdc2c1012c0 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 Processor -- start 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 -- start start 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928070 0x560cd7928450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc28ff9640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 0x7fdc180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7937760 0x560cd7939b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560cd788c4f0 con 0x560cd7928990 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560cd782a870 con 0x560cd7928070 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc34380740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560cd78884b0 con 0x560cd7937760 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc2affd640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 0x7fdc180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.078 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56414/0 (socket says 172.21.15.73:56414) 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc227fc640 1 -- 172.21.15.73:0/3625372655 learned_addr learned my addr 172.21.15.73:0/3625372655 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.070+0000 7fdc28ff9640 1 -- 172.21.15.73:0/2312882824 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fdc140efca0 con 0x7fdc2c1012c0 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928070 0x560cd7928450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7937760 0x560cd7939b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 -- 172.21.15.73:0/3625372655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7937760 msgr2=0x560cd7939b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7937760 0x560cd7939b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 -- 172.21.15.73:0/3625372655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928070 msgr2=0x560cd7928450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928070 0x560cd7928450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 -- 172.21.15.73:0/3625372655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd78469b0 con 0x560cd7928990 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc2affd640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 0x7fdc180a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005d90 tx=0x7fdc1c005d20 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.079 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc227fc640 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdc140bbd40 tx=0x7fdc1402ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3d45db38758a5a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc217fa640 1 -- 172.21.15.73:0/3625372655 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14048040 con 0x560cd7928990 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc217fa640 1 -- 172.21.15.73:0/3625372655 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc140bc4b0 con 0x560cd7928990 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.071+0000 7fdc217fa640 1 -- 172.21.15.73:0/3625372655 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140b9400 con 0x560cd7928990 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- 172.21.15.73:0/3625372655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 msgr2=0x560cd7937220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fdc140bbd40 tx=0x7fdc1402ee70 comp rx=0 tx=0).stop 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- 172.21.15.73:0/3625372655 shutdown_connections 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7937760 0x560cd7939b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd7937220 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 --2- 172.21.15.73:0/3625372655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928070 0x560cd7928450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- 172.21.15.73:0/3625372655 >> 172.21.15.73:0/3625372655 conn(0x560cd7924680 msgr2=0x560cd7924a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- 172.21.15.73:0/3625372655 shutdown_connections 2024-06-06T07:12:14.080 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- 172.21.15.73:0/3625372655 wait complete. 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 -- start start 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.072+0000 7fdc34380740 1 Processor -- start 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79346c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd7934c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560cd793a150 con 0x560cd7937760 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560cd7815ae0 con 0x560cd7928990 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560cd793a2d0 con 0x560cd7928070 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 10 client.0 _ll_register_callbacks cb 0x560cd784e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:14.081 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56426/0 (socket says 172.21.15.73:56426) 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/1008663329 learned_addr learned my addr 172.21.15.73:0/1008663329 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2affd640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79346c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc227fc640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd7934c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 msgr2=0x560cd79346c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79346c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/1008663329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 msgr2=0x560cd7934c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd7934c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/1008663329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd75a0210 con 0x560cd7937760 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc227fc640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd7934c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdc24003e90 tx=0x7fdc24004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a317c6ea4fc8aac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.082 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc127fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24014a20 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fdc24014bc0 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.073+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc140bc4b0 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc24024050 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 10 client.16746 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560cd766c620 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc140bc4b0 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fdc24011450 con 0x560cd7937760 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 10 client.16746 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 10 client.16746 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:14.083 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 20 client.16746 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79476f0 0x560cd7949ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 4 client.16746 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cd789ff10 con 0x560cd79476f0 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.074+0000 7fdc2affd640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79476f0 0x560cd7949ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.075+0000 7fdc127fc640 10 client.16746 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.075+0000 7fdc2affd640 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79476f0 0x560cd7949ae0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddcb36d899d259b9 server_cookie=7ed21d352db09c4c in_seq=0 out_seq=0 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.075+0000 7fdc127fc640 1 -- 172.21.15.73:0/1008663329 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560cd789ff10 con 0x560cd79476f0 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.075+0000 7fdc127fc640 10 client.16746 handle_command_reply: tid=0 2024-06-06T07:12:14.084 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 client.16746 shutdown 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 msgr2=0x560cd7935140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fdc24003e90 tx=0x7fdc24004410 comp rx=0 tx=0).stop 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 shutdown_connections 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79476f0 0x560cd7949ae0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd7935140 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd7934c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc127fc640 0 client.16746 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 --2- 172.21.15.73:0/1008663329 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79346c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 >> 172.21.15.73:0/1008663329 conn(0x560cd7924680 msgr2=0x560cd792a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 shutdown_connections 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 1 -- 172.21.15.73:0/1008663329 wait complete. 2024-06-06T07:12:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.076+0000 7fdc34380740 20 client.16746 trim_cache size 0 max 16384 2024-06-06T07:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 Processor -- start 2024-06-06T07:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 -- start start 2024-06-06T07:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79292b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 0x560cd794f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1accf0 con 0x560cd7928990 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1ae950 con 0x560cd7937760 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc34380740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1ad3c0 con 0x560cd7928070 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56428/0 (socket says 172.21.15.73:56428) 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.091+0000 7fdc227fc640 1 -- 172.21.15.73:0/2398492914 learned_addr learned my addr 172.21.15.73:0/2398492914 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc2affd640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79292b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 0x560cd794f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 -- 172.21.15.73:0/2398492914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 msgr2=0x560cd79292b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79292b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 -- 172.21.15.73:0/2398492914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 msgr2=0x560cd794f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 0x560cd794f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 -- 172.21.15.73:0/2398492914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cd79287c0 con 0x560cd7928990 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 0x560cd794f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc227fc640 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdc140ed090 tx=0x7fdc140bb810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3474a332593abd34 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc20ff9640 1 -- 172.21.15.73:0/2398492914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14048070 con 0x560cd7928990 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc20ff9640 1 -- 172.21.15.73:0/2398492914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc140bc4b0 con 0x560cd7928990 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.092+0000 7fdc20ff9640 1 -- 172.21.15.73:0/2398492914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc140044e0 con 0x560cd7928990 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- 172.21.15.73:0/2398492914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 msgr2=0x560cd794f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fdc140ed090 tx=0x7fdc140bb810 comp rx=0 tx=0).stop 2024-06-06T07:12:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- 172.21.15.73:0/2398492914 shutdown_connections 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7937760 0x560cd794f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7928990 0x560cd794f3d0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- 172.21.15.73:0/2398492914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79292b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- 172.21.15.73:0/2398492914 >> 172.21.15.73:0/2398492914 conn(0x560cd7924680 msgr2=0x560cd7932f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- 172.21.15.73:0/2398492914 shutdown_connections 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- 172.21.15.73:0/2398492914 wait complete. 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- start start 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 Processor -- start 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd79aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1ae220 con 0x560cd7937760 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1adaf0 con 0x560cd7928990 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdc2c1aacf0 con 0x560cd7928070 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc34380740 10 client.0 _ll_register_callbacks cb 0x7fdc24003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc2b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56434/0 (socket says 172.21.15.73:56434) 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.093+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/3718140936 learned_addr learned my addr 172.21.15.73:0/3718140936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc227fc640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd79aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 msgr2=0x560cd79aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2affd640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79aa6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/3718140936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 msgr2=0x560cd79aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd79aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 -- 172.21.15.73:0/3718140936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cd7949e50 con 0x560cd7937760 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc227fc640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd79aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc2b7fe640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000f80 tx=0x7fdc24004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a0c3e0dab4d318c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc127fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc24009b50 con 0x560cd7937760 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fdc24011a90 con 0x560cd7937760 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc140bc4b0 con 0x560cd7937760 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdc24034050 con 0x560cd7937760 2024-06-06T07:12:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc34380740 10 client.16752 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560cd79b1c20 con 0x560cd7937760 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.094+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc140bc4b0 con 0x560cd7937760 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fdc24014cb0 con 0x560cd7937760 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 10 client.16752 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 10 client.16752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 20 client.16752 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79b2400 0x560cd79b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 4 client.16752 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560cd7899c20 con 0x560cd79b2400 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc2affd640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79b2400 0x560cd79b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc2affd640 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79b2400 0x560cd79b4820 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea67575823950a03 server_cookie=f58d12532b2bd4a6 in_seq=0 out_seq=0 2024-06-06T07:12:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.095+0000 7fdc127fc640 10 client.16752 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc127fc640 1 -- 172.21.15.73:0/3718140936 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560cd7899c20 con 0x560cd79b2400 2024-06-06T07:12:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc127fc640 10 client.16752 handle_command_reply: tid=0 2024-06-06T07:12:14.119 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 client.16752 shutdown 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 msgr2=0x560cd793aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fdc24000f80 tx=0x7fdc24004980 comp rx=0 tx=0).stop 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 shutdown_connections 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560cd79b2400 0x560cd79b4820 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560cd7937760 0x560cd793aa10 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560cd7928990 0x560cd79aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc127fc640 0 client.16752 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.118+0000 7fdc34380740 1 --2- 172.21.15.73:0/3718140936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560cd7928070 0x560cd79aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 >> 172.21.15.73:0/3718140936 conn(0x560cd7924680 msgr2=0x560cd7929b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 shutdown_connections 2024-06-06T07:12:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc34380740 1 -- 172.21.15.73:0/3718140936 wait complete. 2024-06-06T07:12:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc34380740 20 client.16752 trim_cache size 0 max 16384 2024-06-06T07:12:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:12:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:12:14.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 msgr2=0x7fdc180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 0x7fdc180a5260 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c005d90 tx=0x7fdc1c005d20 comp rx=0 tx=0).stop 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 msgr2=0x7fdc2c199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fdc14004770 tx=0x7fdc140042f0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 shutdown_connections 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdc180a2da0 0x7fdc180a5260 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdc2c1068c0 0x7fdc2c1a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc2c1012c0 0x7fdc2c199540 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 --2- 172.21.15.73:0/2312882824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdc2c100900 0x7fdc2c199000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 >> 172.21.15.73:0/2312882824 conn(0x7fdc2c077ab0 msgr2=0x7fdc2c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.119+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 shutdown_connections 2024-06-06T07:12:14.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:14.120+0000 7fdc31672640 1 -- 172.21.15.73:0/2312882824 wait complete. 2024-06-06T07:12:14.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216558 1.94 MB/sec execute 441 sec latency 4457.460 ms 2024-06-06T07:12:14.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:14 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:12:15.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:14 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:12:15.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:14 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:12:15.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 216761 1.94 MB/sec execute 442 sec latency 4742.551 ms 2024-06-06T07:12:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:15 smithi170 ceph-mon[36108]: pgmap v342: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 801 KiB/s wr, 101 op/s 2024-06-06T07:12:15.924 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:15 smithi073 ceph-mon[32303]: pgmap v342: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 801 KiB/s wr, 101 op/s 2024-06-06T07:12:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:15 smithi136 ceph-mon[35901]: pgmap v342: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 649 KiB/s rd, 801 KiB/s wr, 101 op/s 2024-06-06T07:12:16.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 217049 1.93 MB/sec execute 443 sec latency 18.369 ms 2024-06-06T07:12:17.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 217587 1.93 MB/sec execute 444 sec latency 10.953 ms 2024-06-06T07:12:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:17 smithi170 ceph-mon[36108]: pgmap v343: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 746 KiB/s wr, 94 op/s 2024-06-06T07:12:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:17 smithi073 ceph-mon[32303]: pgmap v343: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 746 KiB/s wr, 94 op/s 2024-06-06T07:12:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:17 smithi136 ceph-mon[35901]: pgmap v343: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 746 KiB/s wr, 94 op/s 2024-06-06T07:12:18.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218083 1.93 MB/sec execute 445 sec latency 19.779 ms 2024-06-06T07:12:19.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:18 smithi073 ceph-mon[32303]: pgmap v344: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:19.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:18 smithi136 ceph-mon[35901]: pgmap v344: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:19.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:18 smithi170 ceph-mon[36108]: pgmap v344: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:19.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218114 1.93 MB/sec execute 446 sec latency 966.042 ms 2024-06-06T07:12:20.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218114 1.93 MB/sec execute 447 sec latency 1966.144 ms 2024-06-06T07:12:21.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:20 smithi073 ceph-mon[32303]: pgmap v345: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:21.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:20 smithi136 ceph-mon[35901]: pgmap v345: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:20 smithi170 ceph-mon[36108]: pgmap v345: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:21.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218114 1.92 MB/sec execute 448 sec latency 2966.194 ms 2024-06-06T07:12:22.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218114 1.92 MB/sec execute 449 sec latency 3966.226 ms 2024-06-06T07:12:23.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:22 smithi073 ceph-mon[32303]: pgmap v346: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:22 smithi136 ceph-mon[35901]: pgmap v346: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:22 smithi170 ceph-mon[36108]: pgmap v346: 129 pgs: 129 active+clean; 131 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 601 KiB/s wr, 81 op/s 2024-06-06T07:12:23.453 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218290 1.91 MB/sec execute 450 sec latency 4668.360 ms 2024-06-06T07:12:24.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 218853 1.91 MB/sec execute 451 sec latency 18.004 ms 2024-06-06T07:12:25.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:24 smithi073 ceph-mon[32303]: pgmap v347: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-06-06T07:12:25.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:24 smithi136 ceph-mon[35901]: pgmap v347: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-06-06T07:12:25.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:24 smithi170 ceph-mon[36108]: pgmap v347: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 446 KiB/s rd, 575 KiB/s wr, 92 op/s 2024-06-06T07:12:25.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 219591 1.92 MB/sec execute 452 sec latency 14.399 ms 2024-06-06T07:12:26.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 219972 1.93 MB/sec execute 453 sec latency 15.346 ms 2024-06-06T07:12:27.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- 172.21.15.73:0/3709877966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 msgr2=0x7fa32410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/3709877966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa32410e880 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009cd0 tx=0x7fa31c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- 172.21.15.73:0/3709877966 shutdown_connections 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/3709877966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa32410e880 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/3709877966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 0x7fa324101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/3709877966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa324107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- 172.21.15.73:0/3709877966 >> 172.21.15.73:0/3709877966 conn(0x7fa3240fd4a0 msgr2=0x7fa3240ff8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- 172.21.15.73:0/3709877966 shutdown_connections 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- 172.21.15.73:0/3709877966 wait complete. 2024-06-06T07:12:27.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 Processor -- start 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.209+0000 7fa32bc9a640 1 -- start start 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 0x7fa324197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa32419c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111c50 con 0x7fa324102370 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111ad0 con 0x7fa3241079c0 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa324111dd0 con 0x7fa3241019b0 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52776/0 (socket says 172.21.15.73:52776) 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 -- 172.21.15.73:0/2042318167 learned_addr learned my addr 172.21.15.73:0/2042318167 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329a36640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 0x7fa324197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32a237640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa32419c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 -- 172.21.15.73:0/2042318167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 msgr2=0x7fa324197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 0x7fa324197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 -- 172.21.15.73:0/2042318167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 msgr2=0x7fa32419c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa32419c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 -- 172.21.15.73:0/2042318167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa32419cfe0 con 0x7fa324102370 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32a237640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa32419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa329235640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa32000d950 tx=0x7fa32000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa316ffd640 1 -- 172.21.15.73:0/2042318167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa320017030 con 0x7fa324102370 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa316ffd640 1 -- 172.21.15.73:0/2042318167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa320004020 con 0x7fa324102370 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32419d2d0 con 0x7fa324102370 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa316ffd640 1 -- 172.21.15.73:0/2042318167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200047a0 con 0x7fa324102370 2024-06-06T07:12:27.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.210+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3241a4b60 con 0x7fa324102370 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.212+0000 7fa316ffd640 1 -- 172.21.15.73:0/2042318167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fa3200041d0 con 0x7fa324102370 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.213+0000 7fa32e9a7740 1 Processor -- start 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.213+0000 7fa32e9a7740 1 -- start start 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa316ffd640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 0x7fa3100a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e91db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56204e8704f0 con 0x56204e90c050 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56204e80e870 con 0x56204e90c970 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56204e86c4b0 con 0x56204e91b760 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32a237640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 0x7fa3100a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52782/0 (socket says 172.21.15.73:52782) 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa316ffd640 1 -- 172.21.15.73:0/2042318167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fa3200cf2d0 con 0x7fa324102370 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 -- 172.21.15.73:0/3213443000 learned_addr learned my addr 172.21.15.73:0/3213443000 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329235640 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32a237640 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e91db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 -- 172.21.15.73:0/3213443000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 msgr2=0x56204e91db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e91db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 -- 172.21.15.73:0/3213443000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 msgr2=0x56204e91b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 -- 172.21.15.73:0/3213443000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56204e82a9b0 con 0x56204e90c050 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa32a237640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 0x7fa3100a52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006010 tx=0x7fa31c031470 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa329a36640 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa31800cc10 tx=0x7fa31800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa829f9ec174540f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa30ffff640 1 -- 172.21.15.73:0/3213443000 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318005220 con 0x56204e90c050 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa30ffff640 1 -- 172.21.15.73:0/3213443000 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa318004c50 con 0x56204e90c050 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.214+0000 7fa30ffff640 1 -- 172.21.15.73:0/3213443000 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318010690 con 0x56204e90c050 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- 172.21.15.73:0/3213443000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 msgr2=0x56204e90c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fa31800cc10 tx=0x7fa31800e720 comp rx=0 tx=0).stop 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- 172.21.15.73:0/3213443000 shutdown_connections 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e91db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/3213443000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c050 0x56204e90c430 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- 172.21.15.73:0/3213443000 >> 172.21.15.73:0/3213443000 conn(0x56204e908680 msgr2=0x56204e908a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- 172.21.15.73:0/3213443000 shutdown_connections 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- 172.21.15.73:0/3213443000 wait complete. 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.215+0000 7fa32e9a7740 1 -- start start 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 Processor -- start 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e918670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 0x56204e9190f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56204e91e390 con 0x56204e90c970 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56204e7f9ae0 con 0x56204e91b760 2024-06-06T07:12:27.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56204e91e510 con 0x56204e90c050 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32e9a7740 10 client.0 _ll_register_callbacks cb 0x56204e8329b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52786/0 (socket says 172.21.15.73:52786) 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 -- 172.21.15.73:0/222389538 learned_addr learned my addr 172.21.15.73:0/222389538 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32a237640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 0x56204e9190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 -- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 msgr2=0x56204e918670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329a36640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e918670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e918670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 -- 172.21.15.73:0/222389538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 msgr2=0x56204e9190f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 0x56204e9190f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329235640 1 -- 172.21.15.73:0/222389538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56204e584210 con 0x56204e90c970 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa32a237640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 0x56204e9190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.216+0000 7fa329a36640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e918670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa329235640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ccc10 tx=0x7fa32009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e775599ef4cc0484 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200adcf0 con 0x56204e90c970 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fa320097a50 con 0x56204e90c970 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa318004c50 con 0x56204e90c970 2024-06-06T07:12:27.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa32009b3e0 con 0x56204e90c970 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 10 client.16764 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56204e650620 con 0x56204e90c970 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa318004c50 con 0x56204e90c970 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa3200a6050 con 0x56204e90c970 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 10 client.16764 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 10 client.16764 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 20 client.16764 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e92b940 0x56204e92ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 4 client.16764 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.217+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56204e883f10 con 0x56204e92b940 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.218+0000 7fa329a36640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e92b940 0x56204e92ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.218+0000 7fa329a36640 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e92b940 0x56204e92ddd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27dcc93480c8b2dd server_cookie=e411827462d4cd9b in_seq=0 out_seq=0 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.218+0000 7fa30d7fa640 10 client.16764 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa30d7fa640 1 -- 172.21.15.73:0/222389538 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56204e883f10 con 0x56204e92b940 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa30d7fa640 10 client.16764 handle_command_reply: tid=0 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 client.16764 shutdown 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 msgr2=0x56204e918bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fa3200ccc10 tx=0x7fa32009ae70 comp rx=0 tx=0).stop 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 shutdown_connections 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e92b940 0x56204e92ddd0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e91b760 0x56204e9190f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e918bb0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa30d7fa640 0 client.16764 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:27.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/222389538 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e918670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 >> 172.21.15.73:0/222389538 conn(0x56204e908680 msgr2=0x56204e90e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 shutdown_connections 2024-06-06T07:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 1 -- 172.21.15.73:0/222389538 wait complete. 2024-06-06T07:12:27.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.219+0000 7fa32e9a7740 20 client.16764 trim_cache size 0 max 16384 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 Processor -- start 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 -- start start 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c050 0x56204e933690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e906f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241acbe0 con 0x56204e90c970 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ae840 con 0x56204e90c050 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ad2b0 con 0x56204e91b760 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa329235640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52798/0 (socket says 172.21.15.73:52798) 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.234+0000 7fa329235640 1 -- 172.21.15.73:0/4154994482 learned_addr learned my addr 172.21.15.73:0/4154994482 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa32a237640 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e906f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 -- 172.21.15.73:0/4154994482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 msgr2=0x56204e906f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e906f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 -- 172.21.15.73:0/4154994482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c050 msgr2=0x56204e933690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c050 0x56204e933690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 -- 172.21.15.73:0/4154994482 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56204e903f70 con 0x56204e90c970 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa32a237640 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e906f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa329235640 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa320097db0 tx=0x7fa3200b2560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d2e3d5f7924ce server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa30f7fe640 1 -- 172.21.15.73:0/4154994482 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa32009f790 con 0x56204e90c970 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa30f7fe640 1 -- 172.21.15.73:0/4154994482 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa320097a50 con 0x56204e90c970 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa30f7fe640 1 -- 172.21.15.73:0/4154994482 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b22a0 con 0x56204e90c970 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa32e9a7740 1 -- 172.21.15.73:0/4154994482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 msgr2=0x56204e933bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.235+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa320097db0 tx=0x7fa3200b2560 comp rx=0 tx=0).stop 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- 172.21.15.73:0/4154994482 shutdown_connections 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e91b760 0x56204e906f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e90c970 0x56204e933bd0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/4154994482 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c050 0x56204e933690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- 172.21.15.73:0/4154994482 >> 172.21.15.73:0/4154994482 conn(0x56204e908680 msgr2=0x56204e913900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- 172.21.15.73:0/4154994482 shutdown_connections 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- 172.21.15.73:0/4154994482 wait complete. 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- start start 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 Processor -- start 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e923d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ae110 con 0x56204e91b760 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241ad9e0 con 0x56204e90c970 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fa3241aabe0 con 0x56204e90c050 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32e9a7740 10 client.0 _ll_register_callbacks cb 0x7fa32009afb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32a237640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32a237640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52808/0 (socket says 172.21.15.73:52808) 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa32a237640 1 -- 172.21.15.73:0/478741367 learned_addr learned my addr 172.21.15.73:0/478741367 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.236+0000 7fa329a36640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e923d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 -- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 msgr2=0x56204e923d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e923d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa329235640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 -- 172.21.15.73:0/478741367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 msgr2=0x56204e91f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 -- 172.21.15.73:0/478741367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56204e9133f0 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32a237640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa31c035000 tx=0x7fa31c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5be20bcf3b63fd54 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31c04e070 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7fa31c0305a0 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa320097a50 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fa31c044070 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 10 client.16770 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56204e9961f0 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa320097a50 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fa31c030070 con 0x56204e91b760 2024-06-06T07:12:27.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 10 client.16770 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 10 client.16770 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 20 client.16770 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e996a00 0x56204e998e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 4 client.16770 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.237+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56204e87dc20 con 0x56204e996a00 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.238+0000 7fa329a36640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e996a00 0x56204e998e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.238+0000 7fa30d7fa640 10 client.16770 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:27.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.238+0000 7fa329a36640 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e996a00 0x56204e998e20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81d02f470e615039 server_cookie=cdec7f2a8b72e580 in_seq=0 out_seq=0 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.263+0000 7fa30d7fa640 1 -- 172.21.15.73:0/478741367 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56204e87dc20 con 0x56204e996a00 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa30d7fa640 10 client.16770 handle_command_reply: tid=0 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 client.16770 shutdown 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 msgr2=0x56204e91f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa31c035000 tx=0x7fa31c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 shutdown_connections 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56204e996a00 0x56204e998e20 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56204e91b760 0x56204e91f5d0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa30d7fa640 0 client.16770 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56204e90c970 0x56204e91f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 --2- 172.21.15.73:0/478741367 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56204e90c050 0x56204e923d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 >> 172.21.15.73:0/478741367 conn(0x56204e908680 msgr2=0x56204e906f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 shutdown_connections 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 1 -- 172.21.15.73:0/478741367 wait complete. 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32e9a7740 20 client.16770 trim_cache size 0 max 16384 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:12:27.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 msgr2=0x7fa3100a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 0x7fa3100a52e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa31c006010 tx=0x7fa31c031470 comp rx=0 tx=0).stop 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 msgr2=0x7fa324198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.264+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fa32000d950 tx=0x7fa32000de10 comp rx=0 tx=0).stop 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 shutdown_connections 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fa3241079c0 0x7fa32419c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa324102370 0x7fa324198530 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fa3100a2e20 0x7fa3100a52e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 --2- 172.21.15.73:0/2042318167 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fa3241019b0 0x7fa324197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 >> 172.21.15.73:0/2042318167 conn(0x7fa3240fd4a0 msgr2=0x7fa3240fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 shutdown_connections 2024-06-06T07:12:27.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:27.265+0000 7fa32bc9a640 1 -- 172.21.15.73:0/2042318167 wait complete. 2024-06-06T07:12:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:26 smithi073 ceph-mon[32303]: pgmap v348: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-06-06T07:12:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:26 smithi136 ceph-mon[35901]: pgmap v348: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-06-06T07:12:27.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:26 smithi170 ceph-mon[36108]: pgmap v348: 129 pgs: 129 active+clean; 128 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 506 KiB/s wr, 74 op/s 2024-06-06T07:12:27.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220037 1.92 MB/sec execute 454 sec latency 903.717 ms 2024-06-06T07:12:28.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220037 1.92 MB/sec execute 455 sec latency 1903.781 ms 2024-06-06T07:12:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:28 smithi073 ceph-mon[32303]: pgmap v349: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-06T07:12:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:28 smithi136 ceph-mon[35901]: pgmap v349: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-06T07:12:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:28 smithi170 ceph-mon[36108]: pgmap v349: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-06T07:12:29.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220037 1.91 MB/sec execute 456 sec latency 2903.840 ms 2024-06-06T07:12:30.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220104 1.91 MB/sec execute 457 sec latency 3190.317 ms 2024-06-06T07:12:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:30 smithi073 ceph-mon[32303]: pgmap v350: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:30 smithi136 ceph-mon[35901]: pgmap v350: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:30 smithi170 ceph-mon[36108]: pgmap v350: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:31.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220104 1.91 MB/sec execute 458 sec latency 1476.153 ms 2024-06-06T07:12:32.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220104 1.90 MB/sec execute 459 sec latency 2476.236 ms 2024-06-06T07:12:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:32 smithi073 ceph-mon[32303]: pgmap v351: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:32 smithi136 ceph-mon[35901]: pgmap v351: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:32 smithi170 ceph-mon[36108]: pgmap v351: 129 pgs: 129 active+clean; 132 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:33.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220104 1.90 MB/sec execute 460 sec latency 3476.269 ms 2024-06-06T07:12:34.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220104 1.89 MB/sec execute 461 sec latency 4476.376 ms 2024-06-06T07:12:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:34 smithi073 ceph-mon[32303]: pgmap v352: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 623 KiB/s wr, 86 op/s 2024-06-06T07:12:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:34 smithi136 ceph-mon[35901]: pgmap v352: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 623 KiB/s wr, 86 op/s 2024-06-06T07:12:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:34 smithi170 ceph-mon[36108]: pgmap v352: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 623 KiB/s wr, 86 op/s 2024-06-06T07:12:35.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220293 1.89 MB/sec execute 462 sec latency 4762.958 ms 2024-06-06T07:12:36.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 220575 1.89 MB/sec execute 463 sec latency 15.032 ms 2024-06-06T07:12:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:36 smithi073 ceph-mon[32303]: pgmap v353: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 582 KiB/s wr, 66 op/s 2024-06-06T07:12:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:36 smithi136 ceph-mon[35901]: pgmap v353: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 582 KiB/s wr, 66 op/s 2024-06-06T07:12:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:36 smithi170 ceph-mon[36108]: pgmap v353: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 478 KiB/s rd, 582 KiB/s wr, 66 op/s 2024-06-06T07:12:37.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221058 1.89 MB/sec execute 464 sec latency 16.735 ms 2024-06-06T07:12:38.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221560 1.89 MB/sec execute 465 sec latency 20.741 ms 2024-06-06T07:12:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:38 smithi073 ceph-mon[32303]: pgmap v354: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1010 KiB/s wr, 113 op/s 2024-06-06T07:12:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:38 smithi136 ceph-mon[35901]: pgmap v354: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1010 KiB/s wr, 113 op/s 2024-06-06T07:12:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:38 smithi170 ceph-mon[36108]: pgmap v354: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 1010 KiB/s wr, 113 op/s 2024-06-06T07:12:39.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221657 1.89 MB/sec execute 466 sec latency 808.177 ms 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 -- 172.21.15.73:0/3513128332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 msgr2=0x7f11b0106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 --2- 172.21.15.73:0/3513128332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b0106c70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f11a0009930 tx=0x7f11a002a420 comp rx=0 tx=0).stop 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 -- 172.21.15.73:0/3513128332 shutdown_connections 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 --2- 172.21.15.73:0/3513128332 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b0101290 0x7f11b010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 --2- 172.21.15.73:0/3513128332 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b01008d0 0x7f11b0100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 --2- 172.21.15.73:0/3513128332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b0106c70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 -- 172.21.15.73:0/3513128332 >> 172.21.15.73:0/3513128332 conn(0x7f11b0077a20 msgr2=0x7f11b0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 -- 172.21.15.73:0/3513128332 shutdown_connections 2024-06-06T07:12:40.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.359+0000 7f11b4858640 1 -- 172.21.15.73:0/3513128332 wait complete. 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 Processor -- start 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 -- start start 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 0x7f11b0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b0101290 0x7f11b0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11b0110b10 con 0x7f11b0106890 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11b0110990 con 0x7f11b01008d0 2024-06-06T07:12:40.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11b4858640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11b0110c90 con 0x7f11b0101290 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11aed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11aed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54452/0 (socket says 172.21.15.73:54452) 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11aed9d640 1 -- 172.21.15.73:0/4132021410 learned_addr learned my addr 172.21.15.73:0/4132021410 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.360+0000 7f11add9b640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b0101290 0x7f11b0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11ae59c640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 0x7f11b0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 -- 172.21.15.73:0/4132021410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b0101290 msgr2=0x7f11b0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b0101290 0x7f11b0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 -- 172.21.15.73:0/4132021410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 msgr2=0x7f11b0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 0x7f11b0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 -- 172.21.15.73:0/4132021410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11b019d150 con 0x7f11b0106890 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11ae59c640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 0x7f11b0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11aed9d640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11a800eaa0 tx=0x7f11a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f119f7fe640 1 -- 172.21.15.73:0/4132021410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a800cdb0 con 0x7f11b0106890 2024-06-06T07:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f119f7fe640 1 -- 172.21.15.73:0/4132021410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a8005020 con 0x7f11b0106890 2024-06-06T07:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11b019d440 con 0x7f11b0106890 2024-06-06T07:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f119f7fe640 1 -- 172.21.15.73:0/4132021410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a80116d0 con 0x7f11b0106890 2024-06-06T07:12:40.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.361+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11b01a4cd0 con 0x7f11b0106890 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.363+0000 7f119f7fe640 1 -- 172.21.15.73:0/4132021410 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f11a80051d0 con 0x7f11b0106890 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.363+0000 7f11b7564740 1 Processor -- start 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 -- start start 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5591550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a559be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5598a54f54f0 con 0x5598a559c350 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5598a5493870 con 0x5598a5591a90 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11b7564740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5598a54f14b0 con 0x5598a5591170 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f119f7fe640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 0x7f11980a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54454/0 (socket says 172.21.15.73:54454) 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11add9b640 1 -- 172.21.15.73:0/2845948412 learned_addr learned my addr 172.21.15.73:0/2845948412 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11aed9d640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 0x7f11980a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11add9b640 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5591550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f11ae59c640 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a559be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.364+0000 7f119f7fe640 1 -- 172.21.15.73:0/4132021410 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f11a80cfaa0 con 0x7f11b0106890 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 -- 172.21.15.73:0/2845948412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 msgr2=0x5598a5591550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5591550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 -- 172.21.15.73:0/2845948412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 msgr2=0x5598a559be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a559be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 -- 172.21.15.73:0/2845948412 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a54af9b0 con 0x5598a559c350 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11add9b640 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f11a40149b0 tx=0x7f11a400c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8bf8e3ccd2faca server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f11aed9d640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 0x7f11980a5260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11a8098580 tx=0x7f11a800cf50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f119cff9640 1 -- 172.21.15.73:0/2845948412 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a40142d0 con 0x5598a559c350 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f119cff9640 1 -- 172.21.15.73:0/2845948412 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a40054d0 con 0x5598a559c350 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.365+0000 7f119cff9640 1 -- 172.21.15.73:0/2845948412 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a400f760 con 0x5598a559c350 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- 172.21.15.73:0/2845948412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 msgr2=0x5598a559e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f11a40149b0 tx=0x7f11a400c4b0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- 172.21.15.73:0/2845948412 shutdown_connections 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559e740 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a559be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- 172.21.15.73:0/2845948412 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5591550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- 172.21.15.73:0/2845948412 >> 172.21.15.73:0/2845948412 conn(0x5598a558d680 msgr2=0x5598a558da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- 172.21.15.73:0/2845948412 shutdown_connections 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- 172.21.15.73:0/2845948412 wait complete. 2024-06-06T07:12:40.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- start start 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 Processor -- start 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591170 0x5598a5599320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591a90 0x5598a5599860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5598a559ef80 con 0x5598a559c350 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5598a547eae0 con 0x5598a5591170 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.366+0000 7f11b7564740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5598a559f100 con 0x5598a5591a90 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11b7564740 10 client.0 _ll_register_callbacks cb 0x5598a54b79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54456/0 (socket says 172.21.15.73:54456) 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 -- 172.21.15.73:0/571383626 learned_addr learned my addr 172.21.15.73:0/571383626 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11ae59c640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591a90 0x5598a5599860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591170 0x5598a5599320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 -- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591a90 msgr2=0x5598a5599860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591a90 0x5598a5599860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 -- 172.21.15.73:0/571383626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591170 msgr2=0x5598a5599320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591170 0x5598a5599320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 -- 172.21.15.73:0/571383626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a5209210 con 0x5598a559c350 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11add9b640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c450 tx=0x7f11a4004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87811dee3dda595 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f1195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a4005000 con 0x5598a559c350 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f11a40051a0 con 0x5598a559c350 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a40054d0 con 0x5598a559c350 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f11a4004030 con 0x5598a559c350 2024-06-06T07:12:40.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11b7564740 10 client.16782 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.367+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5598a52d5620 con 0x5598a559c350 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a40054d0 con 0x5598a559c350 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f11a40035f0 con 0x5598a559c350 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 10 client.16782 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 10 client.16782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 20 client.16782 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a55ac460 0x5598a55ae850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 4 client.16782 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598a5508f10 con 0x5598a55ac460 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.368+0000 7f11ae59c640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a55ac460 0x5598a55ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.369+0000 7f11ae59c640 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a55ac460 0x5598a55ae850 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf4c6d9ee832de60 server_cookie=497916d6b3b1af77 in_seq=0 out_seq=0 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.369+0000 7f1195ffb640 10 client.16782 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:40.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.369+0000 7f1195ffb640 1 -- 172.21.15.73:0/571383626 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5598a5508f10 con 0x5598a55ac460 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.369+0000 7f1195ffb640 10 client.16782 handle_command_reply: tid=0 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.369+0000 7f11b7564740 1 client.16782 shutdown 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 msgr2=0x5598a5599da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c450 tx=0x7f11a4004b50 comp rx=0 tx=0).stop 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 shutdown_connections 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a55ac460 0x5598a55ae850 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a5599da0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591a90 0x5598a5599860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f1195ffb640 0 client.16782 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 --2- 172.21.15.73:0/571383626 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591170 0x5598a5599320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 >> 172.21.15.73:0/571383626 conn(0x5598a558d680 msgr2=0x5598a55933b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 shutdown_connections 2024-06-06T07:12:40.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 1 -- 172.21.15.73:0/571383626 wait complete. 2024-06-06T07:12:40.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.370+0000 7f11b7564740 20 client.16782 trim_cache size 0 max 16384 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 Processor -- start 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 -- start start 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5593b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a559c350 0x5598a55b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01aef10 con 0x5598a5591a90 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01b0b70 con 0x5598a559c350 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11b7564740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01af5e0 con 0x5598a5591170 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54470/0 (socket says 172.21.15.73:54470) 2024-06-06T07:12:40.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 -- 172.21.15.73:0/2812099185 learned_addr learned my addr 172.21.15.73:0/2812099185 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 -- 172.21.15.73:0/2812099185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 msgr2=0x5598a5593b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5593b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 -- 172.21.15.73:0/2812099185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a559c350 msgr2=0x5598a55b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a559c350 0x5598a55b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.385+0000 7f11ae59c640 1 -- 172.21.15.73:0/2812099185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598a5592200 con 0x5598a5591a90 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11ae59c640 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f11a00096c0 tx=0x7f11a0005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145c2a0f0c2a2bdf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f1197fff640 1 -- 172.21.15.73:0/2812099185 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0035040 con 0x5598a5591a90 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f1197fff640 1 -- 172.21.15.73:0/2812099185 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a0004390 con 0x5598a5591a90 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f1197fff640 1 -- 172.21.15.73:0/2812099185 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a0004b50 con 0x5598a5591a90 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 -- 172.21.15.73:0/2812099185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 msgr2=0x5598a55b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f11a00096c0 tx=0x7f11a0005a20 comp rx=0 tx=0).stop 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 -- 172.21.15.73:0/2812099185 shutdown_connections 2024-06-06T07:12:40.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a559c350 0x5598a55b44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a5591a90 0x5598a55b3f90 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 --2- 172.21.15.73:0/2812099185 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a5593b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 -- 172.21.15.73:0/2812099185 >> 172.21.15.73:0/2812099185 conn(0x5598a558d680 msgr2=0x5598a558c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 -- 172.21.15.73:0/2812099185 shutdown_connections 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.386+0000 7f11b7564740 1 -- 172.21.15.73:0/2812099185 wait complete. 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 -- start start 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 Processor -- start 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a55a4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a55a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01b0440 con 0x5598a559c350 2024-06-06T07:12:40.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01afd10 con 0x5598a5591a90 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11b01acf10 con 0x5598a5591170 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11b7564740 10 client.0 _ll_register_callbacks cb 0x7f11a0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54478/0 (socket says 172.21.15.73:54478) 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 -- 172.21.15.73:0/450380967 learned_addr learned my addr 172.21.15.73:0/450380967 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11ae59c640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a55a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a55a4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 -- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 msgr2=0x5598a55a4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a55a4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 -- 172.21.15.73:0/450380967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 msgr2=0x5598a55a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a55a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.387+0000 7f11add9b640 1 -- 172.21.15.73:0/450380967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598a55a0170 con 0x5598a559c350 2024-06-06T07:12:40.393 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11add9b640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f11a40059f0 tx=0x7f11a4014750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d572c5b170e893 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11a400fea0 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f11a4003db0 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11a0004390 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f11a400c970 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 10 client.16788 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5598a5616a80 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11a0004390 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f11a4005140 con 0x5598a559c350 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 10 client.16788 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:40.394 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 20 client.16788 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a5616ee0 0x5598a5619300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 4 client.16788 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5598a5502c20 con 0x5598a5616ee0 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.388+0000 7f11ae59c640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a5616ee0 0x5598a5619300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.389+0000 7f1195ffb640 10 client.16788 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:40.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.389+0000 7f11ae59c640 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a5616ee0 0x5598a5619300 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded8e7eae963c001 server_cookie=1f4592fc24541438 in_seq=0 out_seq=0 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f1195ffb640 1 -- 172.21.15.73:0/450380967 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5598a5502c20 con 0x5598a5616ee0 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f1195ffb640 10 client.16788 handle_command_reply: tid=0 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 client.16788 shutdown 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 msgr2=0x5598a559f970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f11a40059f0 tx=0x7f11a4014750 comp rx=0 tx=0).stop 2024-06-06T07:12:40.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 shutdown_connections 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5598a5616ee0 0x5598a5619300 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5598a559c350 0x5598a559f970 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5598a5591a90 0x5598a55a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 --2- 172.21.15.73:0/450380967 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5598a5591170 0x5598a55a4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 >> 172.21.15.73:0/450380967 conn(0x5598a558d680 msgr2=0x5598a55992e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f1195ffb640 0 client.16788 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 shutdown_connections 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 1 -- 172.21.15.73:0/450380967 wait complete. 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.417+0000 7f11b7564740 20 client.16788 trim_cache size 0 max 16384 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:12:40.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:12:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:12:40.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 msgr2=0x7f11980a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 0x7f11980a5260 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f11a8098580 tx=0x7f11a800cf50 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 msgr2=0x7f11b019ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f11a800eaa0 tx=0x7f11a800ef60 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 shutdown_connections 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b0106890 0x7f11b019ca20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11980a2da0 0x7f11980a5260 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11b0101290 0x7f11b0198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 --2- 172.21.15.73:0/4132021410 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11b01008d0 0x7f11b0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 >> 172.21.15.73:0/4132021410 conn(0x7f11b0077a20 msgr2=0x7f11b00ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 shutdown_connections 2024-06-06T07:12:40.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:40.418+0000 7f11b4858640 1 -- 172.21.15.73:0/4132021410 wait complete. 2024-06-06T07:12:40.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221657 1.88 MB/sec execute 467 sec latency 1808.218 ms 2024-06-06T07:12:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:40 smithi073 ceph-mon[32303]: pgmap v355: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:40 smithi136 ceph-mon[35901]: pgmap v355: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:40 smithi170 ceph-mon[36108]: pgmap v355: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:41.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221657 1.88 MB/sec execute 468 sec latency 2808.306 ms 2024-06-06T07:12:42.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221657 1.88 MB/sec execute 469 sec latency 3808.388 ms 2024-06-06T07:12:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:42 smithi073 ceph-mon[32303]: pgmap v356: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:42 smithi136 ceph-mon[35901]: pgmap v356: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:43.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:42 smithi170 ceph-mon[36108]: pgmap v356: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 454 KiB/s wr, 59 op/s 2024-06-06T07:12:43.454 INFO:tasks.workunit.client.0.smithi073.stdout: 1 221829 1.87 MB/sec execute 470 sec latency 4508.769 ms 2024-06-06T07:12:44.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 222373 1.87 MB/sec execute 471 sec latency 19.557 ms 2024-06-06T07:12:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:44 smithi073 ceph-mon[32303]: pgmap v357: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 524 KiB/s wr, 86 op/s 2024-06-06T07:12:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:44 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:12:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:44 smithi136 ceph-mon[35901]: pgmap v357: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 524 KiB/s wr, 86 op/s 2024-06-06T07:12:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:44 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:12:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:44 smithi170 ceph-mon[36108]: pgmap v357: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 524 KiB/s wr, 86 op/s 2024-06-06T07:12:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:44 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:12:45.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 222957 1.88 MB/sec execute 472 sec latency 19.619 ms 2024-06-06T07:12:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:45 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:12:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:45 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:12:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:45 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:12:46.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:45 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:12:46.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:45 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:12:46.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:45 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:12:46.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:45 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:12:46.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:45 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:12:46.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:45 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:12:46.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223455 1.89 MB/sec execute 473 sec latency 15.249 ms 2024-06-06T07:12:47.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223582 1.88 MB/sec execute 474 sec latency 744.087 ms 2024-06-06T07:12:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:47 smithi073 ceph-mon[32303]: pgmap v358: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 498 KiB/s wr, 73 op/s 2024-06-06T07:12:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:47 smithi136 ceph-mon[35901]: pgmap v358: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 498 KiB/s wr, 73 op/s 2024-06-06T07:12:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:47 smithi170 ceph-mon[36108]: pgmap v358: 129 pgs: 129 active+clean; 129 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 381 KiB/s rd, 498 KiB/s wr, 73 op/s 2024-06-06T07:12:48.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223582 1.88 MB/sec execute 475 sec latency 1744.143 ms 2024-06-06T07:12:49.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223582 1.87 MB/sec execute 476 sec latency 2744.248 ms 2024-06-06T07:12:49.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:49 smithi170 ceph-mon[36108]: pgmap v359: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:12:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:49 smithi073 ceph-mon[32303]: pgmap v359: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:12:50.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:49 smithi136 ceph-mon[35901]: pgmap v359: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 826 KiB/s rd, 1.0 MiB/s wr, 121 op/s 2024-06-06T07:12:50.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223594 1.87 MB/sec execute 477 sec latency 3030.177 ms 2024-06-06T07:12:51.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223594 1.87 MB/sec execute 478 sec latency 1645.045 ms 2024-06-06T07:12:51.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:51 smithi170 ceph-mon[36108]: pgmap v360: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:52.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:51 smithi073 ceph-mon[32303]: pgmap v360: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:52.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:51 smithi136 ceph-mon[35901]: pgmap v360: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:52.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223594 1.86 MB/sec execute 479 sec latency 2645.150 ms 2024-06-06T07:12:53.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223649 1.86 MB/sec execute 480 sec latency 3350.641 ms 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 -- 172.21.15.73:0/1257009123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 msgr2=0x7f853010d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 --2- 172.21.15.73:0/1257009123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f853010d780 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f8528009cd0 tx=0x7f852802a8e0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 -- 172.21.15.73:0/1257009123 shutdown_connections 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 --2- 172.21.15.73:0/1257009123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f853010d780 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 --2- 172.21.15.73:0/1257009123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8530100900 0x7f8530100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 --2- 172.21.15.73:0/1257009123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f85301068c0 0x7f8530106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 -- 172.21.15.73:0/1257009123 >> 172.21.15.73:0/1257009123 conn(0x7f8530077ab0 msgr2=0x7f8530077ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 -- 172.21.15.73:0/1257009123 shutdown_connections 2024-06-06T07:12:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.505+0000 7f8534e88640 1 -- 172.21.15.73:0/1257009123 wait complete. 2024-06-06T07:12:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 Processor -- start 2024-06-06T07:12:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 -- start start 2024-06-06T07:12:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8530100900 0x7f8530196f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85301068c0 0x7f853019b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110b80 con 0x7f85301012c0 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110a00 con 0x7f85301068c0 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f8534e88640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8530110d00 con 0x7f8530100900 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35590/0 (socket says 172.21.15.73:35590) 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 -- 172.21.15.73:0/3406044908 learned_addr learned my addr 172.21.15.73:0/3406044908 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852e59c640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8530100900 0x7f8530196f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 -- 172.21.15.73:0/3406044908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8530100900 msgr2=0x7f8530196f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8530100900 0x7f8530196f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 -- 172.21.15.73:0/3406044908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85301068c0 msgr2=0x7f853019b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85301068c0 0x7f853019b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 -- 172.21.15.73:0/3406044908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f853019bf30 con 0x7f85301012c0 2024-06-06T07:12:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.506+0000 7f852dd9b640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8524007b50 tx=0x7f852400e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.507+0000 7f851f7fe640 1 -- 172.21.15.73:0/3406044908 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8524007c40 con 0x7f85301012c0 2024-06-06T07:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.507+0000 7f851f7fe640 1 -- 172.21.15.73:0/3406044908 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8524007de0 con 0x7f85301012c0 2024-06-06T07:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.507+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f853019c220 con 0x7f85301012c0 2024-06-06T07:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.507+0000 7f851f7fe640 1 -- 172.21.15.73:0/3406044908 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852400ad40 con 0x7f85301012c0 2024-06-06T07:12:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.507+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f853006a210 con 0x7f85301012c0 2024-06-06T07:12:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.509+0000 7f851f7fe640 1 -- 172.21.15.73:0/3406044908 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f852400eb30 con 0x7f85301012c0 2024-06-06T07:12:53.517 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.509+0000 7f8537b92740 1 Processor -- start 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.509+0000 7f8537b92740 1 -- start start 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 0x563ed2dc0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2dc1350 0x563ed2dc3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ed2d1a4f0 con 0x563ed2db6170 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ed2cb8870 con 0x563ed2db6a90 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f8537b92740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ed2d164b0 con 0x563ed2dc1350 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f851f7fe640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 0x7f85180a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35604/0 (socket says 172.21.15.73:35604) 2024-06-06T07:12:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 0x563ed2dc0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 -- 172.21.15.73:0/232890140 learned_addr learned my addr 172.21.15.73:0/232890140 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 0x7f85180a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f851f7fe640 1 -- 172.21.15.73:0/3406044908 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f85240d0ee0 con 0x7f85301012c0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852dd9b640 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2dc1350 0x563ed2dc3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 -- 172.21.15.73:0/232890140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2dc1350 msgr2=0x563ed2dc3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2dc1350 0x563ed2dc3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 -- 172.21.15.73:0/232890140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 msgr2=0x563ed2db6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 -- 172.21.15.73:0/232890140 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed2cd49b0 con 0x563ed2db6a90 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852ed9d640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 0x7f85180a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f852802ae00 tx=0x7f85280095c0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.510+0000 7f852e59c640 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 0x563ed2dc0e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f852000b760 tx=0x7f852000bc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=465feaebecf1fb01 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f851cff9640 1 -- 172.21.15.73:0/232890140 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852000d950 con 0x563ed2db6a90 2024-06-06T07:12:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f851cff9640 1 -- 172.21.15.73:0/232890140 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520019070 con 0x563ed2db6a90 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f851cff9640 1 -- 172.21.15.73:0/232890140 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520014de0 con 0x563ed2db6a90 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 -- 172.21.15.73:0/232890140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 msgr2=0x563ed2dc0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 0x563ed2dc0e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f852000b760 tx=0x7f852000bc20 comp rx=0 tx=0).stop 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 -- 172.21.15.73:0/232890140 shutdown_connections 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2dc1350 0x563ed2dc3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6a90 0x563ed2dc0e10 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 --2- 172.21.15.73:0/232890140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 -- 172.21.15.73:0/232890140 >> 172.21.15.73:0/232890140 conn(0x563ed2db2680 msgr2=0x563ed2db2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 -- 172.21.15.73:0/232890140 shutdown_connections 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.511+0000 7f8537b92740 1 -- 172.21.15.73:0/232890140 wait complete. 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 -- start start 2024-06-06T07:12:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 Processor -- start 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 0x563ed2dbe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dbe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ed2dc3f80 con 0x563ed2dc1350 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ed2ca3ae0 con 0x563ed2db6170 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ed2dc4100 con 0x563ed2db6a90 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f8537b92740 10 client.0 _ll_register_callbacks cb 0x563ed2cdc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35620/0 (socket says 172.21.15.73:35620) 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 -- 172.21.15.73:0/3157441988 learned_addr learned my addr 172.21.15.73:0/3157441988 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852e59c640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 0x563ed2dbe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852dd9b640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dbe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 -- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 msgr2=0x563ed2dbe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dbe760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 -- 172.21.15.73:0/3157441988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 msgr2=0x563ed2dbe220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 0x563ed2dbe220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852ed9d640 1 -- 172.21.15.73:0/3157441988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed2a2e210 con 0x563ed2dc1350 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.512+0000 7f852e59c640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 0x563ed2dbe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f852ed9d640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8528009cd0 tx=0x7f8528005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0ae4d8d85caf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852804f070 con 0x563ed2dc1350 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f8528005250 con 0x563ed2dc1350 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8520019070 con 0x563ed2dc1350 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f852805b050 con 0x563ed2dc1350 2024-06-06T07:12:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 10 client.16800 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563ed2afa620 con 0x563ed2dc1350 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520019070 con 0x563ed2dc1350 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f8528030070 con 0x563ed2dc1350 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 10 client.16800 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 10 client.16800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 20 client.16800 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2dd1450 0x563ed2dd38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 4 client.16800 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.513+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ed2d2df10 con 0x563ed2dd1450 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.514+0000 7f852e59c640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2dd1450 0x563ed2dd38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.514+0000 7f8515ffb640 10 client.16800 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.514+0000 7f852e59c640 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2dd1450 0x563ed2dd38a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e74f81968c5873c server_cookie=cb37a25f362eed66 in_seq=0 out_seq=0 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8515ffb640 1 -- 172.21.15.73:0/3157441988 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ed2d2df10 con 0x563ed2dd1450 2024-06-06T07:12:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8515ffb640 10 client.16800 handle_command_reply: tid=0 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 client.16800 shutdown 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 msgr2=0x563ed2dbeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f8528009cd0 tx=0x7f8528005ef0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 shutdown_connections 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2dd1450 0x563ed2dd38a0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2dc1350 0x563ed2dbeca0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dbe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8515ffb640 0 client.16800 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 --2- 172.21.15.73:0/3157441988 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2db6170 0x563ed2dbe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 >> 172.21.15.73:0/3157441988 conn(0x563ed2db2680 msgr2=0x563ed2db8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 shutdown_connections 2024-06-06T07:12:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 1 -- 172.21.15.73:0/3157441988 wait complete. 2024-06-06T07:12:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.515+0000 7f8537b92740 20 client.16800 trim_cache size 0 max 16384 2024-06-06T07:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 Processor -- start 2024-06-06T07:12:53.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 -- start start 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dd91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dd96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85301acbd0 con 0x563ed2db6170 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85301ae830 con 0x563ed2dc1350 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f8537b92740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85301ad2a0 con 0x563ed2db6a90 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f852e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f852e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35630/0 (socket says 172.21.15.73:35630) 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.530+0000 7f852e59c640 1 -- 172.21.15.73:0/1261866494 learned_addr learned my addr 172.21.15.73:0/1261866494 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852ed9d640 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dd96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 -- 172.21.15.73:0/1261866494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 msgr2=0x563ed2dd91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dd91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 -- 172.21.15.73:0/1261866494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 msgr2=0x563ed2dd96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dd96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 -- 172.21.15.73:0/1261866494 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ed2db68c0 con 0x563ed2db6170 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852ed9d640 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f852e59c640 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x563ed2db7710 tx=0x7f85200047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e1f19e71f55202 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f8517fff640 1 -- 172.21.15.73:0/1261866494 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520014de0 con 0x563ed2db6170 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f8517fff640 1 -- 172.21.15.73:0/1261866494 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520029070 con 0x563ed2db6170 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.531+0000 7f8517fff640 1 -- 172.21.15.73:0/1261866494 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f852001d980 con 0x563ed2db6170 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- 172.21.15.73:0/1261866494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 msgr2=0x563ed2db8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x563ed2db7710 tx=0x7f85200047d0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- 172.21.15.73:0/1261866494 shutdown_connections 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dd96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dd91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- 172.21.15.73:0/1261866494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2db8b30 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- 172.21.15.73:0/1261866494 >> 172.21.15.73:0/1261866494 conn(0x563ed2db2680 msgr2=0x563ed2db2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- 172.21.15.73:0/1261866494 shutdown_connections 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- 172.21.15.73:0/1261866494 wait complete. 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- start start 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 Processor -- start 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dc9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dc4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85301ae100 con 0x563ed2db6170 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f85301ad9d0 con 0x563ed2dc1350 2024-06-06T07:12:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f85301aabd0 con 0x563ed2db6a90 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f8537b92740 10 client.0 _ll_register_callbacks cb 0x7f8528009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f852e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f852e59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35636/0 (socket says 172.21.15.73:35636) 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.532+0000 7f852e59c640 1 -- 172.21.15.73:0/4172813654 learned_addr learned my addr 172.21.15.73:0/4172813654 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852ed9d640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dc4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 -- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 msgr2=0x563ed2dc9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dc9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 -- 172.21.15.73:0/4172813654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 msgr2=0x563ed2dc4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dc4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 -- 172.21.15.73:0/4172813654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ed2dc5110 con 0x563ed2db6170 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f852e59c640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f852001d560 tx=0x7f85200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad8789ae84d8e5d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:12:53.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8520031020 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 9554+0+0 (secure 0 0 0) 0x7f852000d950 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8520029070 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f852003c050 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 10 client.16806 fetch_fsmap learned FSMap version 48 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x563ed2e3bc10 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8520029070 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f8520013070 con 0x563ed2db6170 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 10 client.16806 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 20 client.16806 populate_metadata read hostname 'smithi073' 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2e3c150 0x563ed2e3e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 4 client.16806 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.533+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ed2d27c20 con 0x563ed2e3c150 2024-06-06T07:12:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.534+0000 7f852dd9b640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2e3c150 0x563ed2e3e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.534+0000 7f852dd9b640 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2e3c150 0x563ed2e3e590 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4567a36218c5fe7 server_cookie=124d2c84c81741df in_seq=0 out_seq=0 2024-06-06T07:12:53.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.534+0000 7f8515ffb640 10 client.16806 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8515ffb640 1 -- 172.21.15.73:0/4172813654 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ed2d27c20 con 0x563ed2e3c150 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8515ffb640 10 client.16806 handle_command_reply: tid=0 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 client.16806 shutdown 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 msgr2=0x563ed2dc9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f852001d560 tx=0x7f85200046c0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 shutdown_connections 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ed2e3c150 0x563ed2e3e590 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ed2dc1350 0x563ed2dc4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ed2db6a90 0x563ed2dc9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 --2- 172.21.15.73:0/4172813654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ed2db6170 0x563ed2dc9530 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 >> 172.21.15.73:0/4172813654 conn(0x563ed2db2680 msgr2=0x563ed2dbe1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8515ffb640 0 client.16806 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 shutdown_connections 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 1 -- 172.21.15.73:0/4172813654 wait complete. 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.562+0000 7f8537b92740 20 client.16806 trim_cache size 0 max 16384 2024-06-06T07:12:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:12:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:12:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:12:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 msgr2=0x7f85180a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 0x7f85180a5260 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f852802ae00 tx=0x7f85280095c0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 msgr2=0x7f8530197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8524007b50 tx=0x7f852400e420 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 shutdown_connections 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f85301068c0 0x7f853019b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85301012c0 0x7f8530197480 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f85180a2da0 0x7f85180a5260 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 --2- 172.21.15.73:0/3406044908 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8530100900 0x7f8530196f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 >> 172.21.15.73:0/3406044908 conn(0x7f8530077ab0 msgr2=0x7f85301032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 shutdown_connections 2024-06-06T07:12:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:12:53.563+0000 7f8534e88640 1 -- 172.21.15.73:0/3406044908 wait complete. 2024-06-06T07:12:53.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:53 smithi170 ceph-mon[36108]: pgmap v361: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:54.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:53 smithi073 ceph-mon[32303]: pgmap v361: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:54.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:53 smithi136 ceph-mon[35901]: pgmap v361: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 597 KiB/s wr, 74 op/s 2024-06-06T07:12:54.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223649 1.86 MB/sec execute 481 sec latency 1107.154 ms 2024-06-06T07:12:55.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 223836 1.85 MB/sec execute 482 sec latency 1392.858 ms 2024-06-06T07:12:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:55 smithi170 ceph-mon[36108]: pgmap v362: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 616 KiB/s wr, 84 op/s 2024-06-06T07:12:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:55 smithi073 ceph-mon[32303]: pgmap v362: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 616 KiB/s wr, 84 op/s 2024-06-06T07:12:56.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:55 smithi136 ceph-mon[35901]: pgmap v362: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 616 KiB/s wr, 84 op/s 2024-06-06T07:12:56.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 224108 1.85 MB/sec execute 483 sec latency 18.414 ms 2024-06-06T07:12:57.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 224556 1.85 MB/sec execute 484 sec latency 15.762 ms 2024-06-06T07:12:57.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:57 smithi170 ceph-mon[36108]: pgmap v363: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 58 op/s 2024-06-06T07:12:58.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:57 smithi073 ceph-mon[32303]: pgmap v363: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 58 op/s 2024-06-06T07:12:58.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:57 smithi136 ceph-mon[35901]: pgmap v363: 129 pgs: 129 active+clean; 133 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 463 KiB/s rd, 546 KiB/s wr, 58 op/s 2024-06-06T07:12:58.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225036 1.85 MB/sec execute 485 sec latency 19.744 ms 2024-06-06T07:12:59.455 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225205 1.85 MB/sec execute 486 sec latency 646.892 ms 2024-06-06T07:12:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:12:59 smithi170 ceph-mon[36108]: pgmap v364: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1003 KiB/s wr, 109 op/s 2024-06-06T07:13:00.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:12:59 smithi073 ceph-mon[32303]: pgmap v364: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1003 KiB/s wr, 109 op/s 2024-06-06T07:13:00.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:12:59 smithi136 ceph-mon[35901]: pgmap v364: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 1003 KiB/s wr, 109 op/s 2024-06-06T07:13:00.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225205 1.85 MB/sec execute 487 sec latency 1647.010 ms 2024-06-06T07:13:01.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225205 1.84 MB/sec execute 488 sec latency 2647.106 ms 2024-06-06T07:13:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:01 smithi170 ceph-mon[36108]: pgmap v365: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:02.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:01 smithi073 ceph-mon[32303]: pgmap v365: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:02.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:01 smithi136 ceph-mon[35901]: pgmap v365: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:02.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225205 1.84 MB/sec execute 489 sec latency 3647.139 ms 2024-06-06T07:13:03.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225362 1.83 MB/sec execute 490 sec latency 4346.416 ms 2024-06-06T07:13:03.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:03 smithi170 ceph-mon[36108]: pgmap v366: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:04.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:03 smithi073 ceph-mon[32303]: pgmap v366: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:04.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:03 smithi136 ceph-mon[35901]: pgmap v366: 129 pgs: 129 active+clean; 136 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 476 KiB/s wr, 62 op/s 2024-06-06T07:13:04.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 225883 1.83 MB/sec execute 491 sec latency 22.655 ms 2024-06-06T07:13:05.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:04 smithi073 ceph-mon[32303]: pgmap v367: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 536 KiB/s wr, 87 op/s 2024-06-06T07:13:05.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:04 smithi073 ceph-mon[32303]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T07:13:05.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:04 smithi136 ceph-mon[35901]: pgmap v367: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 536 KiB/s wr, 87 op/s 2024-06-06T07:13:05.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:04 smithi136 ceph-mon[35901]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T07:13:05.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:04 smithi170 ceph-mon[36108]: pgmap v367: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 414 KiB/s rd, 536 KiB/s wr, 87 op/s 2024-06-06T07:13:05.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:04 smithi170 ceph-mon[36108]: osdmap e115: 12 total, 12 up, 12 in 2024-06-06T07:13:05.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 226449 1.84 MB/sec execute 492 sec latency 19.387 ms 2024-06-06T07:13:06.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 226888 1.84 MB/sec execute 493 sec latency 15.178 ms 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1735438969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406a1b0 msgr2=0x7fdec406a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1735438969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406a1b0 0x7fdec406a630 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009930 tx=0x7fdeb402a420 comp rx=0 tx=0).stop 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1735438969 shutdown_connections 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1735438969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec406ab70 0x7fdec410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1735438969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406a1b0 0x7fdec406a630 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1735438969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec4107a70 0x7fdec4107e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1735438969 >> 172.21.15.73:0/1735438969 conn(0x7fdec4077710 msgr2=0x7fdec4077b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1735438969 shutdown_connections 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1735438969 wait complete. 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.661+0000 7fdecc4ee640 1 Processor -- start 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 -- start start 2024-06-06T07:13:06.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec406a1b0 0x7fdec4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec4107a70 0x7fdec419ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4111c80 con 0x7fdec406ab70 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4111b00 con 0x7fdec4107a70 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecc4ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdec4111e00 con 0x7fdec406a1b0 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58532/0 (socket says 172.21.15.73:58532) 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 -- 172.21.15.73:0/1642842303 learned_addr learned my addr 172.21.15.73:0/1642842303 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec4107a70 0x7fdec419ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec406a1b0 0x7fdec4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.665 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 -- 172.21.15.73:0/1642842303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec406a1b0 msgr2=0x7fdec4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec406a1b0 0x7fdec4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 -- 172.21.15.73:0/1642842303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec4107a70 msgr2=0x7fdec419ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec4107a70 0x7fdec419ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 -- 172.21.15.73:0/1642842303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec419d150 con 0x7fdec406ab70 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009230 tx=0x7fdeb4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdebb7fe640 1 -- 172.21.15.73:0/1642842303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb403f030 con 0x7fdec406ab70 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.662+0000 7fdebb7fe640 1 -- 172.21.15.73:0/1642842303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb4003aa0 con 0x7fdec406ab70 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.663+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec419d3e0 con 0x7fdec406ab70 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.663+0000 7fdebb7fe640 1 -- 172.21.15.73:0/1642842303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4035070 con 0x7fdec406ab70 2024-06-06T07:13:06.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.663+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec41a4cd0 con 0x7fdec406ab70 2024-06-06T07:13:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.665+0000 7fdebb7fe640 1 -- 172.21.15.73:0/1642842303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdeb4042030 con 0x7fdec406ab70 2024-06-06T07:13:06.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.665+0000 7fdecf1f9740 1 Processor -- start 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 -- start start 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a30220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a30760 0x558668a32b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5586689854f0 con 0x558668a219b0 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558668923870 con 0x558668a30760 2024-06-06T07:13:06.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5586689814b0 con 0x558668a21090 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a30220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43698/0 (socket says 172.21.15.73:43698) 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecaa8b640 1 -- 172.21.15.73:0/2200268213 learned_addr learned my addr 172.21.15.73:0/2200268213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdebb7fe640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 0x7fdeb00a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdec9a89640 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a30760 0x558668a32b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 0x7fdeb00a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 -- 172.21.15.73:0/2200268213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 msgr2=0x558668a21470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 -- 172.21.15.73:0/2200268213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a30760 msgr2=0x558668a32b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a30760 0x558668a32b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdeca28a640 1 -- 172.21.15.73:0/2200268213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55866893f9b0 con 0x558668a219b0 2024-06-06T07:13:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.666+0000 7fdebb7fe640 1 -- 172.21.15.73:0/1642842303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 9642+0+0 (secure 0 0 0) 0x7fdeb40efeb0 con 0x7fdec406ab70 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdeca28a640 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a30220 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009c00 tx=0x7fdec0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4583cce88a4f86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdeb8bfe640 1 -- 172.21.15.73:0/2200268213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0004050 con 0x558668a219b0 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 0x7fdeb00a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005f40 tx=0x7fdebc004ab0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdeb8bfe640 1 -- 172.21.15.73:0/2200268213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdec00054d0 con 0x558668a219b0 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdeb8bfe640 1 -- 172.21.15.73:0/2200268213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0004c60 con 0x558668a219b0 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.667+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2200268213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 msgr2=0x558668a30220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a30220 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fdec0009c00 tx=0x7fdec0011810 comp rx=0 tx=0).stop 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2200268213 shutdown_connections 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a30760 0x558668a32b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a30220 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2200268213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a21470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2200268213 >> 172.21.15.73:0/2200268213 conn(0x558668a1d680 msgr2=0x558668a1da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2200268213 shutdown_connections 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2200268213 wait complete. 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- start start 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 Processor -- start 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a2d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a2e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558668a33390 con 0x558668a219b0 2024-06-06T07:13:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55866890eae0 con 0x558668a21090 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558668a33510 con 0x558668a30760 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecf1f9740 10 client.0 _ll_register_callbacks cb 0x5586689479b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a2e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58558/0 (socket says 172.21.15.73:58558) 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdec9a89640 1 -- 172.21.15.73:0/1573115593 learned_addr learned my addr 172.21.15.73:0/1573115593 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.668+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a2d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 -- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 msgr2=0x558668a2e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a2e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 -- 172.21.15.73:0/1573115593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 msgr2=0x558668a2d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a2d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 -- 172.21.15.73:0/1573115593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558668699210 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ed250 tx=0x7fdeb4005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1251edc138938ac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4049040 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9642+0+0 (secure 0 0 0) 0x7fdeb40ed420 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdec00054d0 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdeb40bbdf0 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdecf1f9740 10 client.16818 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558668765620 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdec00054d0 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.669+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fdeb40c4750 con 0x558668a219b0 2024-06-06T07:13:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 10 client.16818 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 10 client.16818 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 20 client.16818 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668a408c0 0x558668a42d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 4 client.16818 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558668998f10 con 0x558668a408c0 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668a408c0 0x558668a42d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdeadffb640 10 client.16818 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.670+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668a408c0 0x558668a42d50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2444e2513bb986ea server_cookie=e97139986b3877d in_seq=0 out_seq=0 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdeadffb640 1 -- 172.21.15.73:0/1573115593 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558668998f10 con 0x558668a408c0 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdeadffb640 10 client.16818 handle_command_reply: tid=0 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 client.16818 shutdown 2024-06-06T07:13:06.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 msgr2=0x558668a2dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40ed250 tx=0x7fdeb4005bd0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 shutdown_connections 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668a408c0 0x558668a42d50 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a2e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a2dc00 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdeadffb640 0 client.16818 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1573115593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a2d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 >> 172.21.15.73:0/1573115593 conn(0x558668a1d680 msgr2=0x558668a234f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 shutdown_connections 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1573115593 wait complete. 2024-06-06T07:13:06.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.671+0000 7fdecf1f9740 20 client.16818 trim_cache size 0 max 16384 2024-06-06T07:13:06.696 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 Processor -- start 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 -- start start 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a221e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a48ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41acd10 con 0x558668a219b0 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ae970 con 0x558668a21090 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ad3e0 con 0x558668a30760 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdec9a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58568/0 (socket says 172.21.15.73:58568) 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdeca28a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a221e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a48ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdec9a89640 1 -- 172.21.15.73:0/2072673534 learned_addr learned my addr 172.21.15.73:0/2072673534 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdec9a89640 1 -- 172.21.15.73:0/2072673534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 msgr2=0x558668a48ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.688+0000 7fdec9a89640 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a48ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdec9a89640 1 -- 172.21.15.73:0/2072673534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 msgr2=0x558668a221e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdec9a89640 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a221e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdec9a89640 1 -- 172.21.15.73:0/2072673534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558668a217e0 con 0x558668a219b0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdec9a89640 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40b8480 tx=0x7fdeb4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e15479a15efce7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdeaffff640 1 -- 172.21.15.73:0/2072673534 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb4049070 con 0x558668a219b0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdeaffff640 1 -- 172.21.15.73:0/2072673534 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb40c4be0 con 0x558668a219b0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdeaffff640 1 -- 172.21.15.73:0/2072673534 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb40d6080 con 0x558668a219b0 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2072673534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 msgr2=0x558668a48660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.689+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdeb40b8480 tx=0x7fdeb4005c90 comp rx=0 tx=0).stop 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2072673534 shutdown_connections 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a30760 0x558668a48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a219b0 0x558668a48660 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/2072673534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a21090 0x558668a221e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2072673534 >> 172.21.15.73:0/2072673534 conn(0x558668a1d680 msgr2=0x558668a1da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2072673534 shutdown_connections 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- 172.21.15.73:0/2072673534 wait complete. 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- start start 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 Processor -- start 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a34430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a219b0 0x558668a34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41ae240 con 0x558668a30760 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41adb10 con 0x558668a219b0 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdec41aad10 con 0x558668a21090 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecf1f9740 10 client.0 _ll_register_callbacks cb 0x7fdeb40b87c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecaa8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58572/0 (socket says 172.21.15.73:58572) 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdecaa8b640 1 -- 172.21.15.73:0/1996206798 learned_addr learned my addr 172.21.15.73:0/1996206798 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a34430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.690+0000 7fdec9a89640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a219b0 0x558668a34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 -- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 msgr2=0x558668a34430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a34430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 -- 172.21.15.73:0/1996206798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a219b0 msgr2=0x558668a34970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:06.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a219b0 0x558668a34970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 -- 172.21.15.73:0/1996206798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558668aaa630 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecaa8b640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009fc0 tx=0x7fdebc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25799a478f19d8e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdebc01a070 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 9642+0+0 (secure 0 0 0) 0x7fdebc004b20 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdeb40c4be0 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fdebc011e70 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecf1f9740 10 client.16824 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x558668aab8a0 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdeb40c4be0 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fdebc01d070 con 0x558668a30760 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecf1f9740 10 client.16824 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:06.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.691+0000 7fdecf1f9740 10 client.16824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdecf1f9740 20 client.16824 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668aac0b0 0x558668aae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdecf1f9740 4 client.16824 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558668992c20 con 0x558668aac0b0 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668aac0b0 0x558668aae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdeadffb640 10 client.16824 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.692+0000 7fdeca28a640 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668aac0b0 0x558668aae4d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c0d79ae8129d65e server_cookie=17faf75d5b20b3c7 in_seq=0 out_seq=0 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.718+0000 7fdeadffb640 1 -- 172.21.15.73:0/1996206798 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558668992c20 con 0x558668aac0b0 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.718+0000 7fdeadffb640 10 client.16824 handle_command_reply: tid=0 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.718+0000 7fdecf1f9740 1 client.16824 shutdown 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.718+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 msgr2=0x558668a34eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.718+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009fc0 tx=0x7fdebc004a40 comp rx=0 tx=0).stop 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 shutdown_connections 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558668aac0b0 0x558668aae4d0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558668a30760 0x558668a34eb0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdeadffb640 0 client.16824 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558668a219b0 0x558668a34970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 --2- 172.21.15.73:0/1996206798 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558668a21090 0x558668a34430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 >> 172.21.15.73:0/1996206798 conn(0x558668a1d680 msgr2=0x558668a221e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 shutdown_connections 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 1 -- 172.21.15.73:0/1996206798 wait complete. 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecf1f9740 20 client.16824 trim_cache size 0 max 16384 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:13:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 msgr2=0x7fdeb00a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 0x7fdeb00a52e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005f40 tx=0x7fdebc004ab0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 msgr2=0x7fdec41986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fdeb4009230 tx=0x7fdeb4004270 comp rx=0 tx=0).stop 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 shutdown_connections 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdec4107a70 0x7fdec419ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdec406ab70 0x7fdec41986a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdeb00a2e20 0x7fdeb00a52e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 --2- 172.21.15.73:0/1642842303 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdec406a1b0 0x7fdec4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 >> 172.21.15.73:0/1642842303 conn(0x7fdec4077710 msgr2=0x7fdec40feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 shutdown_connections 2024-06-06T07:13:06.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:06.719+0000 7fdecc4ee640 1 -- 172.21.15.73:0/1642842303 wait complete. 2024-06-06T07:13:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:06 smithi073 ceph-mon[32303]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 620 KiB/s wr, 92 op/s 2024-06-06T07:13:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:06 smithi136 ceph-mon[35901]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 620 KiB/s wr, 92 op/s 2024-06-06T07:13:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:06 smithi170 ceph-mon[36108]: pgmap v369: 129 pgs: 129 active+clean; 134 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 620 KiB/s wr, 92 op/s 2024-06-06T07:13:07.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227128 1.84 MB/sec execute 494 sec latency 618.302 ms 2024-06-06T07:13:08.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227128 1.84 MB/sec execute 495 sec latency 1618.382 ms 2024-06-06T07:13:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:08 smithi073 ceph-mon[32303]: pgmap v370: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:08 smithi136 ceph-mon[35901]: pgmap v370: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:08 smithi170 ceph-mon[36108]: pgmap v370: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:09.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227128 1.84 MB/sec execute 496 sec latency 2618.418 ms 2024-06-06T07:13:10.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227195 1.83 MB/sec execute 497 sec latency 2905.082 ms 2024-06-06T07:13:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:10 smithi073 ceph-mon[32303]: pgmap v371: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:10 smithi136 ceph-mon[35901]: pgmap v371: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:10 smithi170 ceph-mon[36108]: pgmap v371: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:11.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227195 1.83 MB/sec execute 498 sec latency 1466.369 ms 2024-06-06T07:13:12.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227195 1.83 MB/sec execute 499 sec latency 2466.459 ms 2024-06-06T07:13:13.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227195 1.82 MB/sec execute 500 sec latency 3466.578 ms 2024-06-06T07:13:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:13 smithi170 ceph-mon[36108]: pgmap v372: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:13 smithi170 ceph-mon[36108]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T07:13:14.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:13 smithi073 ceph-mon[32303]: pgmap v372: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:14.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:13 smithi073 ceph-mon[32303]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T07:13:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:13 smithi136 ceph-mon[35901]: pgmap v372: 129 pgs: 129 active+clean; 131 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 856 KiB/s rd, 808 KiB/s wr, 115 op/s 2024-06-06T07:13:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:13 smithi136 ceph-mon[35901]: osdmap e116: 12 total, 12 up, 12 in 2024-06-06T07:13:14.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227195 1.82 MB/sec execute 501 sec latency 4466.621 ms 2024-06-06T07:13:15.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227391 1.82 MB/sec execute 502 sec latency 4754.510 ms 2024-06-06T07:13:15.825 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:15 smithi073 ceph-mon[32303]: pgmap v374: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 827 KiB/s wr, 107 op/s 2024-06-06T07:13:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:15 smithi170 ceph-mon[36108]: pgmap v374: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 827 KiB/s wr, 107 op/s 2024-06-06T07:13:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:15 smithi136 ceph-mon[35901]: pgmap v374: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 658 KiB/s rd, 827 KiB/s wr, 107 op/s 2024-06-06T07:13:16.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 227672 1.81 MB/sec execute 503 sec latency 13.148 ms 2024-06-06T07:13:17.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228163 1.82 MB/sec execute 504 sec latency 16.504 ms 2024-06-06T07:13:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:17 smithi170 ceph-mon[36108]: pgmap v375: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 768 KiB/s wr, 99 op/s 2024-06-06T07:13:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:17 smithi073 ceph-mon[32303]: pgmap v375: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 768 KiB/s wr, 99 op/s 2024-06-06T07:13:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:17 smithi136 ceph-mon[35901]: pgmap v375: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 768 KiB/s wr, 99 op/s 2024-06-06T07:13:18.456 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228668 1.82 MB/sec execute 505 sec latency 18.585 ms 2024-06-06T07:13:19.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228748 1.81 MB/sec execute 506 sec latency 832.452 ms 2024-06-06T07:13:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 -- 172.21.15.73:0/3021815127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a810bf70 msgr2=0x7f83a810e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 --2- 172.21.15.73:0/3021815127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a810bf70 0x7f83a810e410 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f8398009ca0 tx=0x7f839802a9e0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 -- 172.21.15.73:0/3021815127 shutdown_connections 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 --2- 172.21.15.73:0/3021815127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a810bf70 0x7f83a810e410 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 --2- 172.21.15.73:0/3021815127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83a8103550 0x7f83a810ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 --2- 172.21.15.73:0/3021815127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a8102c30 0x7f83a8103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.810+0000 7f83af276640 1 -- 172.21.15.73:0/3021815127 >> 172.21.15.73:0/3021815127 conn(0x7f83a8077a20 msgr2=0x7f83a8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- 172.21.15.73:0/3021815127 shutdown_connections 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- 172.21.15.73:0/3021815127 wait complete. 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 Processor -- start 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- start start 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83a8102c30 0x7f83a81980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 0x7f83a819c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111c20 con 0x7f83a8103550 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111aa0 con 0x7f83a810bf70 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83af276640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83a8111da0 con 0x7f83a8102c30 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57634/0 (socket says 172.21.15.73:57634) 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.811+0000 7f83ac811640 1 -- 172.21.15.73:0/2923633621 learned_addr learned my addr 172.21.15.73:0/2923633621 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ad813640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 0x7f83a819c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 -- 172.21.15.73:0/2923633621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83a8102c30 msgr2=0x7f83a81980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83a8102c30 0x7f83a81980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 -- 172.21.15.73:0/2923633621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 msgr2=0x7f83a819c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 0x7f83a819c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 -- 172.21.15.73:0/2923633621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a819d0b0 con 0x7f83a8103550 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ad813640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 0x7f83a819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83ac811640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f839c00d770 tx=0x7f839c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83a67fc640 1 -- 172.21.15.73:0/2923633621 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c017070 con 0x7f83a8103550 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a819d3a0 con 0x7f83a8103550 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83a67fc640 1 -- 172.21.15.73:0/2923633621 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839c0054d0 con 0x7f83a8103550 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83a67fc640 1 -- 172.21.15.73:0/2923633621 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c004cf0 con 0x7f83a8103550 2024-06-06T07:13:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.812+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a81a4c30 con 0x7f83a8103550 2024-06-06T07:13:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83a67fc640 1 -- 172.21.15.73:0/2923633621 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f839c004e90 con 0x7f83a8103550 2024-06-06T07:13:19.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 Processor -- start 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 -- start start 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc31c960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 0x560bfc327220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560bfc27c4f0 con 0x560bfc327760 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bfc21a870 con 0x560bfc31cea0 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83b1f84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560bfc2784b0 con 0x560bfc31c580 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57638/0 (socket says 172.21.15.73:57638) 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83ac811640 1 -- 172.21.15.73:0/3577780335 learned_addr learned my addr 172.21.15.73:0/3577780335 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83ad813640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc31c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83a67fc640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 0x7f83900a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.815+0000 7f83ad012640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 0x560bfc327220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ad813640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 0x7f83900a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83a67fc640 1 -- 172.21.15.73:0/2923633621 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f839c0cf2e0 con 0x7f83a8103550 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 -- 172.21.15.73:0/3577780335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 msgr2=0x560bfc31c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc31c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 -- 172.21.15.73:0/3577780335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 msgr2=0x560bfc327220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 0x560bfc327220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 -- 172.21.15.73:0/3577780335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfc2369b0 con 0x560bfc327760 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ad813640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc31c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ad012640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 0x560bfc327220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ac811640 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f839c0cc590 tx=0x7f839c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a62325720271993 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f838f7fe640 1 -- 172.21.15.73:0/3577780335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c020070 con 0x560bfc327760 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f83ad813640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 0x7f83900a5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8398009430 tx=0x7f839802a9e0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f838f7fe640 1 -- 172.21.15.73:0/3577780335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839c009c30 con 0x560bfc327760 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.816+0000 7f838f7fe640 1 -- 172.21.15.73:0/3577780335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c09b3f0 con 0x560bfc327760 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- 172.21.15.73:0/3577780335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 msgr2=0x560bfc329b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f839c0cc590 tx=0x7f839c005680 comp rx=0 tx=0).stop 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- 172.21.15.73:0/3577780335 shutdown_connections 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc329b50 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31cea0 0x560bfc327220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3577780335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc31c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- 172.21.15.73:0/3577780335 >> 172.21.15.73:0/3577780335 conn(0x560bfc318a90 msgr2=0x560bfc318e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- 172.21.15.73:0/3577780335 shutdown_connections 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- 172.21.15.73:0/3577780335 wait complete. 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 -- start start 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.817+0000 7f83b1f84740 1 Processor -- start 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31c580 0x560bfc324650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc324b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560bfc32a330 con 0x560bfc327760 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560bfc205ae0 con 0x560bfc31c580 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560bfc32a4b0 con 0x560bfc31cea0 2024-06-06T07:13:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 10 client.0 _ll_register_callbacks cb 0x560bfc23e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57648/0 (socket says 172.21.15.73:57648) 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 -- 172.21.15.73:0/3658163351 learned_addr learned my addr 172.21.15.73:0/3658163351 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ac811640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc324b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad012640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31c580 0x560bfc324650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 -- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 msgr2=0x560bfc324b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc324b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 -- 172.21.15.73:0/3658163351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31c580 msgr2=0x560bfc324650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31c580 0x560bfc324650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 -- 172.21.15.73:0/3658163351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfbf90210 con 0x560bfc327760 2024-06-06T07:13:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83ad813640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f839802e670 tx=0x7f83980095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2fe1e2b26e76dcf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f838cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8398041050 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f839802f640 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bfc28ff10 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f839805a050 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 10 client.16836 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.818+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560bfc05c620 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560bfc28ff10 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f83980387c0 con 0x560bfc327760 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 10 client.16836 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 20 client.16836 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc337960 0x560bfc339d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 4 client.16836 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bfc289c20 con 0x560bfc337960 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.819+0000 7f83ad012640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc337960 0x560bfc339d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f838cff9640 10 client.16836 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83ad012640 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc337960 0x560bfc339d50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ded8d5e96027c712 server_cookie=983f795f7d775bd1 in_seq=0 out_seq=0 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f838cff9640 1 -- 172.21.15.73:0/3658163351 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560bfc289c20 con 0x560bfc337960 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f838cff9640 10 client.16836 handle_command_reply: tid=0 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 client.16836 shutdown 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 msgr2=0x560bfc3250d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f839802e670 tx=0x7f83980095c0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 shutdown_connections 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc337960 0x560bfc339d50 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc327760 0x560bfc3250d0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc324b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 --2- 172.21.15.73:0/3658163351 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc31c580 0x560bfc324650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f838cff9640 0 client.16836 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.820+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 >> 172.21.15.73:0/3658163351 conn(0x560bfc318a90 msgr2=0x560bfc3137e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.821+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 shutdown_connections 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.821+0000 7f83b1f84740 1 -- 172.21.15.73:0/3658163351 wait complete. 2024-06-06T07:13:19.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.821+0000 7f83b1f84740 20 client.16836 trim_cache size 0 max 16384 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 Processor -- start 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 -- start start 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc33f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc325e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81acc00 con 0x560bfc31cea0 2024-06-06T07:13:19.841 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ae860 con 0x560bfc327760 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83b1f84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ad2d0 con 0x560bfc31c580 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57652/0 (socket says 172.21.15.73:57652) 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 -- 172.21.15.73:0/2973689914 learned_addr learned my addr 172.21.15.73:0/2973689914 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 -- 172.21.15.73:0/2973689914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 msgr2=0x560bfc33f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc33f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 -- 172.21.15.73:0/2973689914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 msgr2=0x560bfc325e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc325e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 -- 172.21.15.73:0/2973689914 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfc33a3b0 con 0x560bfc31cea0 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f83ac811640 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f839c0ccc20 tx=0x7f839c097690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4de877827a85767 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f838effd640 1 -- 172.21.15.73:0/2973689914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c020040 con 0x560bfc31cea0 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f838effd640 1 -- 172.21.15.73:0/2973689914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f839c0cf120 con 0x560bfc31cea0 2024-06-06T07:13:19.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.836+0000 7f838effd640 1 -- 172.21.15.73:0/2973689914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c09b3f0 con 0x560bfc31cea0 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- 172.21.15.73:0/2973689914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 msgr2=0x560bfc33fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f839c0ccc20 tx=0x7f839c097690 comp rx=0 tx=0).stop 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- 172.21.15.73:0/2973689914 shutdown_connections 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc325e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31cea0 0x560bfc33fb80 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2973689914 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31c580 0x560bfc33f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- 172.21.15.73:0/2973689914 >> 172.21.15.73:0/2973689914 conn(0x560bfc318a90 msgr2=0x560bfc31f940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- 172.21.15.73:0/2973689914 shutdown_connections 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- 172.21.15.73:0/2973689914 wait complete. 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 -- start start 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.837+0000 7f83b1f84740 1 Processor -- start 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc32af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc32b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ae130 con 0x560bfc31c580 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81ada00 con 0x560bfc327760 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f83a81aac00 con 0x560bfc31cea0 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 10 client.0 _ll_register_callbacks cb 0x7f839802ece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57660/0 (socket says 172.21.15.73:57660) 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 -- 172.21.15.73:0/2763496633 learned_addr learned my addr 172.21.15.73:0/2763496633 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ac811640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc32af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad813640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc32b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 -- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 msgr2=0x560bfc32af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc32af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 -- 172.21.15.73:0/2763496633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 msgr2=0x560bfc32b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc32b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 -- 172.21.15.73:0/2763496633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfc3188c0 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad813640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc32b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83ad012640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x560bfc33a5b0 tx=0x7f83a0012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f6992991abe226 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f838cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a0010d70 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f83a001a3f0 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.838+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bfc283f70 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f83a0029050 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 10 client.16842 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560bfc3a21a0 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x560bfc283f70 con 0x560bfc31c580 2024-06-06T07:13:19.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f83a0024950 con 0x560bfc31c580 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 10 client.16842 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 20 client.16842 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc3a2980 0x560bfc3a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 4 client.16842 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bfc2018e0 con 0x560bfc3a2980 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.839+0000 7f83ac811640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc3a2980 0x560bfc3a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.840+0000 7f838cff9640 10 client.16842 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:19.846 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.840+0000 7f83ac811640 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc3a2980 0x560bfc3a4da0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f6981bc91f06bf server_cookie=cc2b7c46c52d61cb in_seq=0 out_seq=0 2024-06-06T07:13:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:19 smithi170 ceph-mon[36108]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.866+0000 7f838cff9640 1 -- 172.21.15.73:0/2763496633 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560bfc2018e0 con 0x560bfc3a2980 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.866+0000 7f838cff9640 10 client.16842 handle_command_reply: tid=0 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.866+0000 7f83b1f84740 1 client.16842 shutdown 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.866+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 msgr2=0x560bfc32fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.866+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x560bfc33a5b0 tx=0x7f83a0012b10 comp rx=0 tx=0).stop 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 shutdown_connections 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560bfc3a2980 0x560bfc3a4da0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560bfc327760 0x560bfc32b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f838cff9640 0 client.16842 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560bfc31cea0 0x560bfc32af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 --2- 172.21.15.73:0/2763496633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560bfc31c580 0x560bfc32fb80 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 >> 172.21.15.73:0/2763496633 conn(0x560bfc318a90 msgr2=0x560bfc312f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 shutdown_connections 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 1 -- 172.21.15.73:0/2763496633 wait complete. 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83b1f84740 20 client.16842 trim_cache size 0 max 16384 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:13:19.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 msgr2=0x7f83900a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 0x7f83900a5260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8398009430 tx=0x7f839802a9e0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 msgr2=0x7f83a8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f839c00d770 tx=0x7f839c00dc30 comp rx=0 tx=0).stop 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 shutdown_connections 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f83a810bf70 0x7f83a819c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f83a8103550 0x7f83a8198600 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f83900a2da0 0x7f83900a5260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 --2- 172.21.15.73:0/2923633621 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f83a8102c30 0x7f83a81980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.867+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 >> 172.21.15.73:0/2923633621 conn(0x7f83a8077a20 msgr2=0x7f83a80fd930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.868+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 shutdown_connections 2024-06-06T07:13:19.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:19.868+0000 7f83af276640 1 -- 172.21.15.73:0/2923633621 wait complete. 2024-06-06T07:13:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:19 smithi073 ceph-mon[32303]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:20.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:19 smithi136 ceph-mon[35901]: pgmap v376: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:20.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228748 1.81 MB/sec execute 507 sec latency 1832.551 ms 2024-06-06T07:13:21.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228748 1.81 MB/sec execute 508 sec latency 2832.630 ms 2024-06-06T07:13:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:21 smithi170 ceph-mon[36108]: pgmap v377: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:21 smithi073 ceph-mon[32303]: pgmap v377: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:22.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:21 smithi136 ceph-mon[35901]: pgmap v377: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:22.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228748 1.80 MB/sec execute 509 sec latency 3832.708 ms 2024-06-06T07:13:23.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 228917 1.80 MB/sec execute 510 sec latency 4534.042 ms 2024-06-06T07:13:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:23 smithi170 ceph-mon[36108]: pgmap v378: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:23 smithi073 ceph-mon[32303]: pgmap v378: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:23 smithi136 ceph-mon[35901]: pgmap v378: 129 pgs: 129 active+clean; 135 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 563 KiB/s wr, 71 op/s 2024-06-06T07:13:24.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 229469 1.80 MB/sec execute 511 sec latency 20.120 ms 2024-06-06T07:13:25.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230065 1.81 MB/sec execute 512 sec latency 18.939 ms 2024-06-06T07:13:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:25 smithi170 ceph-mon[36108]: pgmap v379: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 571 KiB/s wr, 92 op/s 2024-06-06T07:13:26.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:25 smithi073 ceph-mon[32303]: pgmap v379: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 571 KiB/s wr, 92 op/s 2024-06-06T07:13:26.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:25 smithi136 ceph-mon[35901]: pgmap v379: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 451 KiB/s rd, 571 KiB/s wr, 92 op/s 2024-06-06T07:13:26.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230565 1.81 MB/sec execute 513 sec latency 15.943 ms 2024-06-06T07:13:27.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:26 smithi073 ceph-mon[32303]: pgmap v380: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:13:27.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:26 smithi136 ceph-mon[35901]: pgmap v380: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:13:27.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:26 smithi170 ceph-mon[36108]: pgmap v380: 129 pgs: 129 active+clean; 132 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 512 KiB/s wr, 74 op/s 2024-06-06T07:13:27.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230672 1.81 MB/sec execute 514 sec latency 795.832 ms 2024-06-06T07:13:28.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230672 1.81 MB/sec execute 515 sec latency 1795.935 ms 2024-06-06T07:13:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:28 smithi073 ceph-mon[32303]: pgmap v381: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-06T07:13:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:28 smithi136 ceph-mon[35901]: pgmap v381: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-06T07:13:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:28 smithi170 ceph-mon[36108]: pgmap v381: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-06T07:13:29.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230672 1.80 MB/sec execute 516 sec latency 2796.055 ms 2024-06-06T07:13:30.457 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230684 1.80 MB/sec execute 517 sec latency 3080.492 ms 2024-06-06T07:13:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:30 smithi073 ceph-mon[32303]: pgmap v382: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:30 smithi136 ceph-mon[35901]: pgmap v382: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:30 smithi170 ceph-mon[36108]: pgmap v382: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:31.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230684 1.80 MB/sec execute 518 sec latency 1650.305 ms 2024-06-06T07:13:32.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230684 1.79 MB/sec execute 519 sec latency 2650.358 ms 2024-06-06T07:13:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 -- 172.21.15.73:0/1092407900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16181008c0 msgr2=0x7f1618100d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 --2- 172.21.15.73:0/1092407900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16181008c0 0x7f1618100d40 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f1608009930 tx=0x7f160802a420 comp rx=0 tx=0).stop 2024-06-06T07:13:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 -- 172.21.15.73:0/1092407900 shutdown_connections 2024-06-06T07:13:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 --2- 172.21.15.73:0/1092407900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1618101280 0x7f161810d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 --2- 172.21.15.73:0/1092407900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16181008c0 0x7f1618100d40 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 --2- 172.21.15.73:0/1092407900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 0x7f1618106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 -- 172.21.15.73:0/1092407900 >> 172.21.15.73:0/1092407900 conn(0x7f1618076c20 msgr2=0x7f1618077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 -- 172.21.15.73:0/1092407900 shutdown_connections 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.969+0000 7f16203dc640 1 -- 172.21.15.73:0/1092407900 wait complete. 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 Processor -- start 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 -- start start 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16181008c0 0x7f16181980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 0x7f161819c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1618110b90 con 0x7f1618101280 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1618110a10 con 0x7f16181008c0 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f16203dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1618110d10 con 0x7f16181068d0 2024-06-06T07:13:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42114/0 (socket says 172.21.15.73:42114) 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 -- 172.21.15.73:0/1747248076 learned_addr learned my addr 172.21.15.73:0/1747248076 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161e178640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16181008c0 0x7f16181980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161e979640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 0x7f161819c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 -- 172.21.15.73:0/1747248076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 msgr2=0x7f161819c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 0x7f161819c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 -- 172.21.15.73:0/1747248076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16181008c0 msgr2=0x7f16181980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16181008c0 0x7f16181980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.970+0000 7f161d977640 1 -- 172.21.15.73:0/1747248076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f161819d0d0 con 0x7f1618101280 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f161d977640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1608004770 tx=0x7f16080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f160f7fe640 1 -- 172.21.15.73:0/1747248076 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f160803f070 con 0x7f1618101280 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f161819d360 con 0x7f1618101280 2024-06-06T07:13:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f160f7fe640 1 -- 172.21.15.73:0/1747248076 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f160803b460 con 0x7f1618101280 2024-06-06T07:13:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f160f7fe640 1 -- 172.21.15.73:0/1747248076 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1608035070 con 0x7f1618101280 2024-06-06T07:13:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.971+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16181a4c50 con 0x7f1618101280 2024-06-06T07:13:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.974+0000 7f160f7fe640 1 -- 172.21.15.73:0/1747248076 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f160803b610 con 0x7f1618101280 2024-06-06T07:13:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.974+0000 7f16231c6740 1 Processor -- start 2024-06-06T07:13:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.974+0000 7f16231c6740 1 -- start start 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2cc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ddb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56390c2304f0 con 0x56390c2cc970 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56390c1ce870 con 0x56390c2db760 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f16231c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56390c22c4b0 con 0x56390c2cc050 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42130/0 (socket says 172.21.15.73:42130) 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 -- 172.21.15.73:0/2475833745 learned_addr learned my addr 172.21.15.73:0/2475833745 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e979640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161d977640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ddb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f160f7fe640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 0x7f16040a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 -- 172.21.15.73:0/2475833745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 msgr2=0x56390c2cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 -- 172.21.15.73:0/2475833745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 msgr2=0x56390c2ddb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ddb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 -- 172.21.15.73:0/2475833745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390c1ea9b0 con 0x56390c2cc970 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161d977640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e979640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 0x7f16040a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f160f7fe640 1 -- 172.21.15.73:0/1747248076 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f16080efbd0 con 0x7f1618101280 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.975+0000 7f161e178640 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f161400bcc0 tx=0x7f16140107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddc3e4c758ce08c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.976+0000 7f160cbfe640 1 -- 172.21.15.73:0/2475833745 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614011440 con 0x56390c2cc970 2024-06-06T07:13:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.976+0000 7f160cbfe640 1 -- 172.21.15.73:0/2475833745 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1614011a90 con 0x56390c2cc970 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.976+0000 7f161e979640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 0x7f16040a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1610005fd0 tx=0x7f1610004a40 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.976+0000 7f160cbfe640 1 -- 172.21.15.73:0/2475833745 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614019aa0 con 0x56390c2cc970 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.976+0000 7f16231c6740 1 -- 172.21.15.73:0/2475833745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 msgr2=0x56390c2db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f161400bcc0 tx=0x7f16140107a0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- 172.21.15.73:0/2475833745 shutdown_connections 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ddb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc970 0x56390c2db220 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- 172.21.15.73:0/2475833745 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- 172.21.15.73:0/2475833745 >> 172.21.15.73:0/2475833745 conn(0x56390c2c8680 msgr2=0x56390c2c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- 172.21.15.73:0/2475833745 shutdown_connections 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- 172.21.15.73:0/2475833745 wait complete. 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- start start 2024-06-06T07:13:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 Processor -- start 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2d86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56390c2de390 con 0x56390c2db760 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56390c1b9ae0 con 0x56390c2cc970 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56390c2de510 con 0x56390c2cc050 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f16231c6740 10 client.0 _ll_register_callbacks cb 0x56390c1f29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.977+0000 7f161e979640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42134/0 (socket says 172.21.15.73:42134) 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 -- 172.21.15.73:0/602159127 learned_addr learned my addr 172.21.15.73:0/602159127 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161d977640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e178640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2d86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 -- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 msgr2=0x56390c2d86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2d86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 -- 172.21.15.73:0/602159127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 msgr2=0x56390c2d8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 -- 172.21.15.73:0/602159127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390bf44210 con 0x56390c2db760 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f161e979640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1610013970 tx=0x7f1610004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22de41194f8f434a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f1601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161001ed70 con 0x56390c2db760 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f1610028470 con 0x56390c2db760 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1614011a90 con 0x56390c2db760 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1610004b20 con 0x56390c2db760 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f16231c6740 10 client.16854 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.978+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56390c010620 con 0x56390c2db760 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1614011a90 con 0x56390c2db760 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f161000f040 con 0x56390c2db760 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 10 client.16854 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 20 client.16854 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c2eb940 0x56390c2edd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 4 client.16854 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56390c243f10 con 0x56390c2eb940 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f161e178640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c2eb940 0x56390c2edd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f1601ffb640 10 client.16854 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.979+0000 7f161e178640 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c2eb940 0x56390c2edd70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9308036e9a14ebcd server_cookie=c9d2d19138322d1 in_seq=0 out_seq=0 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.980+0000 7f1601ffb640 1 -- 172.21.15.73:0/602159127 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56390c243f10 con 0x56390c2eb940 2024-06-06T07:13:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.980+0000 7f1601ffb640 10 client.16854 handle_command_reply: tid=0 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.980+0000 7f16231c6740 1 client.16854 shutdown 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.980+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 msgr2=0x56390c2d9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.980+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1610013970 tx=0x7f1610004ab0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 shutdown_connections 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c2eb940 0x56390c2edd70 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2db760 0x56390c2d9140 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 --2- 172.21.15.73:0/602159127 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc050 0x56390c2d86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 >> 172.21.15.73:0/602159127 conn(0x56390c2c8680 msgr2=0x56390c2ce4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f1601ffb640 0 client.16854 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 shutdown_connections 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 1 -- 172.21.15.73:0/602159127 wait complete. 2024-06-06T07:13:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.981+0000 7f16231c6740 20 client.16854 trim_cache size 0 max 16384 2024-06-06T07:13:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 Processor -- start 2024-06-06T07:13:33.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 -- start start 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 0x56390c2f3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16181acd30 con 0x56390c2cc050 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16181ae990 con 0x56390c2cc970 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f16231c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16181ad400 con 0x56390c2db760 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42140/0 (socket says 172.21.15.73:42140) 2024-06-06T07:13:33.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.996+0000 7f161e178640 1 -- 172.21.15.73:0/1890381929 learned_addr learned my addr 172.21.15.73:0/1890381929 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e979640 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 0x56390c2f3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 -- 172.21.15.73:0/1890381929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 msgr2=0x56390c2f3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 0x56390c2f3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 -- 172.21.15.73:0/1890381929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 msgr2=0x56390c2d74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 -- 172.21.15.73:0/1890381929 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56390c2d6c10 con 0x56390c2cc050 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e979640 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 0x56390c2f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f161e178640 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1614010ca0 tx=0x7f1614010fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0f657b91643785 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f1603fff640 1 -- 172.21.15.73:0/1890381929 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f161401f040 con 0x56390c2cc050 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f1603fff640 1 -- 172.21.15.73:0/1890381929 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140054f0 con 0x56390c2cc050 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.997+0000 7f1603fff640 1 -- 172.21.15.73:0/1890381929 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1614019e30 con 0x56390c2cc050 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- 172.21.15.73:0/1890381929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 msgr2=0x56390c2d6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f1614010ca0 tx=0x7f1614010fd0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- 172.21.15.73:0/1890381929 shutdown_connections 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2db760 0x56390c2f3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2cc970 0x56390c2d74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- 172.21.15.73:0/1890381929 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2d6f90 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- 172.21.15.73:0/1890381929 >> 172.21.15.73:0/1890381929 conn(0x56390c2c8680 msgr2=0x56390c2c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- 172.21.15.73:0/1890381929 shutdown_connections 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- 172.21.15.73:0/1890381929 wait complete. 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- start start 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 Processor -- start 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc970 0x56390c2e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ded20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f16181ae260 con 0x56390c2cc050 2024-06-06T07:13:33.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f16181adb30 con 0x56390c2db760 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f16181aad30 con 0x56390c2cc970 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f16231c6740 10 client.0 _ll_register_callbacks cb 0x7f1610009cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.998+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42148/0 (socket says 172.21.15.73:42148) 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 -- 172.21.15.73:0/3239427751 learned_addr learned my addr 172.21.15.73:0/3239427751 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161d977640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc970 0x56390c2e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 -- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc970 msgr2=0x56390c2e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc970 0x56390c2e3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e979640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ded20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 -- 172.21.15.73:0/3239427751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 msgr2=0x56390c2ded20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ded20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 -- 172.21.15.73:0/3239427751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56390c2df520 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f161e178640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f161400b620 tx=0x7f161400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0eb3ad963415e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f1601ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16140196c0 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f1614005070 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16140054f0 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f1614040050 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f16231c6740 10 client.16860 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x56390c3560d0 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:32.999+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16140054f0 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f161402d020 con 0x56390c2cc050 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 10 client.16860 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:33.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 20 client.16860 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c356600 0x56390c358a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 4 client.16860 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56390c23dc20 con 0x56390c356600 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f161d977640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c356600 0x56390c358a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f161d977640 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c356600 0x56390c358a20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f181e6354af9fe server_cookie=224ad82eb642ef6a in_seq=0 out_seq=0 2024-06-06T07:13:33.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.000+0000 7f1601ffb640 10 client.16860 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f1601ffb640 1 -- 172.21.15.73:0/3239427751 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56390c23dc20 con 0x56390c356600 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f1601ffb640 10 client.16860 handle_command_reply: tid=0 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 client.16860 shutdown 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 msgr2=0x56390c2e38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f161400b620 tx=0x7f161400be30 comp rx=0 tx=0).stop 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 shutdown_connections 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56390c356600 0x56390c358a20 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56390c2db760 0x56390c2ded20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56390c2cc970 0x56390c2e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 --2- 172.21.15.73:0/3239427751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56390c2cc050 0x56390c2e38f0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f1601ffb640 0 client.16860 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 >> 172.21.15.73:0/3239427751 conn(0x56390c2c8680 msgr2=0x56390c2d6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 shutdown_connections 2024-06-06T07:13:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 1 -- 172.21.15.73:0/3239427751 wait complete. 2024-06-06T07:13:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16231c6740 20 client.16860 trim_cache size 0 max 16384 2024-06-06T07:13:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:13:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:13:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.028+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 msgr2=0x7f16040a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 0x7f16040a5260 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1610005fd0 tx=0x7f1610004a40 comp rx=0 tx=0).stop 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 msgr2=0x7f1618198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1608004770 tx=0x7f16080042f0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 shutdown_connections 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f16040a2da0 0x7f16040a5260 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f16181068d0 0x7f161819c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1618101280 0x7f1618198620 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 --2- 172.21.15.73:0/1747248076 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f16181008c0 0x7f16181980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 >> 172.21.15.73:0/1747248076 conn(0x7f1618076c20 msgr2=0x7f1618104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 shutdown_connections 2024-06-06T07:13:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:33.029+0000 7f16203dc640 1 -- 172.21.15.73:0/1747248076 wait complete. 2024-06-06T07:13:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:32 smithi073 ceph-mon[32303]: pgmap v383: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:32 smithi136 ceph-mon[35901]: pgmap v383: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:33.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:32 smithi170 ceph-mon[36108]: pgmap v383: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 596 KiB/s wr, 75 op/s 2024-06-06T07:13:33.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230739 1.79 MB/sec execute 520 sec latency 3356.165 ms 2024-06-06T07:13:34.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230739 1.79 MB/sec execute 521 sec latency 1087.144 ms 2024-06-06T07:13:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:34 smithi073 ceph-mon[32303]: pgmap v384: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-06-06T07:13:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:34 smithi136 ceph-mon[35901]: pgmap v384: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-06-06T07:13:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:34 smithi170 ceph-mon[36108]: pgmap v384: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 687 KiB/s rd, 617 KiB/s wr, 87 op/s 2024-06-06T07:13:35.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230751 1.78 MB/sec execute 522 sec latency 1373.257 ms 2024-06-06T07:13:36.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230751 1.78 MB/sec execute 523 sec latency 1655.818 ms 2024-06-06T07:13:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:36 smithi073 ceph-mon[32303]: pgmap v385: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 548 KiB/s wr, 59 op/s 2024-06-06T07:13:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:36 smithi136 ceph-mon[35901]: pgmap v385: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 548 KiB/s wr, 59 op/s 2024-06-06T07:13:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:36 smithi170 ceph-mon[36108]: pgmap v385: 129 pgs: 129 active+clean; 136 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 450 KiB/s rd, 548 KiB/s wr, 59 op/s 2024-06-06T07:13:37.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230751 1.78 MB/sec execute 524 sec latency 2655.890 ms 2024-06-06T07:13:38.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 230840 1.77 MB/sec execute 525 sec latency 3356.168 ms 2024-06-06T07:13:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:38 smithi073 ceph-mon[32303]: pgmap v386: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 604 KiB/s wr, 74 op/s 2024-06-06T07:13:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:38 smithi136 ceph-mon[35901]: pgmap v386: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 604 KiB/s wr, 74 op/s 2024-06-06T07:13:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:38 smithi170 ceph-mon[36108]: pgmap v386: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 494 KiB/s rd, 604 KiB/s wr, 74 op/s 2024-06-06T07:13:39.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 231075 1.77 MB/sec execute 526 sec latency 15.630 ms 2024-06-06T07:13:40.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 231513 1.77 MB/sec execute 527 sec latency 14.436 ms 2024-06-06T07:13:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:40 smithi073 ceph-mon[32303]: pgmap v387: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:40 smithi136 ceph-mon[35901]: pgmap v387: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:40 smithi170 ceph-mon[36108]: pgmap v387: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:41.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 232008 1.77 MB/sec execute 528 sec latency 16.603 ms 2024-06-06T07:13:42.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 232296 1.77 MB/sec execute 529 sec latency 379.631 ms 2024-06-06T07:13:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:42 smithi073 ceph-mon[32303]: pgmap v388: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:42 smithi136 ceph-mon[35901]: pgmap v388: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:43.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:42 smithi170 ceph-mon[36108]: pgmap v388: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 77 KiB/s wr, 26 op/s 2024-06-06T07:13:43.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 232469 1.77 MB/sec execute 530 sec latency 1078.193 ms 2024-06-06T07:13:44.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 233006 1.77 MB/sec execute 531 sec latency 20.678 ms 2024-06-06T07:13:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:44 smithi073 ceph-mon[32303]: pgmap v389: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 537 KiB/s wr, 86 op/s 2024-06-06T07:13:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:44 smithi136 ceph-mon[35901]: pgmap v389: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 537 KiB/s wr, 86 op/s 2024-06-06T07:13:45.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:44 smithi170 ceph-mon[36108]: pgmap v389: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 422 KiB/s rd, 537 KiB/s wr, 86 op/s 2024-06-06T07:13:45.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 233592 1.78 MB/sec execute 532 sec latency 38.299 ms 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 -- 172.21.15.73:0/2840431997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1068c0 msgr2=0x7fbb1c106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 --2- 172.21.15.73:0/2840431997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c106ca0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c009cd0 tx=0x7fbb0c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 -- 172.21.15.73:0/2840431997 shutdown_connections 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 --2- 172.21.15.73:0/2840431997 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 --2- 172.21.15.73:0/2840431997 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c100900 0x7fbb1c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 --2- 172.21.15.73:0/2840431997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c106ca0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 -- 172.21.15.73:0/2840431997 >> 172.21.15.73:0/2840431997 conn(0x7fbb1c0779f0 msgr2=0x7fbb1c077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 -- 172.21.15.73:0/2840431997 shutdown_connections 2024-06-06T07:13:46.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.125+0000 7fbb24691640 1 -- 172.21.15.73:0/2840431997 wait complete. 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 Processor -- start 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 -- start start 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 0x7fbb1c0fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c0fe2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c110b80 con 0x7fbb1c1012c0 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c110a00 con 0x7fbb1c1068c0 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb24691640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c110d00 con 0x7fbb1c100900 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43776/0 (socket says 172.21.15.73:43776) 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb2242d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 0x7fbb1c0fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.126+0000 7fbb21c2c640 1 -- 172.21.15.73:0/2218965782 learned_addr learned my addr 172.21.15.73:0/2218965782 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c0fe2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 -- 172.21.15.73:0/2218965782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 msgr2=0x7fbb1c0fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 0x7fbb1c0fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 -- 172.21.15.73:0/2218965782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 msgr2=0x7fbb1c0fe2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c0fe2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 -- 172.21.15.73:0/2218965782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1c1a6660 con 0x7fbb1c1012c0 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 0x7fbb1c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb1b7fe640 1 -- 172.21.15.73:0/2218965782 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10017070 con 0x7fbb1c1012c0 2024-06-06T07:13:46.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb1b7fe640 1 -- 172.21.15.73:0/2218965782 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb10009c30 con 0x7fbb1c1012c0 2024-06-06T07:13:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb1b7fe640 1 -- 172.21.15.73:0/2218965782 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000f620 con 0x7fbb1c1012c0 2024-06-06T07:13:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1c1a6950 con 0x7fbb1c1012c0 2024-06-06T07:13:46.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.127+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1c1a6ef0 con 0x7fbb1c1012c0 2024-06-06T07:13:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.130+0000 7fbb1b7fe640 1 -- 172.21.15.73:0/2218965782 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbb100109a0 con 0x7fbb1c1012c0 2024-06-06T07:13:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.130+0000 7fbb1b7fe640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 0x7fbb040a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.130+0000 7fbb27395740 1 Processor -- start 2024-06-06T07:13:46.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 0x7fbb040a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb1b7fe640 1 -- 172.21.15.73:0/2218965782 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7fbb100cf430 con 0x7fbb1c1012c0 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 -- start start 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8721550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 0x5589c872e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5589c86854f0 con 0x5589c8721170 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5589c8623870 con 0x5589c872c350 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb27395740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5589c86814b0 con 0x5589c8721a90 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:32948/0 (socket says 172.21.15.73:32948) 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 -- 172.21.15.73:0/4258092933 learned_addr learned my addr 172.21.15.73:0/4258092933 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:46.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb2242d640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8721550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 0x5589c872e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 -- 172.21.15.73:0/4258092933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 msgr2=0x5589c872e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 0x5589c872e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 0x7fbb040a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006010 tx=0x7fbb0c0099f0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 -- 172.21.15.73:0/4258092933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 msgr2=0x5589c8721550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8721550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb21c2c640 1 -- 172.21.15.73:0/4258092933 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c863f9b0 con 0x5589c8721a90 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.131+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 0x5589c872e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.132+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb100a3c80 tx=0x7fbb10009ea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acb4f33904c737d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.132+0000 7fbb18ff9640 1 -- 172.21.15.73:0/4258092933 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10020040 con 0x5589c8721a90 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.132+0000 7fbb18ff9640 1 -- 172.21.15.73:0/4258092933 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb100bb040 con 0x5589c8721a90 2024-06-06T07:13:46.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.132+0000 7fbb18ff9640 1 -- 172.21.15.73:0/4258092933 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb10097bb0 con 0x5589c8721a90 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- 172.21.15.73:0/4258092933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 msgr2=0x5589c872be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbb100a3c80 tx=0x7fbb10009ea0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- 172.21.15.73:0/4258092933 shutdown_connections 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c872c350 0x5589c872e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8721a90 0x5589c872be10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- 172.21.15.73:0/4258092933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8721550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- 172.21.15.73:0/4258092933 >> 172.21.15.73:0/4258092933 conn(0x5589c871d680 msgr2=0x5589c871da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- 172.21.15.73:0/4258092933 shutdown_connections 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- 172.21.15.73:0/4258092933 wait complete. 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 -- start start 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 Processor -- start 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.139 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.133+0000 7fbb27395740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 0x5589c8729d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb27395740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5589c872efd0 con 0x5589c8721170 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb27395740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5589c860eae0 con 0x5589c8721a90 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb27395740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5589c872f150 con 0x5589c872c350 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb27395740 10 client.0 _ll_register_callbacks cb 0x5589c86479b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43788/0 (socket says 172.21.15.73:43788) 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb22c2e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 0x5589c8729d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 -- 172.21.15.73:0/615515593 learned_addr learned my addr 172.21.15.73:0/615515593 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 -- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 msgr2=0x5589c8729d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 0x5589c8729d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.140 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 -- 172.21.15.73:0/615515593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 msgr2=0x5589c87297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 -- 172.21.15.73:0/615515593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c8399210 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 0x5589c8729d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb2242d640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c003b30 tx=0x7fbb0c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a120768c300e9aef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c042040 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7fbb0c004750 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.134+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb100bb040 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.135+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbb0c05a050 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.135+0000 7fbb27395740 10 client.16872 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.135+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5589c8465620 con 0x5589c8721170 2024-06-06T07:13:46.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.135+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb100bb040 con 0x5589c8721170 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.135+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fbb0c04f020 con 0x5589c8721170 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 10 client.16872 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 10 client.16872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 20 client.16872 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c873c4d0 0x5589c873e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 4 client.16872 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589c8698f10 con 0x5589c873c4d0 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c873c4d0 0x5589c873e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb01ffb640 10 client.16872 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:46.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.136+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c873c4d0 0x5589c873e8c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bece8474d8df3b server_cookie=ce8f01b0633732e3 in_seq=0 out_seq=0 2024-06-06T07:13:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb01ffb640 1 -- 172.21.15.73:0/615515593 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5589c8698f10 con 0x5589c873c4d0 2024-06-06T07:13:46.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb01ffb640 10 client.16872 handle_command_reply: tid=0 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 client.16872 shutdown 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 msgr2=0x5589c8729290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c003b30 tx=0x7fbb0c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 shutdown_connections 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c873c4d0 0x5589c873e8c0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c872c350 0x5589c8729d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 --2- 172.21.15.73:0/615515593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c8721170 0x5589c8729290 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 >> 172.21.15.73:0/615515593 conn(0x5589c871d680 msgr2=0x5589c87233b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb01ffb640 0 client.16872 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 shutdown_connections 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.137+0000 7fbb27395740 1 -- 172.21.15.73:0/615515593 wait complete. 2024-06-06T07:13:46.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.138+0000 7fbb27395740 20 client.16872 trim_cache size 0 max 16384 2024-06-06T07:13:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 Processor -- start 2024-06-06T07:13:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 -- start start 2024-06-06T07:13:46.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c872b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b0fe0 con 0x5589c872c350 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b2c40 con 0x5589c8721a90 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb27395740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b16b0 con 0x5589c8724c00 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43796/0 (socket says 172.21.15.73:43796) 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb21c2c640 1 -- 172.21.15.73:0/1305630985 learned_addr learned my addr 172.21.15.73:0/1305630985 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c872b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb2242d640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 -- 172.21.15.73:0/1305630985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 msgr2=0x5589c87189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.157 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 -- 172.21.15.73:0/1305630985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 msgr2=0x5589c8723b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.155+0000 7fbb22c2e640 1 -- 172.21.15.73:0/1305630985 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589c8724070 con 0x5589c8724c00 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.156+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.156+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c872b1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5589c871e810 tx=0x7fbb140128e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22a46643a3dac0d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.156+0000 7fbb03fff640 1 -- 172.21.15.73:0/1305630985 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb14005020 con 0x5589c8724c00 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.156+0000 7fbb03fff640 1 -- 172.21.15.73:0/1305630985 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb140047a0 con 0x5589c8724c00 2024-06-06T07:13:46.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.156+0000 7fbb03fff640 1 -- 172.21.15.73:0/1305630985 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400f860 con 0x5589c8724c00 2024-06-06T07:13:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- 172.21.15.73:0/1305630985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 msgr2=0x5589c872b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c872b1d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5589c871e810 tx=0x7fbb140128e0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- 172.21.15.73:0/1305630985 shutdown_connections 2024-06-06T07:13:46.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c872b1d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8723b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- 172.21.15.73:0/1305630985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c87189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- 172.21.15.73:0/1305630985 >> 172.21.15.73:0/1305630985 conn(0x5589c871d680 msgr2=0x5589c872aac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- 172.21.15.73:0/1305630985 shutdown_connections 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- 172.21.15.73:0/1305630985 wait complete. 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- start start 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 Processor -- start 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c8725c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8726170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b2510 con 0x5589c872c350 2024-06-06T07:13:46.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.157+0000 7fbb27395740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1b1de0 con 0x5589c8721a90 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb27395740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbb1c1aefe0 con 0x5589c8724c00 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb27395740 10 client.0 _ll_register_callbacks cb 0x7fbb0c0319b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb2242d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8726170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb21c2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c8725c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb2242d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56334/0 (socket says 172.21.15.73:56334) 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb2242d640 1 -- 172.21.15.73:0/2718444601 learned_addr learned my addr 172.21.15.73:0/2718444601 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 -- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 msgr2=0x5589c8725c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c8725c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 -- 172.21.15.73:0/2718444601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 msgr2=0x5589c8734820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 -- 172.21.15.73:0/2718444601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589c87266b0 con 0x5589c872c350 2024-06-06T07:13:46.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb21c2c640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c8725c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb22c2e640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8726170 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009bd0 tx=0x7fbb140052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c80d1fee8b2f212 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb01ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1400f5e0 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7fbb14018070 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb140047a0 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbb14031050 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb27395740 10 client.16878 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.158+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5589c87a6e80 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbb140047a0 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fbb1401b640 con 0x5589c872c350 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 10 client.16878 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 10 client.16878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:46.164 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 20 client.16878 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c87a7540 0x5589c87a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 4 client.16878 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589c8692c20 con 0x5589c87a7540 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c87a7540 0x5589c87a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb01ffb640 10 client.16878 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:46.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.159+0000 7fbb2242d640 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c87a7540 0x5589c87a9960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=476f5c82a288d518 server_cookie=16a1507496cd380a in_seq=0 out_seq=0 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb01ffb640 1 -- 172.21.15.73:0/2718444601 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5589c8692c20 con 0x5589c87a7540 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb01ffb640 10 client.16878 handle_command_reply: tid=0 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 client.16878 shutdown 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 msgr2=0x5589c8726170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8726170 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fbb14009bd0 tx=0x7fbb140052d0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 shutdown_connections 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5589c87a7540 0x5589c87a9960 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5589c872c350 0x5589c8726170 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5589c8724c00 0x5589c8725c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 --2- 172.21.15.73:0/2718444601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5589c8721a90 0x5589c8734820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 >> 172.21.15.73:0/2718444601 conn(0x5589c871d680 msgr2=0x5589c871c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb01ffb640 0 client.16878 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 shutdown_connections 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 1 -- 172.21.15.73:0/2718444601 wait complete. 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.198+0000 7fbb27395740 20 client.16878 trim_cache size 0 max 16384 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:13:46.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 msgr2=0x7fbb040a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 0x7fbb040a52e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006010 tx=0x7fbb0c0099f0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 msgr2=0x7fbb1c0fdda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000d950 tx=0x7fbb1000de10 comp rx=0 tx=0).stop 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 shutdown_connections 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbb1c1068c0 0x7fbb1c0fe2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb1c1012c0 0x7fbb1c0fdda0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbb040a2e20 0x7fbb040a52e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 --2- 172.21.15.73:0/2218965782 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbb1c100900 0x7fbb1c0fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 >> 172.21.15.73:0/2218965782 conn(0x7fbb1c0779f0 msgr2=0x7fbb1c0ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 shutdown_connections 2024-06-06T07:13:46.202 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:46.199+0000 7fbb24691640 1 -- 172.21.15.73:0/2218965782 wait complete. 2024-06-06T07:13:46.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:45 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:13:46.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:45 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:13:46.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:45 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:13:46.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 233981 1.78 MB/sec execute 533 sec latency 27.851 ms 2024-06-06T07:13:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:46 smithi073 ceph-mon[32303]: pgmap v390: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 516 KiB/s wr, 74 op/s 2024-06-06T07:13:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:46 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:13:47.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:46 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:13:47.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:46 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:13:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:46 smithi136 ceph-mon[35901]: pgmap v390: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 516 KiB/s wr, 74 op/s 2024-06-06T07:13:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:46 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:13:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:46 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:13:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:46 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:13:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:46 smithi170 ceph-mon[36108]: pgmap v390: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 516 KiB/s wr, 74 op/s 2024-06-06T07:13:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:46 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:13:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:46 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:13:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:46 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:13:47.458 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234220 1.78 MB/sec execute 534 sec latency 578.727 ms 2024-06-06T07:13:48.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234220 1.77 MB/sec execute 535 sec latency 1578.825 ms 2024-06-06T07:13:49.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:48 smithi073 ceph-mon[32303]: pgmap v391: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:13:49.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:48 smithi136 ceph-mon[35901]: pgmap v391: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:13:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:48 smithi170 ceph-mon[36108]: pgmap v391: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 863 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-06T07:13:49.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234220 1.77 MB/sec execute 536 sec latency 2578.922 ms 2024-06-06T07:13:50.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234287 1.77 MB/sec execute 537 sec latency 2862.469 ms 2024-06-06T07:13:51.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:50 smithi073 ceph-mon[32303]: pgmap v392: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:51.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:50 smithi136 ceph-mon[35901]: pgmap v392: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:50 smithi170 ceph-mon[36108]: pgmap v392: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:51.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234287 1.77 MB/sec execute 538 sec latency 1463.808 ms 2024-06-06T07:13:52.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234287 1.76 MB/sec execute 539 sec latency 2463.856 ms 2024-06-06T07:13:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:52 smithi073 ceph-mon[32303]: pgmap v393: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:52 smithi136 ceph-mon[35901]: pgmap v393: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:52 smithi170 ceph-mon[36108]: pgmap v393: 129 pgs: 129 active+clean; 137 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 819 KiB/s rd, 1009 KiB/s wr, 115 op/s 2024-06-06T07:13:53.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234287 1.76 MB/sec execute 540 sec latency 3463.974 ms 2024-06-06T07:13:54.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234287 1.76 MB/sec execute 541 sec latency 4464.097 ms 2024-06-06T07:13:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:54 smithi073 ceph-mon[32303]: pgmap v394: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-06-06T07:13:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:54 smithi136 ceph-mon[35901]: pgmap v394: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-06-06T07:13:55.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:54 smithi170 ceph-mon[36108]: pgmap v394: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.0 MiB/s wr, 127 op/s 2024-06-06T07:13:55.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234467 1.75 MB/sec execute 542 sec latency 4750.092 ms 2024-06-06T07:13:56.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 234729 1.75 MB/sec execute 543 sec latency 15.151 ms 2024-06-06T07:13:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:56 smithi073 ceph-mon[32303]: pgmap v395: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-06-06T07:13:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:56 smithi136 ceph-mon[35901]: pgmap v395: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-06-06T07:13:57.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:56 smithi170 ceph-mon[36108]: pgmap v395: 129 pgs: 129 active+clean; 137 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 581 KiB/s wr, 68 op/s 2024-06-06T07:13:57.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235186 1.75 MB/sec execute 544 sec latency 17.791 ms 2024-06-06T07:13:58.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235678 1.75 MB/sec execute 545 sec latency 20.097 ms 2024-06-06T07:13:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:13:58 smithi073 ceph-mon[32303]: pgmap v396: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1007 KiB/s wr, 112 op/s 2024-06-06T07:13:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.286+0000 7f8e5b777640 1 -- 172.21.15.73:0/15152424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e54103570 msgr2=0x7f8e5410ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.286+0000 7f8e5b777640 1 --2- 172.21.15.73:0/15152424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e54103570 0x7f8e5410ba50 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009900 tx=0x7f8e4402a480 comp rx=0 tx=0).stop 2024-06-06T07:13:59.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.286+0000 7f8e5b777640 1 -- 172.21.15.73:0/15152424 shutdown_connections 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.286+0000 7f8e5b777640 1 --2- 172.21.15.73:0/15152424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e5410bf90 0x7f8e5410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.286+0000 7f8e5b777640 1 --2- 172.21.15.73:0/15152424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e54103570 0x7f8e5410ba50 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 --2- 172.21.15.73:0/15152424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54102c50 0x7f8e54103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- 172.21.15.73:0/15152424 >> 172.21.15.73:0/15152424 conn(0x7f8e54077a20 msgr2=0x7f8e54077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- 172.21.15.73:0/15152424 shutdown_connections 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- 172.21.15.73:0/15152424 wait complete. 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 Processor -- start 2024-06-06T07:13:59.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- start start 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e54102c50 0x7f8e541981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54103570 0x7f8e541986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54111c10 con 0x7f8e5410bf90 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54111a90 con 0x7f8e54102c50 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.287+0000 7f8e5b777640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54111d90 con 0x7f8e54103570 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48492/0 (socket says 172.21.15.73:48492) 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 -- 172.21.15.73:0/1374564018 learned_addr learned my addr 172.21.15.73:0/1374564018 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59513640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e54102c50 0x7f8e541981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e58d12640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54103570 0x7f8e541986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 -- 172.21.15.73:0/1374564018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54103570 msgr2=0x7f8e541986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54103570 0x7f8e541986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 -- 172.21.15.73:0/1374564018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e54102c50 msgr2=0x7f8e541981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e54102c50 0x7f8e541981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 -- 172.21.15.73:0/1374564018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5419d1a0 con 0x7f8e5410bf90 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e59d14640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00cad0 tx=0x7f8e4c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e4a7fc640 1 -- 172.21.15.73:0/1374564018 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00ae20 con 0x7f8e5410bf90 2024-06-06T07:13:59.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e4a7fc640 1 -- 172.21.15.73:0/1374564018 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e4c005020 con 0x7f8e5410bf90 2024-06-06T07:13:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e4a7fc640 1 -- 172.21.15.73:0/1374564018 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00f6b0 con 0x7f8e5410bf90 2024-06-06T07:13:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e5419d490 con 0x7f8e5410bf90 2024-06-06T07:13:59.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.288+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e541a4d20 con 0x7f8e5410bf90 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.291+0000 7f8e4a7fc640 1 -- 172.21.15.73:0/1374564018 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8e4c0051d0 con 0x7f8e5410bf90 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.291+0000 7f8e5e565740 1 Processor -- start 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.291+0000 7f8e5e565740 1 -- start start 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214dfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5623214324f0 con 0x5623214ce070 2024-06-06T07:13:59.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623213d0870 con 0x5623214ce990 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e5e565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56232142e4b0 con 0x5623214dd760 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48500/0 (socket says 172.21.15.73:48500) 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 -- 172.21.15.73:0/1603211043 learned_addr learned my addr 172.21.15.73:0/1603211043 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e4a7fc640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 0x7f8e400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e59513640 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214dfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 -- 172.21.15.73:0/1603211043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 msgr2=0x5623214dfb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214dfb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e59d14640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 0x7f8e400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 -- 172.21.15.73:0/1603211043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 msgr2=0x5623214dd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 -- 172.21.15.73:0/1603211043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623213ec9b0 con 0x5623214ce070 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e4a7fc640 1 -- 172.21.15.73:0/1374564018 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f8e4c0cf850 con 0x7f8e5410bf90 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e58d12640 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005940 tx=0x7f8e440058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=495022ee330b3498 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e3f7fe640 1 -- 172.21.15.73:0/1603211043 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44035050 con 0x5623214ce070 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.292+0000 7f8e59d14640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 0x7f8e400a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c09b660 tx=0x7f8e4c005c50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e3f7fe640 1 -- 172.21.15.73:0/1603211043 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e440316f0 con 0x5623214ce070 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e3f7fe640 1 -- 172.21.15.73:0/1603211043 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e440046c0 con 0x5623214ce070 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 -- 172.21.15.73:0/1603211043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 msgr2=0x5623214ce450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005940 tx=0x7f8e440058b0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 -- 172.21.15.73:0/1603211043 shutdown_connections 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214dfb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 --2- 172.21.15.73:0/1603211043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214ce450 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 -- 172.21.15.73:0/1603211043 >> 172.21.15.73:0/1603211043 conn(0x5623214ca680 msgr2=0x5623214caa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 -- 172.21.15.73:0/1603211043 shutdown_connections 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.293+0000 7f8e5e565740 1 -- 172.21.15.73:0/1603211043 wait complete. 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 -- start start 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 Processor -- start 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214db100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5623214e0330 con 0x5623214ce070 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5623213bbae0 con 0x5623214ce990 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5623214e04b0 con 0x5623214dd760 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e5e565740 10 client.0 _ll_register_callbacks cb 0x5623213f49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48506/0 (socket says 172.21.15.73:48506) 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 -- 172.21.15.73:0/389638642 learned_addr learned my addr 172.21.15.73:0/389638642 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e59513640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214db100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 -- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 msgr2=0x5623214db100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214db100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 -- 172.21.15.73:0/389638642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 msgr2=0x5623214dabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.294+0000 7f8e58d12640 1 -- 172.21.15.73:0/389638642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562321146210 con 0x5623214ce070 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e58d12640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8e44032c10 tx=0x7f8e44005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60920302a1e9560 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44035040 con 0x5623214ce070 2024-06-06T07:13:59.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f8e440036b0 con 0x5623214ce070 2024-06-06T07:13:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e440316f0 con 0x5623214ce070 2024-06-06T07:13:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8e44058050 con 0x5623214ce070 2024-06-06T07:13:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 10 client.16890 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x562321212620 con 0x5623214ce070 2024-06-06T07:13:59.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e440316f0 con 0x5623214ce070 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f8e4403a9e0 con 0x5623214ce070 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 10 client.16890 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 20 client.16890 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5623214ed730 0x5623214efb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 4 client.16890 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562321445f10 con 0x5623214ed730 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.295+0000 7f8e59513640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5623214ed730 0x5623214efb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.296+0000 7f8e59513640 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5623214ed730 0x5623214efb20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=704e59078e364e1c server_cookie=c58844aed48c810c in_seq=0 out_seq=0 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.296+0000 7f8e3cff9640 10 client.16890 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:59.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.296+0000 7f8e3cff9640 1 -- 172.21.15.73:0/389638642 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562321445f10 con 0x5623214ed730 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e3cff9640 10 client.16890 handle_command_reply: tid=0 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 client.16890 shutdown 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 msgr2=0x5623214da680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f8e44032c10 tx=0x7f8e44005d50 comp rx=0 tx=0).stop 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 shutdown_connections 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5623214ed730 0x5623214efb20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x5623214db100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e3cff9640 0 client.16890 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:59.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214dabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 --2- 172.21.15.73:0/389638642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214da680 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 >> 172.21.15.73:0/389638642 conn(0x5623214ca680 msgr2=0x5623214d04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 shutdown_connections 2024-06-06T07:13:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 1 -- 172.21.15.73:0/389638642 wait complete. 2024-06-06T07:13:59.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.297+0000 7f8e5e565740 20 client.16890 trim_cache size 0 max 16384 2024-06-06T07:13:59.311 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:13:58 smithi136 ceph-mon[35901]: pgmap v396: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1007 KiB/s wr, 112 op/s 2024-06-06T07:13:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 Processor -- start 2024-06-06T07:13:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 -- start start 2024-06-06T07:13:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce070 0x5623214c9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214ce990 0x5623214f5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541acbb0 con 0x5623214dd760 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541ae810 con 0x5623214ce070 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e5e565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541ad280 con 0x5623214ce990 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48520/0 (socket says 172.21.15.73:48520) 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 -- 172.21.15.73:0/2253935911 learned_addr learned my addr 172.21.15.73:0/2253935911 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 -- 172.21.15.73:0/2253935911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214ce990 msgr2=0x5623214f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214ce990 0x5623214f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 -- 172.21.15.73:0/2253935911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce070 msgr2=0x5623214c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.319 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce070 0x5623214c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.312+0000 7f8e58d12640 1 -- 172.21.15.73:0/2253935911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623214ceed0 con 0x5623214dd760 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8e44033000 tx=0x7f8e440043d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22053dd4845fe8e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e3effd640 1 -- 172.21.15.73:0/2253935911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44061070 con 0x5623214dd760 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e3effd640 1 -- 172.21.15.73:0/2253935911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e4404f070 con 0x5623214dd760 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e3effd640 1 -- 172.21.15.73:0/2253935911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44035070 con 0x5623214dd760 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 -- 172.21.15.73:0/2253935911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 msgr2=0x5623214f56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f8e44033000 tx=0x7f8e440043d0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.320 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 -- 172.21.15.73:0/2253935911 shutdown_connections 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214dd760 0x5623214f56c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214ce990 0x5623214f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2253935911 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce070 0x5623214c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 -- 172.21.15.73:0/2253935911 >> 172.21.15.73:0/2253935911 conn(0x5623214ca680 msgr2=0x5623214c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 -- 172.21.15.73:0/2253935911 shutdown_connections 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.313+0000 7f8e5e565740 1 -- 172.21.15.73:0/2253935911 wait complete. 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 -- start start 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 Processor -- start 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.321 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214e2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x562321557150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541ae0e0 con 0x5623214ce070 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541ad9b0 con 0x5623214ce990 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8e541aabb0 con 0x5623214dd760 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e5e565740 10 client.0 _ll_register_callbacks cb 0x7f8e440314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48522/0 (socket says 172.21.15.73:48522) 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 -- 172.21.15.73:0/2037690290 learned_addr learned my addr 172.21.15.73:0/2037690290 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:13:59.322 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e59513640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214e2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 -- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 msgr2=0x562321557150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x562321557150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 -- 172.21.15.73:0/2037690290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 msgr2=0x5623214e2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214e2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.314+0000 7f8e58d12640 1 -- 172.21.15.73:0/2037690290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5623214c5f70 con 0x5623214ce070 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e58d12640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005ec0 tx=0x7f8e4404ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3f8dcf8ff1814a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44035070 con 0x5623214ce070 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 9554+0+0 (secure 0 0 0) 0x7f8e44032d80 con 0x5623214ce070 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e4404f070 con 0x5623214ce070 2024-06-06T07:13:59.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f8e44061040 con 0x5623214ce070 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 10 client.16896 fetch_fsmap learned FSMap version 48 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5623215586d0 con 0x5623214ce070 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e4404f070 con 0x5623214ce070 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f8e4404bc40 con 0x5623214ce070 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 10 client.16896 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 20 client.16896 populate_metadata read hostname 'smithi073' 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562321558ce0 0x56232155b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:13:59.324 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 4 client.16896 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:13:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.315+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56232143fc20 con 0x562321558ce0 2024-06-06T07:13:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.316+0000 7f8e59513640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562321558ce0 0x56232155b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:13:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.316+0000 7f8e59513640 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562321558ce0 0x56232155b100 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513e1bea443e6b38 server_cookie=403cfd3d4738b31f in_seq=0 out_seq=0 2024-06-06T07:13:59.325 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.316+0000 7f8e3cff9640 10 client.16896 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e3cff9640 1 -- 172.21.15.73:0/2037690290 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56232143fc20 con 0x562321558ce0 2024-06-06T07:13:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e3cff9640 10 client.16896 handle_command_reply: tid=0 2024-06-06T07:13:59.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 client.16896 shutdown 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 msgr2=0x5623214e1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f8e44005ec0 tx=0x7f8e4404ba50 comp rx=0 tx=0).stop 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 shutdown_connections 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562321558ce0 0x56232155b100 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5623214dd760 0x562321557150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5623214ce990 0x5623214e2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e3cff9640 0 client.16896 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 --2- 172.21.15.73:0/2037690290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5623214ce070 0x5623214e1bd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 >> 172.21.15.73:0/2037690290 conn(0x5623214ca680 msgr2=0x5623214c97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.341 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 shutdown_connections 2024-06-06T07:13:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 1 -- 172.21.15.73:0/2037690290 wait complete. 2024-06-06T07:13:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.339+0000 7f8e5e565740 20 client.16896 trim_cache size 0 max 16384 2024-06-06T07:13:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:13:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:13:59.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:13:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 msgr2=0x7f8e400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 0x7f8e400a5260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c09b660 tx=0x7f8e4c005c50 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 msgr2=0x7f8e5419ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c00cad0 tx=0x7f8e4c00cf90 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 shutdown_connections 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e5410bf90 0x7f8e5419ca70 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8e400a2da0 0x7f8e400a5260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8e54103570 0x7f8e541986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 --2- 172.21.15.73:0/1374564018 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8e54102c50 0x7f8e541981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 >> 172.21.15.73:0/1374564018 conn(0x7f8e54077a20 msgr2=0x7f8e540fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 shutdown_connections 2024-06-06T07:13:59.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:13:59.340+0000 7f8e5b777640 1 -- 172.21.15.73:0/1374564018 wait complete. 2024-06-06T07:13:59.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:13:58 smithi170 ceph-mon[36108]: pgmap v396: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 650 KiB/s rd, 1007 KiB/s wr, 112 op/s 2024-06-06T07:13:59.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235839 1.75 MB/sec execute 546 sec latency 679.511 ms 2024-06-06T07:14:00.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235839 1.75 MB/sec execute 547 sec latency 1679.559 ms 2024-06-06T07:14:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:00 smithi073 ceph-mon[32303]: pgmap v397: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:00 smithi136 ceph-mon[35901]: pgmap v397: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:00 smithi170 ceph-mon[36108]: pgmap v397: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:01.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235839 1.75 MB/sec execute 548 sec latency 2679.654 ms 2024-06-06T07:14:02.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 235839 1.74 MB/sec execute 549 sec latency 3679.751 ms 2024-06-06T07:14:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:02 smithi073 ceph-mon[32303]: pgmap v398: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:02 smithi136 ceph-mon[35901]: pgmap v398: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:03.350 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:02 smithi170 ceph-mon[36108]: pgmap v398: 129 pgs: 129 active+clean; 140 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 458 KiB/s wr, 57 op/s 2024-06-06T07:14:03.459 INFO:tasks.workunit.client.0.smithi073.stdout: 1 236008 1.74 MB/sec execute 550 sec latency 4380.482 ms 2024-06-06T07:14:04.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 236408 1.74 MB/sec execute 551 sec latency 342.669 ms 2024-06-06T07:14:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:04 smithi073 ceph-mon[32303]: pgmap v399: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 540 KiB/s wr, 88 op/s 2024-06-06T07:14:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:04 smithi073 ceph-mon[32303]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T07:14:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:04 smithi136 ceph-mon[35901]: pgmap v399: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 540 KiB/s wr, 88 op/s 2024-06-06T07:14:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:04 smithi136 ceph-mon[35901]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T07:14:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:04 smithi170 ceph-mon[36108]: pgmap v399: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 540 KiB/s wr, 88 op/s 2024-06-06T07:14:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:04 smithi170 ceph-mon[36108]: osdmap e117: 12 total, 12 up, 12 in 2024-06-06T07:14:05.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 236719 1.74 MB/sec execute 552 sec latency 625.020 ms 2024-06-06T07:14:06.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237430 1.75 MB/sec execute 553 sec latency 15.159 ms 2024-06-06T07:14:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:06 smithi073 ceph-mon[32303]: pgmap v401: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 610 KiB/s wr, 90 op/s 2024-06-06T07:14:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:06 smithi136 ceph-mon[35901]: pgmap v401: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 610 KiB/s wr, 90 op/s 2024-06-06T07:14:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:06 smithi170 ceph-mon[36108]: pgmap v401: 129 pgs: 129 active+clean; 134 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 476 KiB/s rd, 610 KiB/s wr, 90 op/s 2024-06-06T07:14:07.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237764 1.75 MB/sec execute 554 sec latency 153.064 ms 2024-06-06T07:14:08.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237764 1.75 MB/sec execute 555 sec latency 1153.111 ms 2024-06-06T07:14:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:08 smithi073 ceph-mon[32303]: pgmap v402: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:08 smithi136 ceph-mon[35901]: pgmap v402: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:08 smithi170 ceph-mon[36108]: pgmap v402: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:09.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237764 1.74 MB/sec execute 556 sec latency 2153.175 ms 2024-06-06T07:14:10.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237776 1.74 MB/sec execute 557 sec latency 2434.794 ms 2024-06-06T07:14:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:10 smithi073 ceph-mon[32303]: pgmap v403: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:10 smithi136 ceph-mon[35901]: pgmap v403: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:10 smithi170 ceph-mon[36108]: pgmap v403: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:11.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237776 1.74 MB/sec execute 558 sec latency 1651.617 ms 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 -- 172.21.15.73:0/3681083830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 msgr2=0x7f7108100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 --2- 172.21.15.73:0/3681083830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108100d50 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f70fc009930 tx=0x7f70fc02a420 comp rx=0 tx=0).stop 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 -- 172.21.15.73:0/3681083830 shutdown_connections 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 --2- 172.21.15.73:0/3681083830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108101290 0x7f710810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 --2- 172.21.15.73:0/3681083830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108100d50 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 --2- 172.21.15.73:0/3681083830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108106890 0x7f7108106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 -- 172.21.15.73:0/3681083830 >> 172.21.15.73:0/3681083830 conn(0x7f7108077a20 msgr2=0x7f7108077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 -- 172.21.15.73:0/3681083830 shutdown_connections 2024-06-06T07:14:12.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.435+0000 7f710c82a640 1 -- 172.21.15.73:0/3681083830 wait complete. 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 Processor -- start 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 -- start start 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108101290 0x7f7108198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 0x7f710819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7108111c50 con 0x7f71081008d0 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7108111ad0 con 0x7f7108106890 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710c82a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7108111dd0 con 0x7f7108101290 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43186/0 (socket says 172.21.15.73:43186) 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 -- 172.21.15.73:0/3403944284 learned_addr learned my addr 172.21.15.73:0/3403944284 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f7106d9d640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 0x7f710819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f7105d9b640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108101290 0x7f7108198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 -- 172.21.15.73:0/3403944284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108101290 msgr2=0x7f7108198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108101290 0x7f7108198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 -- 172.21.15.73:0/3403944284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 msgr2=0x7f710819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 0x7f710819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f710659c640 1 -- 172.21.15.73:0/3403944284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710819d020 con 0x7f71081008d0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.436+0000 7f7106d9d640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 0x7f710819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f710659c640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c920 tx=0x7f70f800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f70f77fe640 1 -- 172.21.15.73:0/3403944284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8007c70 con 0x7f71081008d0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710819d310 con 0x7f71081008d0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f70f77fe640 1 -- 172.21.15.73:0/3403944284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70f8007e10 con 0x7f71081008d0 2024-06-06T07:14:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f70f77fe640 1 -- 172.21.15.73:0/3403944284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8018820 con 0x7f71081008d0 2024-06-06T07:14:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.437+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081a4ba0 con 0x7f71081008d0 2024-06-06T07:14:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.439+0000 7f70f77fe640 1 -- 172.21.15.73:0/3403944284 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f70f80189c0 con 0x7f71081008d0 2024-06-06T07:14:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 Processor -- start 2024-06-06T07:14:12.447 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 -- start start 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d62550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3cc64f0 con 0x55c6f3d62a90 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3c64870 con 0x55c6f3d6d350 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710f615740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3cc24b0 con 0x55c6f3d62170 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f70f77fe640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 0x7f70f00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.440+0000 7f710659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43190/0 (socket says 172.21.15.73:43190) 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 -- 172.21.15.73:0/889876237 learned_addr learned my addr 172.21.15.73:0/889876237 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f7105d9b640 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f7106d9d640 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d62550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f7106d9d640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 0x7f70f00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f70f77fe640 1 -- 172.21.15.73:0/3403944284 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7f70f80cfa70 con 0x7f71081008d0 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 -- 172.21.15.73:0/889876237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 msgr2=0x55c6f3d62550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d62550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 -- 172.21.15.73:0/889876237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 msgr2=0x55c6f3d6f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 -- 172.21.15.73:0/889876237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f3c809b0 con 0x55c6f3d62a90 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f7106d9d640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 0x7f70f00a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7100007310 tx=0x7f71000072a0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f710659c640 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f70f8017000 tx=0x7f70f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e85f124bd559a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f70f4ff9640 1 -- 172.21.15.73:0/889876237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f80cfe60 con 0x55c6f3d62a90 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f70f4ff9640 1 -- 172.21.15.73:0/889876237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70f800fb00 con 0x55c6f3d62a90 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.441+0000 7f70f4ff9640 1 -- 172.21.15.73:0/889876237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8098140 con 0x55c6f3d62a90 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- 172.21.15.73:0/889876237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 msgr2=0x55c6f3d6ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f70f8017000 tx=0x7f70f8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- 172.21.15.73:0/889876237 shutdown_connections 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6ce10 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 --2- 172.21.15.73:0/889876237 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d62550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- 172.21.15.73:0/889876237 >> 172.21.15.73:0/889876237 conn(0x55c6f3d5e680 msgr2=0x55c6f3d5ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- 172.21.15.73:0/889876237 shutdown_connections 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- 172.21.15.73:0/889876237 wait complete. 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.442+0000 7f710f615740 1 -- start start 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 Processor -- start 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d6a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3d6ff20 con 0x55c6f3d62a90 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3c4fae0 con 0x55c6f3d6d350 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c6f3d700a0 con 0x55c6f3d62170 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710f615740 10 client.0 _ll_register_callbacks cb 0x55c6f3c889b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43196/0 (socket says 172.21.15.73:43196) 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 -- 172.21.15.73:0/2563267063 learned_addr learned my addr 172.21.15.73:0/2563267063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f710659c640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d6a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7106d9d640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 -- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 msgr2=0x55c6f3d6a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d6a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 -- 172.21.15.73:0/2563267063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 msgr2=0x55c6f3d6ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 -- 172.21.15.73:0/2563267063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f39da210 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f7105d9b640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f70fc02a940 tx=0x7f70fc031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260c8f726d09b8ed server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.443+0000 7f70edffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc035040 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7f70fc004300 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6f3cd9f10 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f70fc04e050 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 10 client.16908 fetch_fsmap learned FSMap version 48 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c6f3aa6620 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c6f3cd9f10 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f70fc0455a0 con 0x55c6f3d62a90 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 10 client.16908 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:14:12.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 20 client.16908 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3d7d4f0 0x55c6f3d7f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 4 client.16908 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.444+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6f3cd3c20 con 0x55c6f3d7d4f0 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.445+0000 7f710659c640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3d7d4f0 0x55c6f3d7f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.445+0000 7f70edffb640 10 client.16908 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.445+0000 7f710659c640 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3d7d4f0 0x55c6f3d7f920 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50cf521d72f42e84 server_cookie=4f48fcf6448c9aae in_seq=0 out_seq=0 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f70edffb640 1 -- 172.21.15.73:0/2563267063 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c6f3cd3c20 con 0x55c6f3d7d4f0 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f70edffb640 10 client.16908 handle_command_reply: tid=0 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 client.16908 shutdown 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 msgr2=0x55c6f3d6a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f70fc02a940 tx=0x7f70fc031810 comp rx=0 tx=0).stop 2024-06-06T07:14:12.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 shutdown_connections 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3d7d4f0 0x55c6f3d7f920 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d6ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f70edffb640 0 client.16908 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d6a810 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 --2- 172.21.15.73:0/2563267063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d6a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 >> 172.21.15.73:0/2563267063 conn(0x55c6f3d5e680 msgr2=0x55c6f3d643b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 shutdown_connections 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 1 -- 172.21.15.73:0/2563267063 wait complete. 2024-06-06T07:14:12.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.446+0000 7f710f615740 20 client.16908 trim_cache size 0 max 16384 2024-06-06T07:14:12.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237776 1.73 MB/sec execute 559 sec latency 2651.701 ms 2024-06-06T07:14:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 Processor -- start 2024-06-06T07:14:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 -- start start 2024-06-06T07:14:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d69210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d851f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71081aef00 con 0x55c6f3d62a90 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b0b60 con 0x55c6f3d6d350 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f710f615740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71081af5d0 con 0x55c6f3d62170 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43200/0 (socket says 172.21.15.73:43200) 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.461+0000 7f7105d9b640 1 -- 172.21.15.73:0/1547893936 learned_addr learned my addr 172.21.15.73:0/1547893936 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 -- 172.21.15.73:0/1547893936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 msgr2=0x55c6f3d69210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d69210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 -- 172.21.15.73:0/1547893936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 msgr2=0x55c6f3d851f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d851f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 -- 172.21.15.73:0/1547893936 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6f3d628c0 con 0x55c6f3d62a90 2024-06-06T07:14:12.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f7105d9b640 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70fc045b10 tx=0x7f70fc031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3302363b6064d752 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f70effff640 1 -- 172.21.15.73:0/1547893936 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc058020 con 0x55c6f3d62a90 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f70effff640 1 -- 172.21.15.73:0/1547893936 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70fc045b90 con 0x55c6f3d62a90 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.462+0000 7f70effff640 1 -- 172.21.15.73:0/1547893936 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc035050 con 0x55c6f3d62a90 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- 172.21.15.73:0/1547893936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 msgr2=0x55c6f3d64b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f70fc045b10 tx=0x7f70fc031400 comp rx=0 tx=0).stop 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- 172.21.15.73:0/1547893936 shutdown_connections 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d851f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d64b30 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- 172.21.15.73:0/1547893936 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d69210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- 172.21.15.73:0/1547893936 >> 172.21.15.73:0/1547893936 conn(0x55c6f3d5e680 msgr2=0x55c6f3d6c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- 172.21.15.73:0/1547893936 shutdown_connections 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- 172.21.15.73:0/1547893936 wait complete. 2024-06-06T07:14:12.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- start start 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 Processor -- start 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d674a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b0430 con 0x55c6f3d62a90 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f71081afd00 con 0x55c6f3d6d350 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f71081acf00 con 0x55c6f3d62170 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710f615740 10 client.0 _ll_register_callbacks cb 0x7f70fc0314a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f7105d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43210/0 (socket says 172.21.15.73:43210) 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f7105d9b640 1 -- 172.21.15.73:0/4130383294 learned_addr learned my addr 172.21.15.73:0/4130383294 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f7106d9d640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d674a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.463+0000 7f710659c640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 -- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 msgr2=0x55c6f3d759d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d759d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 -- 172.21.15.73:0/4130383294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 msgr2=0x55c6f3d674a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d674a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 -- 172.21.15.73:0/4130383294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6f3d67c80 con 0x55c6f3d62a90 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f7105d9b640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f70fc031ac0 tx=0x7f70fc031f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a801b69d70fd989 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70fc061060 con 0x55c6f3d62a90 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 9642+0+0 (secure 0 0 0) 0x7f70fc045480 con 0x55c6f3d62a90 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70fc045b90 con 0x55c6f3d62a90 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f70fc058070 con 0x55c6f3d62a90 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 10 client.16914 fetch_fsmap learned FSMap version 48 2024-06-06T07:14:12.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55c6f3de80c0 con 0x55c6f3d62a90 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70fc045b90 con 0x55c6f3d62a90 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7f70fc0048c0 con 0x55c6f3d62a90 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 10 client.16914 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 20 client.16914 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3de8820 0x55c6f3deac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 4 client.16914 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.464+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c6f3ccdf70 con 0x55c6f3de8820 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.465+0000 7f710659c640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3de8820 0x55c6f3deac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.465+0000 7f710659c640 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3de8820 0x55c6f3deac40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e113f544466557c server_cookie=297ca5db46e7b028 in_seq=0 out_seq=0 2024-06-06T07:14:12.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.465+0000 7f70edffb640 10 client.16914 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f70edffb640 1 -- 172.21.15.73:0/4130383294 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c6f3ccdf70 con 0x55c6f3de8820 2024-06-06T07:14:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f70edffb640 10 client.16914 handle_command_reply: tid=0 2024-06-06T07:14:12.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 client.16914 shutdown 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 msgr2=0x55c6f3d66f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f70fc031ac0 tx=0x7f70fc031f40 comp rx=0 tx=0).stop 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 shutdown_connections 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c6f3de8820 0x55c6f3deac40 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c6f3d6d350 0x55c6f3d674a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c6f3d62a90 0x55c6f3d66f60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f70edffb640 0 client.16914 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 --2- 172.21.15.73:0/4130383294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c6f3d62170 0x55c6f3d759d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 >> 172.21.15.73:0/4130383294 conn(0x55c6f3d5e680 msgr2=0x55c6f3d6a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 shutdown_connections 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 1 -- 172.21.15.73:0/4130383294 wait complete. 2024-06-06T07:14:12.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710f615740 20 client.16914 trim_cache size 0 max 16384 2024-06-06T07:14:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:14:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:14:12.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 msgr2=0x7f70f00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 0x7f70f00a5260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7100007310 tx=0x7f71000072a0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 msgr2=0x7f7108198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f70f800c920 tx=0x7f70f800cde0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 shutdown_connections 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7108106890 0x7f710819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f70f00a2da0 0x7f70f00a5260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7108101290 0x7f7108198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 --2- 172.21.15.73:0/3403944284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71081008d0 0x7f7108198030 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 >> 172.21.15.73:0/3403944284 conn(0x7f7108077a20 msgr2=0x7f71081033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:12.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 shutdown_connections 2024-06-06T07:14:12.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:12.492+0000 7f710c82a640 1 -- 172.21.15.73:0/3403944284 wait complete. 2024-06-06T07:14:13.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237831 1.73 MB/sec execute 560 sec latency 3358.180 ms 2024-06-06T07:14:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:13 smithi170 ceph-mon[36108]: pgmap v404: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:13 smithi170 ceph-mon[36108]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T07:14:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:13 smithi073 ceph-mon[32303]: pgmap v404: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:13 smithi073 ceph-mon[32303]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T07:14:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:13 smithi136 ceph-mon[35901]: pgmap v404: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 801 KiB/s wr, 111 op/s 2024-06-06T07:14:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:13 smithi136 ceph-mon[35901]: osdmap e118: 12 total, 12 up, 12 in 2024-06-06T07:14:14.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 237831 1.73 MB/sec execute 561 sec latency 1114.114 ms 2024-06-06T07:14:15.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 238013 1.73 MB/sec execute 562 sec latency 1397.713 ms 2024-06-06T07:14:15.826 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:15 smithi073 ceph-mon[32303]: pgmap v406: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 764 KiB/s wr, 95 op/s 2024-06-06T07:14:15.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:15 smithi170 ceph-mon[36108]: pgmap v406: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 764 KiB/s wr, 95 op/s 2024-06-06T07:14:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:15 smithi136 ceph-mon[35901]: pgmap v406: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 609 KiB/s rd, 764 KiB/s wr, 95 op/s 2024-06-06T07:14:16.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 238284 1.72 MB/sec execute 563 sec latency 17.581 ms 2024-06-06T07:14:17.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 238758 1.72 MB/sec execute 564 sec latency 14.947 ms 2024-06-06T07:14:17.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:17 smithi170 ceph-mon[36108]: pgmap v407: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 756 KiB/s wr, 94 op/s 2024-06-06T07:14:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:17 smithi073 ceph-mon[32303]: pgmap v407: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 756 KiB/s wr, 94 op/s 2024-06-06T07:14:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:17 smithi136 ceph-mon[35901]: pgmap v407: 129 pgs: 129 active+clean; 135 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 603 KiB/s rd, 756 KiB/s wr, 94 op/s 2024-06-06T07:14:18.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239255 1.73 MB/sec execute 565 sec latency 23.483 ms 2024-06-06T07:14:19.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239388 1.72 MB/sec execute 566 sec latency 726.614 ms 2024-06-06T07:14:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:19 smithi170 ceph-mon[36108]: pgmap v408: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:19 smithi073 ceph-mon[32303]: pgmap v408: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:20.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:19 smithi136 ceph-mon[35901]: pgmap v408: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:20.460 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239388 1.72 MB/sec execute 567 sec latency 1726.701 ms 2024-06-06T07:14:21.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239388 1.72 MB/sec execute 568 sec latency 2726.742 ms 2024-06-06T07:14:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:21 smithi170 ceph-mon[36108]: pgmap v409: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:21 smithi073 ceph-mon[32303]: pgmap v409: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:22.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:21 smithi136 ceph-mon[35901]: pgmap v409: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:22.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239388 1.71 MB/sec execute 569 sec latency 3726.801 ms 2024-06-06T07:14:23.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 239557 1.71 MB/sec execute 570 sec latency 4426.415 ms 2024-06-06T07:14:23.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:23 smithi170 ceph-mon[36108]: pgmap v410: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:24.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:23 smithi073 ceph-mon[32303]: pgmap v410: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:24.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:23 smithi136 ceph-mon[35901]: pgmap v410: 129 pgs: 129 active+clean; 139 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 582 KiB/s wr, 76 op/s 2024-06-06T07:14:24.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 240107 1.71 MB/sec execute 571 sec latency 20.057 ms 2024-06-06T07:14:25.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:24 smithi073 ceph-mon[32303]: pgmap v411: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 574 KiB/s wr, 93 op/s 2024-06-06T07:14:25.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:24 smithi136 ceph-mon[35901]: pgmap v411: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 574 KiB/s wr, 93 op/s 2024-06-06T07:14:25.096 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:24 smithi170 ceph-mon[36108]: pgmap v411: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 474 KiB/s rd, 574 KiB/s wr, 93 op/s 2024-06-06T07:14:25.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 240718 1.72 MB/sec execute 572 sec latency 11.232 ms 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 -- 172.21.15.73:0/598702878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc1068d0 msgr2=0x7fbccc106cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/598702878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc1068d0 0x7fbccc106cb0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc009930 tx=0x7fbcbc02a420 comp rx=0 tx=0).stop 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 -- 172.21.15.73:0/598702878 shutdown_connections 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/598702878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc101280 0x7fbccc10d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/598702878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1008c0 0x7fbccc100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/598702878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc1068d0 0x7fbccc106cb0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 -- 172.21.15.73:0/598702878 >> 172.21.15.73:0/598702878 conn(0x7fbccc076c20 msgr2=0x7fbccc077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 -- 172.21.15.73:0/598702878 shutdown_connections 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.587+0000 7fbcd12e1640 1 -- 172.21.15.73:0/598702878 wait complete. 2024-06-06T07:14:25.588 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 Processor -- start 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 -- start start 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 0x7fbccc198f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1068d0 0x7fbccc1a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc110b90 con 0x7fbccc101280 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc110a10 con 0x7fbccc1068d0 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcd12e1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc110d10 con 0x7fbccc1008c0 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34772/0 (socket says 172.21.15.73:34772) 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 0x7fbccc198f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbcca7fc640 1 -- 172.21.15.73:0/2592902861 learned_addr learned my addr 172.21.15.73:0/2592902861 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.588+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1068d0 0x7fbccc1a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 -- 172.21.15.73:0/2592902861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1068d0 msgr2=0x7fbccc1a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1068d0 0x7fbccc1a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 -- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 msgr2=0x7fbccc199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 -- 172.21.15.73:0/2592902861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbccc1a4510 con 0x7fbccc1008c0 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:25.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 0x7fbccc198f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031440 tx=0x7fbcbc004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcb3fff640 1 -- 172.21.15.73:0/2592902861 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc03f070 con 0x7fbccc1008c0 2024-06-06T07:14:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbccc1a46e0 con 0x7fbccc1008c0 2024-06-06T07:14:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcb3fff640 1 -- 172.21.15.73:0/2592902861 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcbc003ca0 con 0x7fbccc1008c0 2024-06-06T07:14:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcb3fff640 1 -- 172.21.15.73:0/2592902861 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc035070 con 0x7fbccc1008c0 2024-06-06T07:14:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.589+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbccc1a4b50 con 0x7fbccc1008c0 2024-06-06T07:14:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.591+0000 7fbcb3fff640 1 -- 172.21.15.73:0/2592902861 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbcbc03b4f0 con 0x7fbccc1008c0 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 Processor -- start 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 -- start start 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41e9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 0x5568f41f8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568f414d4f0 con 0x5568f41f8760 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5568f40eb870 con 0x5568f41e99b0 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568f41494b0 con 0x5568f41e9090 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcb3fff640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 0x7fbcb40a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 0x5568f41f8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34774/0 (socket says 172.21.15.73:34774) 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcca7fc640 1 -- 172.21.15.73:0/1040121300 learned_addr learned my addr 172.21.15.73:0/1040121300 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 0x7fbcb40a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcb3fff640 1 -- 172.21.15.73:0/2592902861 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fbcbc0efcc0 con 0x7fbccc1008c0 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.592+0000 7fbcca7fc640 1 -- 172.21.15.73:0/1040121300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 msgr2=0x5568f41e9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41e9470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcca7fc640 1 -- 172.21.15.73:0/1040121300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 msgr2=0x5568f41f8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 0x5568f41f8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcca7fc640 1 -- 172.21.15.73:0/1040121300 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f41079b0 con 0x5568f41f8760 2024-06-06T07:14:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbccaffd640 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 0x5568f41f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 0x7fbcb40a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4005fd0 tx=0x7fbcc4009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0010580 tx=0x7fbcc0010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8c7bff900fca34b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcb17fa640 1 -- 172.21.15.73:0/1040121300 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc000e830 con 0x5568f41f8760 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcb17fa640 1 -- 172.21.15.73:0/1040121300 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcc000ee80 con 0x5568f41f8760 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.593+0000 7fbcb17fa640 1 -- 172.21.15.73:0/1040121300 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc0011440 con 0x5568f41f8760 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/1040121300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 msgr2=0x5568f41fab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fbcc0010580 tx=0x7fbcc0010a40 comp rx=0 tx=0).stop 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/1040121300 shutdown_connections 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41f8760 0x5568f41fab50 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41e99b0 0x5568f41f8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/1040121300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41e9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/1040121300 >> 172.21.15.73:0/1040121300 conn(0x5568f41e5680 msgr2=0x5568f41e5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/1040121300 shutdown_connections 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/1040121300 wait complete. 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- start start 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 Processor -- start 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f41f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5568f41fb390 con 0x5568f41e9090 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5568f40d6ae0 con 0x5568f41f8760 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5568f41fb510 con 0x5568f41e99b0 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.594+0000 7fbcd3ff3740 10 client.0 _ll_register_callbacks cb 0x5568f410f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:59840/0 (socket says 172.21.15.73:59840) 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34776/0 (socket says 172.21.15.73:34776) 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcca7fc640 1 -- 172.21.15.73:0/3691909998 learned_addr learned my addr 172.21.15.73:0/3691909998 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f41f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 -- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 msgr2=0x5568f41f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 -- 172.21.15.73:0/3691909998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 msgr2=0x5568f41f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f41f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 -- 172.21.15.73:0/3691909998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f3e61210 con 0x5568f41e9090 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbccaffd640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ed000 tx=0x7fbcbc005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1256d31fbbc27f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc0f1030 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fbcbc0c45b0 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5568f4160f10 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbcbc0bf070 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcd3ff3740 10 client.16926 fetch_fsmap learned FSMap version 48 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568f3f2d620 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.595+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x5568f4160f10 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fbcbc0d1070 con 0x5568f41e9090 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 10 client.16926 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 20 client.16926 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f42088d0 0x5568f420ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 4 client.16926 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568f415ac20 con 0x5568f42088d0 2024-06-06T07:14:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f42088d0 0x5568f420ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcaeff5640 10 client.16926 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.596+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f42088d0 0x5568f420ad90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=466e2b7e3c1b5b43 server_cookie=7112d303516d85be in_seq=0 out_seq=0 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcaeff5640 1 -- 172.21.15.73:0/3691909998 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568f415ac20 con 0x5568f42088d0 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcaeff5640 10 client.16926 handle_command_reply: tid=0 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 client.16926 shutdown 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 msgr2=0x5568f41f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc0ed000 tx=0x7fbcbc005c90 comp rx=0 tx=0).stop 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 shutdown_connections 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f42088d0 0x5568f420ad90 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f41f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f41f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/3691909998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f5740 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 >> 172.21.15.73:0/3691909998 conn(0x5568f41e5680 msgr2=0x5568f41eb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcaeff5640 0 client.16926 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 shutdown_connections 2024-06-06T07:14:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.597+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/3691909998 wait complete. 2024-06-06T07:14:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.598+0000 7fbcd3ff3740 20 client.16926 trim_cache size 0 max 16384 2024-06-06T07:14:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 Processor -- start 2024-06-06T07:14:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 -- start start 2024-06-06T07:14:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f42106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4210be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1acbe0 con 0x5568f41e9090 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1ae840 con 0x5568f41f8760 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1ad2b0 con 0x5568f41e99b0 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34780/0 (socket says 172.21.15.73:34780) 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccaffd640 1 -- 172.21.15.73:0/2142761881 learned_addr learned my addr 172.21.15.73:0/2142761881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f42106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4210be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 -- 172.21.15.73:0/2142761881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 msgr2=0x5568f42106a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f42106a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 -- 172.21.15.73:0/2142761881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 msgr2=0x5568f41f7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.615+0000 7fbccb7fe640 1 -- 172.21.15.73:0/2142761881 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f41ea230 con 0x5568f41f8760 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.616+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4210be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40099d0 tx=0x7fbcc4008000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4ae7a08e2e45611 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.616+0000 7fbcb0ff9640 1 -- 172.21.15.73:0/2142761881 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4011d30 con 0x5568f41f8760 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.616+0000 7fbcb0ff9640 1 -- 172.21.15.73:0/2142761881 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcc4005020 con 0x5568f41f8760 2024-06-06T07:14:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.616+0000 7fbcb0ff9640 1 -- 172.21.15.73:0/2142761881 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4010690 con 0x5568f41f8760 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2142761881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 msgr2=0x5568f4210be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4210be0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbcc40099d0 tx=0x7fbcc4008000 comp rx=0 tx=0).stop 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2142761881 shutdown_connections 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4210be0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e99b0 0x5568f42106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2142761881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e9090 0x5568f41f7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2142761881 >> 172.21.15.73:0/2142761881 conn(0x5568f41e5680 msgr2=0x5568f41f6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2142761881 shutdown_connections 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2142761881 wait complete. 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- start start 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 Processor -- start 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4271660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1ae110 con 0x5568f41e99b0 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1ad9e0 con 0x5568f41f8760 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbccc1aabe0 con 0x5568f41e9090 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcd3ff3740 10 client.0 _ll_register_callbacks cb 0x7fbcbc009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34790/0 (socket says 172.21.15.73:34790) 2024-06-06T07:14:25.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbcca7fc640 1 -- 172.21.15.73:0/2080975668 learned_addr learned my addr 172.21.15.73:0/2080975668 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.617+0000 7fbccb7fe640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4271660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 -- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 msgr2=0x5568f41fbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41fbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 -- 172.21.15.73:0/2080975668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 msgr2=0x5568f4271660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4271660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 -- 172.21.15.73:0/2080975668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f4271c50 con 0x5568f41e99b0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcca7fc640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbcc001cc30 tx=0x7fbcc001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13406f8c89b57983 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcaeff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcc001c6c0 con 0x5568f41e99b0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcc4005020 con 0x5568f41e99b0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 9554+0+0 (secure 0 0 0) 0x7fbcc001c860 con 0x5568f41e99b0 2024-06-06T07:14:25.624 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fbcc003a050 con 0x5568f41e99b0 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcd3ff3740 10 client.16932 fetch_fsmap learned FSMap version 48 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.618+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5568f4272f40 con 0x5568f41e99b0 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcc4005020 con 0x5568f41e99b0 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 48) ==== 8386+0+0 (secure 0 0 0) 0x7fbcc00035f0 con 0x5568f41e99b0 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 10 client.16932 fetch_fsmap finished waiting for FSMap version 48 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 10 client.16932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 20 client.16932 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f4273440 0x5568f4275860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 4 client.16932 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5568f4154f70 con 0x5568f4273440 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f4273440 0x5568f4275860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:25.625 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbcaeff5640 10 client.16932 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:25.626 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.619+0000 7fbccaffd640 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f4273440 0x5568f4275860 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20cec8edcc98421a server_cookie=2d7747f4ae4d9c1c in_seq=0 out_seq=0 2024-06-06T07:14:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcaeff5640 1 -- 172.21.15.73:0/2080975668 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5568f4154f70 con 0x5568f4273440 2024-06-06T07:14:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcaeff5640 10 client.16932 handle_command_reply: tid=0 2024-06-06T07:14:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 client.16932 shutdown 2024-06-06T07:14:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 msgr2=0x5568f41fc410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fbcc001cc30 tx=0x7fbcc001c480 comp rx=0 tx=0).stop 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 shutdown_connections 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5568f4273440 0x5568f4275860 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5568f41f8760 0x5568f4271660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5568f41e99b0 0x5568f41fc410 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 --2- 172.21.15.73:0/2080975668 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5568f41e9090 0x5568f41fbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 >> 172.21.15.73:0/2080975668 conn(0x5568f41e5680 msgr2=0x5568f41e47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcaeff5640 0 client.16932 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 shutdown_connections 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.646+0000 7fbcd3ff3740 1 -- 172.21.15.73:0/2080975668 wait complete. 2024-06-06T07:14:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd3ff3740 20 client.16932 trim_cache size 0 max 16384 2024-06-06T07:14:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:14:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:14:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:14:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 msgr2=0x7fbcb40a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 0x7fbcb40a5260 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbcc4005fd0 tx=0x7fbcc4009500 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 msgr2=0x7fbccc198f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 0x7fbccc198f50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc031440 tx=0x7fbcbc004270 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 shutdown_connections 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbccc1068d0 0x7fbccc1a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbccc101280 0x7fbccc199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbcb40a2da0 0x7fbcb40a5260 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 --2- 172.21.15.73:0/2592902861 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbccc1008c0 0x7fbccc198f50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 >> 172.21.15.73:0/2592902861 conn(0x7fbccc076c20 msgr2=0x7fbccc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.647+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 shutdown_connections 2024-06-06T07:14:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:25.648+0000 7fbcd12e1640 1 -- 172.21.15.73:0/2592902861 wait complete. 2024-06-06T07:14:26.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241212 1.72 MB/sec execute 573 sec latency 14.079 ms 2024-06-06T07:14:26.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setuser ceph since I am not root 2024-06-06T07:14:26.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ignoring --setgroup ceph since I am not root 2024-06-06T07:14:26.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-mds[69750]: -- 172.21.15.136:0/1192820136 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55909c2f5a40 con 0x55909c2c9000 2024-06-06T07:14:26.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: starting mds.e at 2024-06-06T07:14:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:26 smithi073 ceph-mon[32303]: pgmap v412: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 497 KiB/s wr, 72 op/s 2024-06-06T07:14:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:26 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:14:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-mon[35901]: pgmap v412: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 497 KiB/s wr, 72 op/s 2024-06-06T07:14:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:26 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:14:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:26 smithi170 ceph-mon[36108]: pgmap v412: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 497 KiB/s wr, 72 op/s 2024-06-06T07:14:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:26 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:14:27.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241310 1.72 MB/sec execute 574 sec latency 805.975 ms 2024-06-06T07:14:28.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:27 smithi073 ceph-mon[32303]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T07:14:28.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:27 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/1442663864,v1:172.21.15.136:6835/1442663864] up:boot 2024-06-06T07:14:28.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:27 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:14:28.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:27 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:14:28.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:27 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:14:28.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:27 smithi136 ceph-mon[35901]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T07:14:28.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:27 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/1442663864,v1:172.21.15.136:6835/1442663864] up:boot 2024-06-06T07:14:28.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:27 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:14:28.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:27 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:14:28.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:27 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:14:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:27 smithi170 ceph-mon[36108]: osdmap e119: 12 total, 12 up, 12 in 2024-06-06T07:14:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:27 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/1442663864,v1:172.21.15.136:6835/1442663864] up:boot 2024-06-06T07:14:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:27 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:14:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:27 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:14:28.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:27 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:14:28.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241310 1.72 MB/sec execute 575 sec latency 1806.081 ms 2024-06-06T07:14:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:28 smithi073 ceph-mon[32303]: pgmap v414: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:28 smithi136 ceph-mon[35901]: pgmap v414: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:28 smithi170 ceph-mon[36108]: pgmap v414: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:29.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241310 1.72 MB/sec execute 576 sec latency 2806.189 ms 2024-06-06T07:14:30.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241377 1.71 MB/sec execute 577 sec latency 3089.945 ms 2024-06-06T07:14:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:30 smithi073 ceph-mon[32303]: pgmap v415: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:30 smithi136 ceph-mon[35901]: pgmap v415: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:30 smithi170 ceph-mon[36108]: pgmap v415: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:31.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241377 1.71 MB/sec execute 578 sec latency 1455.899 ms 2024-06-06T07:14:32.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241377 1.71 MB/sec execute 579 sec latency 2455.953 ms 2024-06-06T07:14:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:32 smithi073 ceph-mon[32303]: pgmap v416: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:32 smithi136 ceph-mon[35901]: pgmap v416: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:32 smithi170 ceph-mon[36108]: pgmap v416: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 835 KiB/s rd, 731 KiB/s wr, 93 op/s 2024-06-06T07:14:33.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241377 1.70 MB/sec execute 580 sec latency 3456.036 ms 2024-06-06T07:14:34.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241377 1.70 MB/sec execute 581 sec latency 4456.127 ms 2024-06-06T07:14:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:34 smithi073 ceph-mon[32303]: pgmap v417: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:34 smithi136 ceph-mon[35901]: pgmap v417: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:34 smithi170 ceph-mon[36108]: pgmap v417: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:35.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241557 1.70 MB/sec execute 582 sec latency 4740.000 ms 2024-06-06T07:14:36.461 INFO:tasks.workunit.client.0.smithi073.stdout: 1 241822 1.70 MB/sec execute 583 sec latency 14.363 ms 2024-06-06T07:14:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:36 smithi073 ceph-mon[32303]: pgmap v418: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:36 smithi136 ceph-mon[35901]: pgmap v418: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:36 smithi170 ceph-mon[36108]: pgmap v418: 129 pgs: 129 active+clean; 136 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 583 KiB/s rd, 695 KiB/s wr, 78 op/s 2024-06-06T07:14:37.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242288 1.70 MB/sec execute 584 sec latency 15.257 ms 2024-06-06T07:14:38.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242769 1.70 MB/sec execute 585 sec latency 20.194 ms 2024-06-06T07:14:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- 172.21.15.73:0/4272933669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c106890 msgr2=0x7fe53c106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 --2- 172.21.15.73:0/4272933669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c106890 0x7fe53c106c70 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe530009930 tx=0x7fe53002a420 comp rx=0 tx=0).stop 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- 172.21.15.73:0/4272933669 shutdown_connections 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 --2- 172.21.15.73:0/4272933669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c101290 0x7fe53c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 --2- 172.21.15.73:0/4272933669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c1008d0 0x7fe53c100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 --2- 172.21.15.73:0/4272933669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c106890 0x7fe53c106c70 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- 172.21.15.73:0/4272933669 >> 172.21.15.73:0/4272933669 conn(0x7fe53c077a20 msgr2=0x7fe53c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- 172.21.15.73:0/4272933669 shutdown_connections 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- 172.21.15.73:0/4272933669 wait complete. 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 Processor -- start 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.745+0000 7fe543129640 1 -- start start 2024-06-06T07:14:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 0x7fe53c198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c106890 0x7fe53c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c110b50 con 0x7fe53c1008d0 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1109d0 con 0x7fe53c101290 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe543129640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c110cd0 con 0x7fe53c106890 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55372/0 (socket says 172.21.15.73:55372) 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 -- 172.21.15.73:0/748764478 learned_addr learned my addr 172.21.15.73:0/748764478 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe5416c6640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c106890 0x7fe53c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe53bfff640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 0x7fe53c198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 -- 172.21.15.73:0/748764478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c106890 msgr2=0x7fe53c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c106890 0x7fe53c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 -- 172.21.15.73:0/748764478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 msgr2=0x7fe53c198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 0x7fe53c198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 -- 172.21.15.73:0/748764478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe53c19d0c0 con 0x7fe53c1008d0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe53bfff640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 0x7fe53c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.746+0000 7fe540ec5640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe530005940 tx=0x7fe5300040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.747+0000 7fe539ffb640 1 -- 172.21.15.73:0/748764478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe53003f030 con 0x7fe53c1008d0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.747+0000 7fe539ffb640 1 -- 172.21.15.73:0/748764478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5300044c0 con 0x7fe53c1008d0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.747+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe53c19d350 con 0x7fe53c1008d0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.747+0000 7fe539ffb640 1 -- 172.21.15.73:0/748764478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530035070 con 0x7fe53c1008d0 2024-06-06T07:14:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.747+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe53c1a4c40 con 0x7fe53c1008d0 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.749+0000 7fe539ffb640 1 -- 172.21.15.73:0/748764478 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe530042030 con 0x7fe53c1008d0 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 Processor -- start 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 -- start start 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab8a8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe539ffb640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 0x7fe52c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645ab7ff4f0 con 0x5645ab89b170 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645ab79d870 con 0x5645ab89ba90 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe545e39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5645ab7fb4b0 con 0x5645ab8a6350 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe5416c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.757 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab8a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe5416c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55384/0 (socket says 172.21.15.73:55384) 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe5416c6640 1 -- 172.21.15.73:0/1797029981 learned_addr learned my addr 172.21.15.73:0/1797029981 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe5416c6640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 0x7fe52c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe540ec5640 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe539ffb640 1 -- 172.21.15.73:0/748764478 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7fe5300efa90 con 0x7fe53c1008d0 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 -- 172.21.15.73:0/1797029981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 msgr2=0x5645ab8a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 -- 172.21.15.73:0/1797029981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 msgr2=0x5645ab89b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe53bfff640 1 -- 172.21.15.73:0/1797029981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ab7b99b0 con 0x5645ab8a6350 2024-06-06T07:14:38.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.750+0000 7fe5416c6640 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.751+0000 7fe53bfff640 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab8a8740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe528009fa0 tx=0x7fe5280118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abeb7448ae9373fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.751+0000 7fe5416c6640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 0x7fe52c0a5260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe5340099e0 tx=0x7fe534009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.751+0000 7fe526ffd640 1 -- 172.21.15.73:0/1797029981 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528004030 con 0x5645ab8a6350 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.751+0000 7fe526ffd640 1 -- 172.21.15.73:0/1797029981 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5280035f0 con 0x5645ab8a6350 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.751+0000 7fe526ffd640 1 -- 172.21.15.73:0/1797029981 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52800f7a0 con 0x5645ab8a6350 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- 172.21.15.73:0/1797029981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 msgr2=0x5645ab8a8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab8a8740 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fe528009fa0 tx=0x7fe5280118d0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- 172.21.15.73:0/1797029981 shutdown_connections 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab8a8740 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- 172.21.15.73:0/1797029981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- 172.21.15.73:0/1797029981 >> 172.21.15.73:0/1797029981 conn(0x5645ab897680 msgr2=0x5645ab897a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- 172.21.15.73:0/1797029981 shutdown_connections 2024-06-06T07:14:38.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- 172.21.15.73:0/1797029981 wait complete. 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- start start 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 Processor -- start 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89b170 0x5645ab8a3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5645ab8a8f80 con 0x5645ab8a6350 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5645ab788ae0 con 0x5645ab89b170 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.752+0000 7fe545e39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5645ab8a9100 con 0x5645ab89ba90 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe545e39740 10 client.0 _ll_register_callbacks cb 0x5645ab7c19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe5416c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe5416c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55400/0 (socket says 172.21.15.73:55400) 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe5416c6640 1 -- 172.21.15.73:0/2569551313 learned_addr learned my addr 172.21.15.73:0/2569551313 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe540ec5640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89b170 0x5645ab8a3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 -- 172.21.15.73:0/2569551313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89b170 msgr2=0x5645ab8a3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89b170 0x5645ab8a3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 -- 172.21.15.73:0/2569551313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 msgr2=0x5645ab8a3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 -- 172.21.15.73:0/2569551313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ab513210 con 0x5645ab89ba90 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe5416c6640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe53bfff640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe52801b040 tx=0x7fe528003ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e6bb71549d443c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:14:38.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe528028030 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7fe52801f040 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.753+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5280035f0 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe528032050 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 10 client.26374 fetch_fsmap learned FSMap version 50 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5645ab5df620 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5280035f0 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fe528004850 con 0x5645ab89ba90 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 10 client.26374 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 10 client.26374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 20 client.26374 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab8b6360 0x5645ab8b8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 4 client.26374 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645ab812f10 con 0x5645ab8b6360 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe540ec5640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab8b6360 0x5645ab8b8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe540ec5640 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab8b6360 0x5645ab8b8750 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93a790bf6d8438db server_cookie=514e24fd7d85c3c6 in_seq=0 out_seq=0 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.754+0000 7fe51bfff640 10 client.26374 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe51bfff640 1 -- 172.21.15.73:0/2569551313 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5645ab812f10 con 0x5645ab8b6360 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe51bfff640 10 client.26374 handle_command_reply: tid=0 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 client.26374 shutdown 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 msgr2=0x5645ab8a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a37b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe52801b040 tx=0x7fe528003ef0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 shutdown_connections 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab8b6360 0x5645ab8b8750 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab8a6350 0x5645ab8a3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe51bfff640 0 client.26374 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a37b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 --2- 172.21.15.73:0/2569551313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89b170 0x5645ab8a3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.755+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 >> 172.21.15.73:0/2569551313 conn(0x5645ab897680 msgr2=0x5645ab89d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.756+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 shutdown_connections 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.756+0000 7fe545e39740 1 -- 172.21.15.73:0/2569551313 wait complete. 2024-06-06T07:14:38.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.756+0000 7fe545e39740 20 client.26374 trim_cache size 0 max 16384 2024-06-06T07:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 Processor -- start 2024-06-06T07:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 -- start start 2024-06-06T07:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab8a6350 0x5645ab8bdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1aeda0 con 0x5645ab89b170 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1b09a0 con 0x5645ab8a6350 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe545e39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1af470 con 0x5645ab89ba90 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55416/0 (socket says 172.21.15.73:55416) 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe53bfff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 -- 172.21.15.73:0/1002441978 learned_addr learned my addr 172.21.15.73:0/1002441978 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe5416c6640 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab8a6350 0x5645ab8bdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 -- 172.21.15.73:0/1002441978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 msgr2=0x5645ab8a3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 -- 172.21.15.73:0/1002441978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab8a6350 msgr2=0x5645ab8bdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab8a6350 0x5645ab8bdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 -- 172.21.15.73:0/1002441978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645ab89ee40 con 0x5645ab89b170 2024-06-06T07:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.773+0000 7fe540ec5640 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe5300bbc40 tx=0x7fe530049bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb67b4b37da161d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe5267fc640 1 -- 172.21.15.73:0/1002441978 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530049040 con 0x5645ab89b170 2024-06-06T07:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe5267fc640 1 -- 172.21.15.73:0/1002441978 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5300c4870 con 0x5645ab89b170 2024-06-06T07:14:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe5267fc640 1 -- 172.21.15.73:0/1002441978 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530041d80 con 0x5645ab89b170 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 -- 172.21.15.73:0/1002441978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 msgr2=0x5645ab89db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe5300bbc40 tx=0x7fe530049bd0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 -- 172.21.15.73:0/1002441978 shutdown_connections 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab8a6350 0x5645ab8bdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab89ba90 0x5645ab8a3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 --2- 172.21.15.73:0/1002441978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab89db80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 -- 172.21.15.73:0/1002441978 >> 172.21.15.73:0/1002441978 conn(0x5645ab897680 msgr2=0x5645ab8a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 -- 172.21.15.73:0/1002441978 shutdown_connections 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.774+0000 7fe545e39740 1 -- 172.21.15.73:0/1002441978 wait complete. 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 -- start start 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 Processor -- start 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab91ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1b02d0 con 0x5645ab89b170 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1afba0 con 0x5645ab89ba90 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe53c1acd80 con 0x5645ab8a6350 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe545e39740 10 client.0 _ll_register_callbacks cb 0x7fe534009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe540ec5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55428/0 (socket says 172.21.15.73:55428) 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe540ec5640 1 -- 172.21.15.73:0/408111876 learned_addr learned my addr 172.21.15.73:0/408111876 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.775+0000 7fe5416c6640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab91ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe53bfff640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 -- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 msgr2=0x5645ab91ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab91ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 -- 172.21.15.73:0/408111876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 msgr2=0x5645ab8a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a9c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 -- 172.21.15.73:0/408111876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645ab91f4d0 con 0x5645ab89b170 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe5416c6640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab91ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe540ec5640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ed430 tx=0x7fe530049c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af91ce1c5cc69d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe530049050 con 0x5645ab89b170 2024-06-06T07:14:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7fe5300ed0b0 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5300c4870 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7fe5300b8320 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 10 client.16956 fetch_fsmap learned FSMap version 50 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x5645ab9206f0 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5300c4870 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7fe5300bb100 con 0x5645ab89b170 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 10 client.16956 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.776+0000 7fe545e39740 20 client.16956 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab920e10 0x5645ab923230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe545e39740 4 client.16956 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:14:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645ab80cc20 con 0x5645ab920e10 2024-06-06T07:14:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe53bfff640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab920e10 0x5645ab923230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe53bfff640 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab920e10 0x5645ab923230 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bc9d07e6e07b048 server_cookie=ec2ccdd03248b284 in_seq=0 out_seq=0 2024-06-06T07:14:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.777+0000 7fe51bfff640 10 client.16956 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe51bfff640 1 -- 172.21.15.73:0/408111876 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5645ab80cc20 con 0x5645ab920e10 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe51bfff640 10 client.16956 handle_command_reply: tid=0 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 client.16956 shutdown 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 msgr2=0x5645ab8a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe5300ed430 tx=0x7fe530049c40 comp rx=0 tx=0).stop 2024-06-06T07:14:38.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 shutdown_connections 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5645ab920e10 0x5645ab923230 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5645ab8a6350 0x5645ab91ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe51bfff640 0 client.16956 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5645ab89ba90 0x5645ab8a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 --2- 172.21.15.73:0/408111876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5645ab89b170 0x5645ab8a9720 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 >> 172.21.15.73:0/408111876 conn(0x5645ab897680 msgr2=0x5645ab8b1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 shutdown_connections 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 1 -- 172.21.15.73:0/408111876 wait complete. 2024-06-06T07:14:38.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.808+0000 7fe545e39740 20 client.16956 trim_cache size 0 max 16384 2024-06-06T07:14:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:14:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:14:38.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 msgr2=0x7fe52c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 0x7fe52c0a5260 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fe5340099e0 tx=0x7fe534009500 comp rx=0 tx=0).stop 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 msgr2=0x7fe53c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe530005940 tx=0x7fe5300040c0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 shutdown_connections 2024-06-06T07:14:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe52c0a2da0 0x7fe52c0a5260 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe53c106890 0x7fe53c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe53c101290 0x7fe53c198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 --2- 172.21.15.73:0/748764478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe53c1008d0 0x7fe53c1980d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 >> 172.21.15.73:0/748764478 conn(0x7fe53c077a20 msgr2=0x7fe53c069330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 shutdown_connections 2024-06-06T07:14:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:38.809+0000 7fe543129640 1 -- 172.21.15.73:0/748764478 wait complete. 2024-06-06T07:14:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:38 smithi073 ceph-mon[32303]: pgmap v419: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-06T07:14:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:38 smithi136 ceph-mon[35901]: pgmap v419: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-06T07:14:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:38 smithi170 ceph-mon[36108]: pgmap v419: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 703 KiB/s rd, 1.1 MiB/s wr, 121 op/s 2024-06-06T07:14:39.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242931 1.70 MB/sec execute 586 sec latency 622.878 ms 2024-06-06T07:14:40.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242931 1.69 MB/sec execute 587 sec latency 1622.907 ms 2024-06-06T07:14:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:40 smithi073 ceph-mon[32303]: pgmap v420: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:40 smithi136 ceph-mon[35901]: pgmap v420: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:40 smithi170 ceph-mon[36108]: pgmap v420: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:41.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242931 1.69 MB/sec execute 588 sec latency 2623.028 ms 2024-06-06T07:14:42.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 242931 1.69 MB/sec execute 589 sec latency 3623.123 ms 2024-06-06T07:14:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:42 smithi073 ceph-mon[32303]: pgmap v421: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:42 smithi136 ceph-mon[35901]: pgmap v421: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:43.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:42 smithi170 ceph-mon[36108]: pgmap v421: 129 pgs: 129 active+clean; 139 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 451 KiB/s wr, 57 op/s 2024-06-06T07:14:43.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 243104 1.69 MB/sec execute 590 sec latency 4319.881 ms 2024-06-06T07:14:44.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 243501 1.68 MB/sec execute 591 sec latency 406.804 ms 2024-06-06T07:14:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:44 smithi073 ceph-mon[32303]: pgmap v422: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 529 KiB/s wr, 86 op/s 2024-06-06T07:14:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:44 smithi136 ceph-mon[35901]: pgmap v422: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 529 KiB/s wr, 86 op/s 2024-06-06T07:14:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:44 smithi170 ceph-mon[36108]: pgmap v422: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 529 KiB/s wr, 86 op/s 2024-06-06T07:14:45.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 243798 1.68 MB/sec execute 592 sec latency 686.046 ms 2024-06-06T07:14:46.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244517 1.69 MB/sec execute 593 sec latency 15.719 ms 2024-06-06T07:14:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:46 smithi073 ceph-mon[32303]: pgmap v423: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 503 KiB/s wr, 74 op/s 2024-06-06T07:14:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:46 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:14:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:46 smithi136 ceph-mon[35901]: pgmap v423: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 503 KiB/s wr, 74 op/s 2024-06-06T07:14:47.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:46 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:14:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:46 smithi170 ceph-mon[36108]: pgmap v423: 129 pgs: 129 active+clean; 137 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 391 KiB/s rd, 503 KiB/s wr, 74 op/s 2024-06-06T07:14:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:46 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:14:47.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244855 1.70 MB/sec execute 594 sec latency 57.188 ms 2024-06-06T07:14:48.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:47 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:14:48.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:47 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:14:48.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:47 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:14:48.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:47 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:14:48.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:47 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:14:48.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:47 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:14:48.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:47 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:14:48.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:47 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:14:48.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:47 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:14:48.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244855 1.69 MB/sec execute 595 sec latency 1057.231 ms 2024-06-06T07:14:49.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:49 smithi073 ceph-mon[32303]: pgmap v424: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-06T07:14:49.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:49 smithi136 ceph-mon[35901]: pgmap v424: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-06T07:14:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:49 smithi170 ceph-mon[36108]: pgmap v424: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 830 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-06T07:14:49.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244855 1.69 MB/sec execute 596 sec latency 2057.315 ms 2024-06-06T07:14:50.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244867 1.69 MB/sec execute 597 sec latency 2338.101 ms 2024-06-06T07:14:51.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:51 smithi073 ceph-mon[32303]: pgmap v425: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:51.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:51 smithi136 ceph-mon[35901]: pgmap v425: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:51 smithi170 ceph-mon[36108]: pgmap v425: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:51.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244867 1.68 MB/sec execute 598 sec latency 1653.012 ms 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 -- 172.21.15.73:0/2464530852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 msgr2=0x7f37e4105c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2464530852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e4105c20 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f37d4009900 tx=0x7f37d402a410 comp rx=0 tx=0).stop 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 -- 172.21.15.73:0/2464530852 shutdown_connections 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2464530852 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e410d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2464530852 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 0x7f37e4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2464530852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e4105c20 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 -- 172.21.15.73:0/2464530852 >> 172.21.15.73:0/2464530852 conn(0x7f37e40766a0 msgr2=0x7f37e4076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 -- 172.21.15.73:0/2464530852 shutdown_connections 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 -- 172.21.15.73:0/2464530852 wait complete. 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.905+0000 7f37e93ce640 1 Processor -- start 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 -- start start 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 0x7f37e4198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e4198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4110b20 con 0x7f37e4105840 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41109a0 con 0x7f37e41015b0 2024-06-06T07:14:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e93ce640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4110ca0 con 0x7f37e4100bf0 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53326/0 (socket says 172.21.15.73:53326) 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 -- 172.21.15.73:0/2471710564 learned_addr learned my addr 172.21.15.73:0/2471710564 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 0x7f37e4198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37dbfff640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e4198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 -- 172.21.15.73:0/2471710564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 msgr2=0x7f37e4198030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 0x7f37e4198030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 -- 172.21.15.73:0/2471710564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 msgr2=0x7f37e4198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e4198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 -- 172.21.15.73:0/2471710564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e419cfd0 con 0x7f37e4105840 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37dbfff640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:14:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.906+0000 7f37e37fe640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00cce0 tx=0x7f37dc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.907+0000 7f37e0ff9640 1 -- 172.21.15.73:0/2471710564 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc00f450 con 0x7f37e4105840 2024-06-06T07:14:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.907+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e419d2c0 con 0x7f37e4105840 2024-06-06T07:14:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.907+0000 7f37e0ff9640 1 -- 172.21.15.73:0/2471710564 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37dc00faa0 con 0x7f37e4105840 2024-06-06T07:14:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.907+0000 7f37e0ff9640 1 -- 172.21.15.73:0/2471710564 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37dc018b70 con 0x7f37e4105840 2024-06-06T07:14:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.907+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e41a4b50 con 0x7f37e4105840 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.909+0000 7f37e0ff9640 1 -- 172.21.15.73:0/2471710564 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f37dc018d10 con 0x7f37e4105840 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.909+0000 7f37ec1b8740 1 Processor -- start 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.909+0000 7f37ec1b8740 1 -- start start 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f30fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f312740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e22f2694f0 con 0x55e22f305170 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e22f207870 con 0x55e22f310350 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e22f2654b0 con 0x55e22f305a90 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37e0ff9640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 0x7f37d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53328/0 (socket says 172.21.15.73:53328) 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 -- 172.21.15.73:0/3433927313 learned_addr learned my addr 172.21.15.73:0/3433927313 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f30fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37e37fe640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 0x7f37d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f312740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37e0ff9640 1 -- 172.21.15.73:0/2471710564 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7f37dc0cfc50 con 0x7f37e4105840 2024-06-06T07:14:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 -- 172.21.15.73:0/3433927313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 msgr2=0x55e22f30fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f30fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 -- 172.21.15.73:0/3433927313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 msgr2=0x55e22f312740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f312740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 -- 172.21.15.73:0/3433927313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e22f2239b0 con 0x55e22f305170 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005fd0 tx=0x7f37cc009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1db44b7dd90f6e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.910+0000 7f37e37fe640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 0x7f37d00a5260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cd0a0 tx=0x7f37dc091250 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.911+0000 7f37d97fa640 1 -- 172.21.15.73:0/3433927313 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0104d0 con 0x55e22f305170 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.911+0000 7f37d97fa640 1 -- 172.21.15.73:0/3433927313 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37cc01b070 con 0x55e22f305170 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.911+0000 7f37d97fa640 1 -- 172.21.15.73:0/3433927313 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc013780 con 0x55e22f305170 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.911+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3433927313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 msgr2=0x55e22f305550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.911+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f37cc005fd0 tx=0x7f37cc009650 comp rx=0 tx=0).stop 2024-06-06T07:14:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3433927313 shutdown_connections 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f312740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f30fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3433927313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f305550 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3433927313 >> 172.21.15.73:0/3433927313 conn(0x55e22f301680 msgr2=0x55e22f301a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3433927313 shutdown_connections 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3433927313 wait complete. 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- start start 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 Processor -- start 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305170 0x55e22f30d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f30dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e22f312f20 con 0x55e22f305a90 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e22f1f2ae0 con 0x55e22f310350 2024-06-06T07:14:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e22f3130a0 con 0x55e22f305170 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37ec1b8740 10 client.0 _ll_register_callbacks cb 0x55e22f22b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.912+0000 7f37e2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53340/0 (socket says 172.21.15.73:53340) 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 -- 172.21.15.73:0/3150087383 learned_addr learned my addr 172.21.15.73:0/3150087383 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f30dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305170 0x55e22f30d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 -- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305170 msgr2=0x55e22f30d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305170 0x55e22f30d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 -- 172.21.15.73:0/3150087383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 msgr2=0x55e22f30dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f30dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 -- 172.21.15.73:0/3150087383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22ef7d210 con 0x55e22f305a90 2024-06-06T07:14:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f30dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37d4033000 tx=0x7f37d40316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e49f867db9af80 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d4035040 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7f37d4031cf0 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37cc01b070 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f37d404d050 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ec1b8740 10 client.16968 fetch_fsmap learned FSMap version 50 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e22f049620 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.913+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37cc01b070 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f37d4004750 con 0x55e22f305a90 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 10 client.16968 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T07:14:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 10 client.16968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 20 client.16968 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f320520 0x55e22f322960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 4 client.16968 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e22f27cf10 con 0x55e22f320520 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f320520 0x55e22f322960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f320520 0x55e22f322960 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f8dfdcf07ee5791 server_cookie=3258ed548f278ede in_seq=0 out_seq=0 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.914+0000 7f37ca7fc640 10 client.16968 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ca7fc640 1 -- 172.21.15.73:0/3150087383 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e22f27cf10 con 0x55e22f320520 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ca7fc640 10 client.16968 handle_command_reply: tid=0 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ec1b8740 1 client.16968 shutdown 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 msgr2=0x55e22f30d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f37d4033000 tx=0x7f37d40316d0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 shutdown_connections 2024-06-06T07:14:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.915+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f320520 0x55e22f322960 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f30dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305a90 0x55e22f30d7b0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3150087383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305170 0x55e22f30d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 >> 172.21.15.73:0/3150087383 conn(0x55e22f301680 msgr2=0x55e22f3073b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ca7fc640 0 client.16968 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 shutdown_connections 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3150087383 wait complete. 2024-06-06T07:14:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.916+0000 7f37ec1b8740 20 client.16968 trim_cache size 0 max 16384 2024-06-06T07:14:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 Processor -- start 2024-06-06T07:14:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 -- start start 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f328270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f3287b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41aeb10 con 0x55e22f305170 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41b0710 con 0x55e22f310350 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41af1e0 con 0x55e22f305a90 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53342/0 (socket says 172.21.15.73:53342) 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37dbfff640 1 -- 172.21.15.73:0/3938906176 learned_addr learned my addr 172.21.15.73:0/3938906176 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.931+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f328270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 -- 172.21.15.73:0/3938906176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 msgr2=0x55e22f328270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f328270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 -- 172.21.15.73:0/3938906176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 msgr2=0x55e22f3287b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f3287b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 -- 172.21.15.73:0/3938906176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e22f3058c0 con 0x55e22f305170 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37dbfff640 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00a010 tx=0x7f37cc015420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ed043c33b19cf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37d8ff9640 1 -- 172.21.15.73:0/3938906176 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc027020 con 0x55e22f305170 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37d8ff9640 1 -- 172.21.15.73:0/3938906176 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37cc01b040 con 0x55e22f305170 2024-06-06T07:14:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.932+0000 7f37d8ff9640 1 -- 172.21.15.73:0/3938906176 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc0136f0 con 0x55e22f305170 2024-06-06T07:14:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3938906176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 msgr2=0x55e22f30db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00a010 tx=0x7f37cc015420 comp rx=0 tx=0).stop 2024-06-06T07:14:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3938906176 shutdown_connections 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f3287b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f328270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/3938906176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f30db10 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3938906176 >> 172.21.15.73:0/3938906176 conn(0x55e22f301680 msgr2=0x55e22f308590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3938906176 shutdown_connections 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- 172.21.15.73:0/3938906176 wait complete. 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- start start 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 Processor -- start 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f313df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f389230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41b0040 con 0x55e22f305170 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41af910 con 0x55e22f310350 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f37e41acaf0 con 0x55e22f305a90 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.933+0000 7f37ec1b8740 10 client.0 _ll_register_callbacks cb 0x7f37d4009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53346/0 (socket says 172.21.15.73:53346) 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 -- 172.21.15.73:0/1457828562 learned_addr learned my addr 172.21.15.73:0/1457828562 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f313df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 -- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 msgr2=0x55e22f313df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f313df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 -- 172.21.15.73:0/1457828562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 msgr2=0x55e22f389230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:14:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f389230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 -- 172.21.15.73:0/1457828562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e22f3897a0 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f313df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37dbfff640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009d50 tx=0x7f37cc010540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=747da43b775dab6f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37cc027030 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(119..119 src has 1..119) ==== 9466+0+0 (secure 0 0 0) 0x7f37cc005350 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37cc01b040 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) ==== 24+0+0 (secure 0 0 0) 0x7f37cc005550 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ec1b8740 10 client.16974 fetch_fsmap learned FSMap version 50 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) -- 0x55e22f38aa90 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37cc01b040 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.934+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 50) ==== 8386+0+0 (secure 0 0 0) 0x7f37cc00c070 con 0x55e22f305170 2024-06-06T07:14:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 10 client.16974 fetch_fsmap finished waiting for FSMap version 50 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 10 client.16974 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 20 client.16974 populate_metadata read hostname 'smithi073' 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f38af60 0x55e22f38d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 4 client.16974 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e22f276c20 con 0x55e22f38af60 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.935+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f38af60 0x55e22f38d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.936+0000 7f37e2ffd640 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f38af60 0x55e22f38d380 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a7c0c1f2212099d server_cookie=4b001097b683a5d2 in_seq=0 out_seq=0 2024-06-06T07:14:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:51.936+0000 7f37ca7fc640 10 client.16974 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ca7fc640 1 -- 172.21.15.73:0/1457828562 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e22f276c20 con 0x55e22f38af60 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ca7fc640 10 client.16974 handle_command_reply: tid=0 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 client.16974 shutdown 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 msgr2=0x55e22f3138b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f37cc009d50 tx=0x7f37cc010540 comp rx=0 tx=0).stop 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 shutdown_connections 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e22f38af60 0x55e22f38d380 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e22f310350 0x55e22f389230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ca7fc640 0 client.16974 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:14:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e22f305a90 0x55e22f313df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 --2- 172.21.15.73:0/1457828562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e22f305170 0x55e22f3138b0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 >> 172.21.15.73:0/1457828562 conn(0x55e22f301680 msgr2=0x55e22f30eaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 shutdown_connections 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 1 -- 172.21.15.73:0/1457828562 wait complete. 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.055+0000 7f37ec1b8740 20 client.16974 trim_cache size 0 max 16384 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:14:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:14:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 msgr2=0x7f37d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 0x7f37d00a5260 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f37dc0cd0a0 tx=0x7f37dc091250 comp rx=0 tx=0).stop 2024-06-06T07:14:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 msgr2=0x7f37e419c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f37dc00cce0 tx=0x7f37dc00a620 comp rx=0 tx=0).stop 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 shutdown_connections 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37e4105840 0x7f37e419c8a0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f37e41015b0 0x7f37e4198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f37d00a2da0 0x7f37d00a5260 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 --2- 172.21.15.73:0/2471710564 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f37e4100bf0 0x7f37e4198030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 >> 172.21.15.73:0/2471710564 conn(0x7f37e40766a0 msgr2=0x7f37e41044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 shutdown_connections 2024-06-06T07:14:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:14:52.056+0000 7f37e93ce640 1 -- 172.21.15.73:0/2471710564 wait complete. 2024-06-06T07:14:52.462 INFO:tasks.workunit.client.0.smithi073.stdout: 1 244867 1.68 MB/sec execute 599 sec latency 2653.101 ms 2024-06-06T07:14:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:53 smithi073 ceph-mon[32303]: pgmap v426: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:53 smithi136 ceph-mon[35901]: pgmap v426: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:53.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:53 smithi170 ceph-mon[36108]: pgmap v426: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 608 KiB/s wr, 75 op/s 2024-06-06T07:14:53.463 INFO:tasks.workunit.client.0.smithi073.stdout: 1 cleanup 600 sec 2024-06-06T07:14:54.463 INFO:tasks.workunit.client.0.smithi073.stdout: 1 cleanup 601 sec 2024-06-06T07:14:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:55 smithi073 ceph-mon[32303]: pgmap v427: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 650 KiB/s wr, 91 op/s 2024-06-06T07:14:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:55 smithi136 ceph-mon[35901]: pgmap v427: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 650 KiB/s wr, 91 op/s 2024-06-06T07:14:55.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:55 smithi170 ceph-mon[36108]: pgmap v427: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 650 KiB/s wr, 91 op/s 2024-06-06T07:14:55.463 INFO:tasks.workunit.client.0.smithi073.stdout: 1 cleanup 602 sec 2024-06-06T07:14:55.695 INFO:tasks.workunit.client.0.smithi073.stdout: 0 cleanup 602 sec 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: Operation Count AvgLat MaxLat 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: ---------------------------------------- 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: NTCreateX 32240 3.940 686.041 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: Close 23704 0.047 2.208 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: Rename 1367 155.748 4768.392 2024-06-06T07:14:55.696 INFO:tasks.workunit.client.0.smithi073.stdout: Unlink 6488 4.009 15.301 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Qpathinfo 29271 3.155 67.221 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Qfileinfo 5119 0.070 0.764 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Qfsinfo 5361 0.297 5.283 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Sfileinfo 2616 1.325 19.161 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Find 11323 10.236 89.240 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: WriteX 15950 0.568 77.993 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: ReadX 50764 0.079 7.638 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: LockX 106 0.650 1.187 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: UnlockX 106 0.536 0.946 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: Flush 2255 3.056 18.199 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:14:55.697 INFO:tasks.workunit.client.0.smithi073.stdout:Throughput 1.68111 MB/sec 1 clients 1 procs max_latency=4768.403 ms 2024-06-06T07:14:55.698 INFO:tasks.workunit.client.0.smithi073.stderr:+ dbench 10 2024-06-06T07:14:55.698 INFO:tasks.workunit.client.0.smithi073.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-06T07:14:55.699 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:14:55.699 INFO:tasks.workunit.client.0.smithi073.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-06T07:14:55.699 INFO:tasks.workunit.client.0.smithi073.stdout:8 of 10 processes prepared for launch 0 sec 2024-06-06T07:14:55.797 INFO:tasks.workunit.client.0.smithi073.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-06T07:14:55.797 INFO:tasks.workunit.client.0.smithi073.stdout:releasing clients 2024-06-06T07:14:56.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 258 97.68 MB/sec warmup 1 sec latency 83.075 ms 2024-06-06T07:14:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:57 smithi073 ceph-mon[32303]: pgmap v428: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 572 KiB/s wr, 62 op/s 2024-06-06T07:14:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:57 smithi136 ceph-mon[35901]: pgmap v428: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 572 KiB/s wr, 62 op/s 2024-06-06T07:14:57.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:57 smithi170 ceph-mon[36108]: pgmap v428: 129 pgs: 129 active+clean; 141 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 482 KiB/s rd, 572 KiB/s wr, 62 op/s 2024-06-06T07:14:57.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 470 82.26 MB/sec warmup 2 sec latency 101.302 ms 2024-06-06T07:14:58.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 658 69.11 MB/sec warmup 3 sec latency 113.199 ms 2024-06-06T07:14:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:14:59 smithi073 ceph-mon[32303]: pgmap v429: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 14 MiB/s wr, 150 op/s 2024-06-06T07:14:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:14:59 smithi136 ceph-mon[35901]: pgmap v429: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 14 MiB/s wr, 150 op/s 2024-06-06T07:14:59.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:14:59 smithi170 ceph-mon[36108]: pgmap v429: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 14 MiB/s wr, 150 op/s 2024-06-06T07:14:59.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 807 53.67 MB/sec warmup 4 sec latency 118.767 ms 2024-06-06T07:15:00.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:00 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:00.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:00 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:00.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:00 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:00.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 882 43.32 MB/sec warmup 5 sec latency 414.161 ms 2024-06-06T07:15:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:01 smithi136 ceph-mon[35901]: pgmap v430: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:01.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:01 smithi170 ceph-mon[36108]: pgmap v430: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:01 smithi073 ceph-mon[32303]: pgmap v430: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:01.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 985 36.78 MB/sec warmup 6 sec latency 144.329 ms 2024-06-06T07:15:02.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:02 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:02 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:02 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:02.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1083 32.62 MB/sec warmup 7 sec latency 242.251 ms 2024-06-06T07:15:03.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:03 smithi136 ceph-mon[35901]: pgmap v431: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:03.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:03 smithi170 ceph-mon[36108]: pgmap v431: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:03 smithi073 ceph-mon[32303]: pgmap v431: 129 pgs: 129 active+clean; 298 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 13 MiB/s wr, 103 op/s 2024-06-06T07:15:03.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1177 28.77 MB/sec warmup 8 sec latency 166.817 ms 2024-06-06T07:15:04.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1339 28.22 MB/sec warmup 9 sec latency 1117.999 ms 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 -- 172.21.15.73:0/1931649553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe01008d0 msgr2=0x7f1fe0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/1931649553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe01008d0 0x7f1fe0100d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009930 tx=0x7f1fd402a420 comp rx=0 tx=0).stop 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 -- 172.21.15.73:0/1931649553 shutdown_connections 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/1931649553 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe0101290 0x7f1fe010d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/1931649553 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe01008d0 0x7f1fe0100d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/1931649553 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe0106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.171 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 -- 172.21.15.73:0/1931649553 >> 172.21.15.73:0/1931649553 conn(0x7f1fe0077a20 msgr2=0x7f1fe0077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 -- 172.21.15.73:0/1931649553 shutdown_connections 2024-06-06T07:15:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.170+0000 7f1fe4c15640 1 -- 172.21.15.73:0/1931649553 wait complete. 2024-06-06T07:15:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 Processor -- start 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 -- start start 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 0x7f1fe0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0101290 0x7f1fe01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110ba0 con 0x7f1fe0106890 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110a20 con 0x7f1fe01008d0 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.171+0000 7f1fe4c15640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0110d20 con 0x7f1fe0101290 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fde59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 0x7f1fe0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0101290 0x7f1fe01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44786/0 (socket says 172.21.15.73:44786) 2024-06-06T07:15:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 -- 172.21.15.73:0/3426612840 learned_addr learned my addr 172.21.15.73:0/3426612840 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 0x7f1fe0198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47444/0 (socket says 172.21.15.73:47444) 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 -- 172.21.15.73:0/3426612840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0101290 msgr2=0x7f1fe01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0101290 0x7f1fe01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 -- 172.21.15.73:0/3426612840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 msgr2=0x7f1fe0198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 0x7f1fe0198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 -- 172.21.15.73:0/3426612840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe01a4560 con 0x7f1fe0106890 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.172+0000 7f1fded9d640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800eb10 tx=0x7f1fd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.173+0000 7f1fcf7fe640 1 -- 172.21.15.73:0/3426612840 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004b20 con 0x7f1fe0106890 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.173+0000 7f1fcf7fe640 1 -- 172.21.15.73:0/3426612840 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd8004cc0 con 0x7f1fe0106890 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.173+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe01a4730 con 0x7f1fe0106890 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.173+0000 7f1fcf7fe640 1 -- 172.21.15.73:0/3426612840 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8010690 con 0x7f1fe0106890 2024-06-06T07:15:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.173+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe0196d90 con 0x7f1fe0106890 2024-06-06T07:15:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.176+0000 7f1fcf7fe640 1 -- 172.21.15.73:0/3426612840 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1fd8005020 con 0x7f1fe0106890 2024-06-06T07:15:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.176+0000 7f1fe79ff740 1 Processor -- start 2024-06-06T07:15:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.176+0000 7f1fe79ff740 1 -- start start 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x560696192550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5606960f64f0 con 0x56069619d350 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560696094870 con 0x560696192170 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5606960f24b0 con 0x560696192a90 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48226/0 (socket says 172.21.15.73:48226) 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44790/0 (socket says 172.21.15.73:44790) 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 -- 172.21.15.73:0/3090710372 learned_addr learned my addr 172.21.15.73:0/3090710372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:05.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 -- 172.21.15.73:0/3090710372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 msgr2=0x560696192550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x560696192550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 -- 172.21.15.73:0/3090710372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 msgr2=0x56069619f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fde59c640 1 -- 172.21.15.73:0/3090710372 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606960b09b0 con 0x560696192a90 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.177+0000 7f1fcf7fe640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 0x7f1fc80a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009c70 tx=0x7f1fd0011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=20c447cd697b4d2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fcf7fe640 1 -- 172.21.15.73:0/3426612840 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 9642+0+0 (secure 0 0 0) 0x7f1fd80cff30 con 0x7f1fe0106890 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 0x7f1fc80a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fccff9640 1 -- 172.21.15.73:0/3090710372 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0004230 con 0x560696192a90 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fccff9640 1 -- 172.21.15.73:0/3090710372 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd00043d0 con 0x560696192a90 2024-06-06T07:15:05.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.178+0000 7f1fccff9640 1 -- 172.21.15.73:0/3090710372 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0019730 con 0x560696192a90 2024-06-06T07:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3090710372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 msgr2=0x56069619ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 0x7f1fc80a96f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4031440 tx=0x7f1fd4005780 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009c70 tx=0x7f1fd0011810 comp rx=0 tx=0).stop 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3090710372 shutdown_connections 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619ce10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3090710372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x560696192550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3090710372 >> 172.21.15.73:0/3090710372 conn(0x56069618e680 msgr2=0x56069618ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3090710372 shutdown_connections 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3090710372 wait complete. 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 -- start start 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 Processor -- start 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069619a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.179+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56069619ff80 con 0x56069619d350 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56069607fae0 con 0x560696192170 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5606961a0100 con 0x560696192a90 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fe79ff740 10 client.0 _ll_register_callbacks cb 0x5606960b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48236/0 (socket says 172.21.15.73:48236) 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 -- 172.21.15.73:0/862748186 learned_addr learned my addr 172.21.15.73:0/862748186 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fde59c640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069619a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fded9d640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 -- 172.21.15.73:0/862748186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 msgr2=0x56069619a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069619a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 -- 172.21.15.73:0/862748186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 msgr2=0x56069619ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 -- 172.21.15.73:0/862748186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560695e0a210 con 0x560696192a90 2024-06-06T07:15:05.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fded9d640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009930 tx=0x7f1fd4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2381f026417384b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.180+0000 7f1fc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4003e60 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9642+0+0 (secure 0 0 0) 0x7f1fd40316a0 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd00043d0 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1fd40318a0 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 10 client.26410 fetch_fsmap learned FSMap version 52 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) -- 0x560695ed6620 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd404c070 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 52) ==== 8394+0+0 (secure 0 0 0) 0x7f1fd4044030 con 0x560696192a90 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 10 client.26410 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 10 client.26410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:05.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 20 client.26410 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5606961ad5b0 0x5606961af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 4 client.26410 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1fd00043d0 con 0x5606961ad5b0 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.181+0000 7f1fde59c640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5606961ad5b0 0x5606961af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.182+0000 7f1fc5ffb640 10 client.26410 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:05.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.182+0000 7f1fde59c640 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5606961ad5b0 0x5606961af9a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec9fe58638b8260 server_cookie=849b5febd7019023 in_seq=0 out_seq=0 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/862748186 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f1fd00043d0 con 0x5606961ad5b0 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fc5ffb640 10 client.26410 handle_command_reply: tid=0 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 client.26410 shutdown 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 msgr2=0x56069619a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009930 tx=0x7f1fd4009260 comp rx=0 tx=0).stop 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 shutdown_connections 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5606961ad5b0 0x5606961af9a0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.188 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x56069619ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x56069619a830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/862748186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069619a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fc5ffb640 0 client.26410 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 >> 172.21.15.73:0/862748186 conn(0x56069618e680 msgr2=0x5606961943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 shutdown_connections 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 1 -- 172.21.15.73:0/862748186 wait complete. 2024-06-06T07:15:05.189 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.184+0000 7f1fe79ff740 20 client.26410 trim_cache size 0 max 16384 2024-06-06T07:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 Processor -- start 2024-06-06T07:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 -- start start 2024-06-06T07:15:05.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069618df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56069619d350 0x5606961b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01ace60 con 0x560696192a90 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01aeac0 con 0x560696192170 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01ad560 con 0x56069619d350 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44806/0 (socket says 172.21.15.73:44806) 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 -- 172.21.15.73:0/1024856388 learned_addr learned my addr 172.21.15.73:0/1024856388 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fded9d640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56069619d350 0x5606961b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fde59c640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069618df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 -- 172.21.15.73:0/1024856388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56069619d350 msgr2=0x5606961b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56069619d350 0x5606961b57d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 -- 172.21.15.73:0/1024856388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 msgr2=0x56069618df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069618df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fddd9b640 1 -- 172.21.15.73:0/1024856388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56069618e4a0 con 0x560696192a90 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.202+0000 7f1fde59c640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069618df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.203+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003610 tx=0x7f1fd4003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b527c607b92a9e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.203+0000 7f1fc7fff640 1 -- 172.21.15.73:0/1024856388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd40314a0 con 0x560696192a90 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.203+0000 7f1fc7fff640 1 -- 172.21.15.73:0/1024856388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1fd404c040 con 0x560696192a90 2024-06-06T07:15:05.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.203+0000 7f1fc7fff640 1 -- 172.21.15.73:0/1024856388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4048ba0 con 0x560696192a90 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- 172.21.15.73:0/1024856388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 msgr2=0x5606961b5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4003610 tx=0x7f1fd4003a10 comp rx=0 tx=0).stop 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- 172.21.15.73:0/1024856388 shutdown_connections 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56069619d350 0x5606961b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560696192a90 0x5606961b5290 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/1024856388 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x56069618df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- 172.21.15.73:0/1024856388 >> 172.21.15.73:0/1024856388 conn(0x56069618e680 msgr2=0x5606961955d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- 172.21.15.73:0/1024856388 shutdown_connections 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- 172.21.15.73:0/1024856388 wait complete. 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 -- start start 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.204+0000 7f1fe79ff740 1 Processor -- start 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x560696216250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01ae3f0 con 0x56069619d350 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01adcc0 con 0x560696192170 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01aae60 con 0x560696192a90 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fde59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fde59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47472/0 (socket says 172.21.15.73:47472) 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fded9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x560696216250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48244/0 (socket says 172.21.15.73:48244) 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fde59c640 1 -- 172.21.15.73:0/3250087103 learned_addr learned my addr 172.21.15.73:0/3250087103 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 -- 172.21.15.73:0/3250087103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 msgr2=0x5606961a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 -- 172.21.15.73:0/3250087103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 msgr2=0x560696216250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x560696216250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 -- 172.21.15.73:0/3250087103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560696216940 con 0x560696192a90 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 10 client.0 _ll_register_callbacks cb 0x7f1fd4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd404c040 con 0x560696192a90 2024-06-06T07:15:05.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fded9d640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x560696216250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fddd9b640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x56069619b870 tx=0x7f1fd4048500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1c3575ed82fa1fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.205+0000 7f1fc5ffb640 10 client.26416 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/3250087103 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd406d050 con 0x560696192a90 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/3250087103 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 9642+0+0 (secure 0 0 0) 0x7f1fd4048820 con 0x560696192a90 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/3250087103 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1fd4048a60 con 0x560696192a90 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 10 client.26416 fetch_fsmap learned FSMap version 52 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({fsmap=52}) -- 0x5606962179e0 con 0x560696192a90 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/3250087103 <== mon.1 v2:172.21.15.170:3300/0 4 ==== fsmap(e 52) ==== 8394+0+0 (secure 0 0 0) 0x7f1fd4044070 con 0x560696192a90 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 10 client.26416 fetch_fsmap finished waiting for FSMap version 52 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 10 client.26416 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 20 client.26416 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560696218080 0x56069621a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:05.211 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 4 client.26416 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:15:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560696109f10 con 0x560696218080 2024-06-06T07:15:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.206+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560696218080 0x56069621a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.207+0000 7f1fc5ffb640 10 client.26416 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:05.212 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:05.207+0000 7f1fde59c640 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560696218080 0x56069621a4a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d2e7235f3e06e42 server_cookie=7df9cecb3f0ab97 in_seq=0 out_seq=0 2024-06-06T07:15:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:05 smithi136 ceph-mon[35901]: pgmap v432: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-06-06T07:15:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:05 smithi136 ceph-mon[35901]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T07:15:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:05 smithi170 ceph-mon[36108]: pgmap v432: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-06-06T07:15:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:05 smithi170 ceph-mon[36108]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T07:15:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:05 smithi073 ceph-mon[32303]: pgmap v432: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 766 KiB/s rd, 19 MiB/s wr, 206 op/s 2024-06-06T07:15:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:05 smithi073 ceph-mon[32303]: osdmap e120: 12 total, 12 up, 12 in 2024-06-06T07:15:05.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1425 25.44 MB/sec warmup 10 sec latency 2118.030 ms 2024-06-06T07:15:06.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1532 23.21 MB/sec warmup 11 sec latency 3118.062 ms 2024-06-06T07:15:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:07 smithi170 ceph-mon[36108]: pgmap v434: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 22 MiB/s wr, 228 op/s 2024-06-06T07:15:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:07 smithi073 ceph-mon[32303]: pgmap v434: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 22 MiB/s wr, 228 op/s 2024-06-06T07:15:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:07 smithi136 ceph-mon[35901]: pgmap v434: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 868 KiB/s rd, 22 MiB/s wr, 228 op/s 2024-06-06T07:15:07.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1700 21.40 MB/sec warmup 12 sec latency 4118.089 ms 2024-06-06T07:15:08.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 1861 20.08 MB/sec warmup 13 sec latency 5118.120 ms 2024-06-06T07:15:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:09 smithi170 ceph-mon[36108]: pgmap v435: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:09 smithi073 ceph-mon[32303]: pgmap v435: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:09 smithi136 ceph-mon[35901]: pgmap v435: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:09.797 INFO:tasks.workunit.client.0.smithi073.stdout: 10 2017 18.79 MB/sec warmup 14 sec latency 6118.149 ms 2024-06-06T07:15:10.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 2171 18.15 MB/sec warmup 15 sec latency 7118.324 ms 2024-06-06T07:15:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:11 smithi170 ceph-mon[36108]: pgmap v436: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:11 smithi073 ceph-mon[32303]: pgmap v436: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:11 smithi136 ceph-mon[35901]: pgmap v436: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:11.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 2332 17.57 MB/sec warmup 16 sec latency 8118.366 ms 2024-06-06T07:15:12.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:12 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:12 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:12.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:12 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:12.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 2494 17.02 MB/sec warmup 17 sec latency 9118.394 ms 2024-06-06T07:15:13.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 2791 18.08 MB/sec warmup 18 sec latency 10118.424 ms 2024-06-06T07:15:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:13 smithi170 ceph-mon[36108]: pgmap v437: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:13 smithi170 ceph-mon[36108]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T07:15:13.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.960+0000 7f1fc5ffb640 1 -- 172.21.15.73:0/3250087103 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560696109f10 con 0x560696218080 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.960+0000 7f1fc5ffb640 10 client.26416 handle_command_reply: tid=0 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.960+0000 7f1fe79ff740 1 client.26416 shutdown 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 msgr2=0x5606961a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x56069619b870 tx=0x7f1fd4048500 comp rx=0 tx=0).stop 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 shutdown_connections 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560696218080 0x56069621a4a0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56069619d350 0x560696216250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560696192a90 0x5606961a1080 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fc5ffb640 0 client.26416 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 --2- 172.21.15.73:0/3250087103 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560696192170 0x5606961a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 >> 172.21.15.73:0/3250087103 conn(0x56069618e680 msgr2=0x560696193b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 shutdown_connections 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 1 -- 172.21.15.73:0/3250087103 wait complete. 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe79ff740 20 client.26416 trim_cache size 0 max 16384 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:15:13.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:15:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.961+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 msgr2=0x7f1fc80a96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:13.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 0x7f1fc80a96f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4031440 tx=0x7f1fd4005780 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 msgr2=0x7f1fe01a4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800eb10 tx=0x7f1fd800c2d0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 shutdown_connections 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1fe0106890 0x7f1fe01a4020 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f1fc80a7230 0x7f1fc80a96f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1fe0101290 0x7f1fe01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 --2- 172.21.15.73:0/3426612840 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1fe01008d0 0x7f1fe0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 >> 172.21.15.73:0/3426612840 conn(0x7f1fe0077a20 msgr2=0x7f1fe0103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 shutdown_connections 2024-06-06T07:15:13.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:13.962+0000 7f1fe4c15640 1 -- 172.21.15.73:0/3426612840 wait complete. 2024-06-06T07:15:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:13 smithi073 ceph-mon[32303]: pgmap v437: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:13 smithi073 ceph-mon[32303]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T07:15:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:13 smithi136 ceph-mon[35901]: pgmap v437: 129 pgs: 129 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 7.0 MiB/s wr, 418 op/s 2024-06-06T07:15:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:13 smithi136 ceph-mon[35901]: osdmap e121: 12 total, 12 up, 12 in 2024-06-06T07:15:14.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3006 17.50 MB/sec warmup 19 sec latency 10293.673 ms 2024-06-06T07:15:15.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3185 17.52 MB/sec warmup 20 sec latency 106.579 ms 2024-06-06T07:15:15.827 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:15 smithi073 ceph-mon[32303]: pgmap v439: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 388 op/s 2024-06-06T07:15:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:15 smithi170 ceph-mon[36108]: pgmap v439: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 388 op/s 2024-06-06T07:15:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:15 smithi136 ceph-mon[35901]: pgmap v439: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 887 KiB/s rd, 2.8 MiB/s wr, 388 op/s 2024-06-06T07:15:16.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3303 16.86 MB/sec warmup 21 sec latency 94.500 ms 2024-06-06T07:15:17.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3407 16.28 MB/sec warmup 22 sec latency 115.819 ms 2024-06-06T07:15:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:17 smithi170 ceph-mon[36108]: pgmap v440: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 382 op/s 2024-06-06T07:15:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:17 smithi073 ceph-mon[32303]: pgmap v440: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 382 op/s 2024-06-06T07:15:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:17 smithi136 ceph-mon[35901]: pgmap v440: 129 pgs: 129 active+clean; 371 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 874 KiB/s rd, 2.8 MiB/s wr, 382 op/s 2024-06-06T07:15:18.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3603 16.37 MB/sec warmup 23 sec latency 99.222 ms 2024-06-06T07:15:19.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3788 16.13 MB/sec warmup 24 sec latency 130.322 ms 2024-06-06T07:15:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:19 smithi170 ceph-mon[36108]: pgmap v441: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:20.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:19 smithi073 ceph-mon[32303]: pgmap v441: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:20.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:19 smithi136 ceph-mon[35901]: pgmap v441: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:20.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 3936 15.77 MB/sec warmup 25 sec latency 227.181 ms 2024-06-06T07:15:21.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:20 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:21.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:20 smithi073 ceph-mon[32303]: pgmap v442: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:21.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:20 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:21.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:20 smithi136 ceph-mon[35901]: pgmap v442: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:21.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:20 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:21.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:20 smithi170 ceph-mon[36108]: pgmap v442: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:21.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4094 15.40 MB/sec warmup 26 sec latency 90.450 ms 2024-06-06T07:15:22.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:21 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:22.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:21 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:22.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:21 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:22.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4243 15.03 MB/sec warmup 27 sec latency 121.861 ms 2024-06-06T07:15:23.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:22 smithi073 ceph-mon[32303]: pgmap v443: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:23.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:22 smithi136 ceph-mon[35901]: pgmap v443: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:23.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:22 smithi170 ceph-mon[36108]: pgmap v443: 129 pgs: 129 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.9 MiB/s wr, 189 op/s 2024-06-06T07:15:23.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4418 14.66 MB/sec warmup 28 sec latency 82.894 ms 2024-06-06T07:15:24.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4581 14.36 MB/sec warmup 29 sec latency 144.731 ms 2024-06-06T07:15:25.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:24 smithi073 ceph-mon[32303]: pgmap v444: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 277 op/s 2024-06-06T07:15:25.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:24 smithi136 ceph-mon[35901]: pgmap v444: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 277 op/s 2024-06-06T07:15:25.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:24 smithi170 ceph-mon[36108]: pgmap v444: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.9 MiB/s wr, 277 op/s 2024-06-06T07:15:25.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4756 14.15 MB/sec warmup 30 sec latency 84.062 ms 2024-06-06T07:15:26.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 4941 14.01 MB/sec warmup 31 sec latency 102.351 ms 2024-06-06T07:15:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/718207431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf581007d0 msgr2=0x7fcf58100bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/718207431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf581007d0 0x7fcf58100bb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009900 tx=0x7fcf4802a410 comp rx=0 tx=0).stop 2024-06-06T07:15:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/718207431 shutdown_connections 2024-06-06T07:15:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/718207431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5810e380 0x7fcf58110770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/718207431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5810de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/718207431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf581007d0 0x7fcf58100bb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/718207431 >> 172.21.15.73:0/718207431 conn(0x7fcf58077a20 msgr2=0x7fcf58077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/718207431 shutdown_connections 2024-06-06T07:15:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.089+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/718207431 wait complete. 2024-06-06T07:15:27.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 Processor -- start 2024-06-06T07:15:27.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 -- start start 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf581007d0 0x7fcf5819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf5810e380 0x7fcf5819ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58113e40 con 0x7fcf581007d0 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58113cc0 con 0x7fcf581010f0 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf5d3e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf58113fc0 con 0x7fcf5810e380 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.090+0000 7fcf56ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf581007d0 0x7fcf5819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59706/0 (socket says 172.21.15.73:59706) 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 -- 172.21.15.73:0/1557242581 learned_addr learned my addr 172.21.15.73:0/1557242581 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf577fe640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf5810e380 0x7fcf5819ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 -- 172.21.15.73:0/1557242581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf5810e380 msgr2=0x7fcf5819ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf5810e380 0x7fcf5819ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 -- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf581007d0 msgr2=0x7fcf5819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf581007d0 0x7fcf5819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 -- 172.21.15.73:0/1557242581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5819f2f0 con 0x7fcf581010f0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf567fc640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d950 tx=0x7fcf4c00dd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf3ffff640 1 -- 172.21.15.73:0/1557242581 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c017070 con 0x7fcf581010f0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5819f5e0 con 0x7fcf581010f0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf3ffff640 1 -- 172.21.15.73:0/1557242581 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf4c00de80 con 0x7fcf581010f0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf3ffff640 1 -- 172.21.15.73:0/1557242581 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c00f620 con 0x7fcf581010f0 2024-06-06T07:15:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.091+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf581a6e70 con 0x7fcf581010f0 2024-06-06T07:15:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.094+0000 7fcf601d2740 1 Processor -- start 2024-06-06T07:15:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.094+0000 7fcf3ffff640 1 -- 172.21.15.73:0/1557242581 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcf4c009c30 con 0x7fcf581010f0 2024-06-06T07:15:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 -- start start 2024-06-06T07:15:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b61e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b64740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3abb4f0 con 0x55b3a3b57a90 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3a59870 con 0x55b3a3b57170 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf601d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3ab74b0 con 0x55b3a3b62350 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59712/0 (socket says 172.21.15.73:59712) 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 -- 172.21.15.73:0/1859150438 learned_addr learned my addr 172.21.15.73:0/1859150438 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf567fc640 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b64740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b61e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 -- 172.21.15.73:0/1859150438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 msgr2=0x55b3a3b64740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b64740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 -- 172.21.15.73:0/1859150438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 msgr2=0x55b3a3b61e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b61e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf577fe640 1 -- 172.21.15.73:0/1859150438 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a3a759b0 con 0x55b3a3b57170 2024-06-06T07:15:27.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.095+0000 7fcf3ffff640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 0x7fcf400a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 0x7fcf400a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf577fe640 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009f40 tx=0x7fcf500118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19781d42d3b21a25 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf3ffff640 1 -- 172.21.15.73:0/1557242581 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fcf4c0cf410 con 0x7fcf581010f0 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf3d7fa640 1 -- 172.21.15.73:0/1859150438 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50004d20 con 0x55b3a3b57170 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf3d7fa640 1 -- 172.21.15.73:0/1859150438 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf50004050 con 0x55b3a3b57170 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.096+0000 7fcf3d7fa640 1 -- 172.21.15.73:0/1859150438 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50005260 con 0x55b3a3b57170 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- 172.21.15.73:0/1859150438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 msgr2=0x55b3a3b57550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 0x7fcf400a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf48031440 tx=0x7fcf48005b90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009f40 tx=0x7fcf500118d0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- 172.21.15.73:0/1859150438 shutdown_connections 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b64740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b61e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 --2- 172.21.15.73:0/1859150438 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b57550 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.100 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- 172.21.15.73:0/1859150438 >> 172.21.15.73:0/1859150438 conn(0x55b3a3b53680 msgr2=0x55b3a3b53a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- 172.21.15.73:0/1859150438 shutdown_connections 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- 172.21.15.73:0/1859150438 wait complete. 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 -- start start 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.097+0000 7fcf601d2740 1 Processor -- start 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b5fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3b64f80 con 0x55b3a3b62350 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf56ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf56ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59722/0 (socket says 172.21.15.73:59722) 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf56ffd640 1 -- 172.21.15.73:0/339246419 learned_addr learned my addr 172.21.15.73:0/339246419 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b5fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3a44ae0 con 0x55b3a3b57170 2024-06-06T07:15:27.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55b3a3b65100 con 0x55b3a3b57a90 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf567fc640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf601d2740 10 client.0 _ll_register_callbacks cb 0x55b3a3a7d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 -- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 msgr2=0x55b3a3b5f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 -- 172.21.15.73:0/339246419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 msgr2=0x55b3a3b5f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf577fe640 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a37cf210 con 0x55b3a3b62350 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.098+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf577fe640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b5fd30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009f10 tx=0x7fcf50004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85eafb47f484cfb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf50029020 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fcf50005040 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf50004050 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcf50034050 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf601d2740 10 client.17004 fetch_fsmap learned FSMap version 55 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b3a389b620 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.099+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf50003e40 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fcf50025470 con 0x55b3a3b62350 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 10 client.17004 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 10 client.17004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 20 client.17004 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:27.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3b72760 0x55b3a3b74b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 4 client.17004 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcf50004050 con 0x55b3a3b72760 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3b72760 0x55b3a3b74b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf3aff5640 10 client.17004 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.100+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3b72760 0x55b3a3b74b50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6825eb0b8275eb0 server_cookie=7d5942e7690b963d in_seq=0 out_seq=0 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf3aff5640 1 -- 172.21.15.73:0/339246419 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fcf50004050 con 0x55b3a3b72760 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf3aff5640 10 client.17004 handle_command_reply: tid=0 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 client.17004 shutdown 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 msgr2=0x55b3a3b5fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b5fd30 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcf50009f10 tx=0x7fcf50004cf0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 shutdown_connections 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3b72760 0x55b3a3b74b50 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b5fd30 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 --2- 172.21.15.73:0/339246419 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 >> 172.21.15.73:0/339246419 conn(0x55b3a3b53680 msgr2=0x55b3a3b59400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.103+0000 7fcf3aff5640 0 client.17004 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.104+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 shutdown_connections 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.104+0000 7fcf601d2740 1 -- 172.21.15.73:0/339246419 wait complete. 2024-06-06T07:15:27.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.104+0000 7fcf601d2740 20 client.17004 trim_cache size 0 max 16384 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.125+0000 7fcf601d2740 1 Processor -- start 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.125+0000 7fcf601d2740 1 -- start start 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b5c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581b1120 con 0x55b3a3b57a90 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581b2e10 con 0x55b3a3b57170 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf601d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581b1850 con 0x55b3a3b62350 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35036/0 (socket says 172.21.15.73:35036) 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 -- 172.21.15.73:0/2096805014 learned_addr learned my addr 172.21.15.73:0/2096805014 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf577fe640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b5c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 -- 172.21.15.73:0/2096805014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 msgr2=0x55b3a3b5c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b5c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 -- 172.21.15.73:0/2096805014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 msgr2=0x55b3a3b5db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 -- 172.21.15.73:0/2096805014 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3a3b59aa0 con 0x55b3a3b57a90 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.126+0000 7fcf567fc640 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0cc7b0 tx=0x7fcf4c0cf610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6246ec14b49824 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.127+0000 7fcf3cff9640 1 -- 172.21.15.73:0/2096805014 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c020050 con 0x55b3a3b57a90 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.127+0000 7fcf3cff9640 1 -- 172.21.15.73:0/2096805014 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf4c09bc20 con 0x55b3a3b57a90 2024-06-06T07:15:27.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.127+0000 7fcf3cff9640 1 -- 172.21.15.73:0/2096805014 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c0a4190 con 0x55b3a3b57a90 2024-06-06T07:15:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.127+0000 7fcf601d2740 1 -- 172.21.15.73:0/2096805014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 msgr2=0x55b3a3b5e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.127+0000 7fcf601d2740 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0cc7b0 tx=0x7fcf4c0cf610 comp rx=0 tx=0).stop 2024-06-06T07:15:27.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 -- 172.21.15.73:0/2096805014 shutdown_connections 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b62350 0x55b3a3b5c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b57a90 0x55b3a3b5e0c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 --2- 172.21.15.73:0/2096805014 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b5db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 -- 172.21.15.73:0/2096805014 >> 172.21.15.73:0/2096805014 conn(0x55b3a3b53680 msgr2=0x55b3a3b5a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 -- 172.21.15.73:0/2096805014 shutdown_connections 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 -- 172.21.15.73:0/2096805014 wait complete. 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 -- start start 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.128+0000 7fcf601d2740 1 Processor -- start 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b74980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581b2740 con 0x55b3a3b62350 2024-06-06T07:15:27.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581b1fb0 con 0x55b3a3b57170 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcf581af120 con 0x55b3a3b57a90 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf601d2740 10 client.0 _ll_register_callbacks cb 0x7fcf48009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b74980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf577fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59746/0 (socket says 172.21.15.73:59746) 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf567fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b74980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56498/0 (socket says 172.21.15.73:56498) 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 -- 172.21.15.73:0/3543346357 learned_addr learned my addr 172.21.15.73:0/3543346357 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 -- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 msgr2=0x55b3a3b74980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b74980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 -- 172.21.15.73:0/3543346357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 msgr2=0x55b3a3b74ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b74ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 -- 172.21.15.73:0/3543346357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3a3bdb090 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf577fe640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b74ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf56ffd640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009230 tx=0x7fcf4803f070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ffb55f5fba57638 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.129+0000 7fcf3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf48035040 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fcf48005040 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf4c09bc20 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7fcf48061050 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 10 client.26249 fetch_fsmap learned FSMap version 55 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=4+}) -- 0x55b3a3bdc310 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf4c09bc20 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 55) ==== 8410+0+0 (secure 0 0 0) 0x7fcf480035f0 con 0x55b3a3b57170 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 10 client.26249 fetch_fsmap finished waiting for FSMap version 55 2024-06-06T07:15:27.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 10 client.26249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 20 client.26249 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3bdcb50 0x55b3a3bdef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 4 client.26249 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.130+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3a3acef10 con 0x55b3a3bdcb50 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.131+0000 7fcf567fc640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3bdcb50 0x55b3a3bdef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.131+0000 7fcf3aff5640 10 client.26249 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:27.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.131+0000 7fcf567fc640 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3bdcb50 0x55b3a3bdef70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a13a97e996c5eb5 server_cookie=ad9bc166e3005c0b in_seq=0 out_seq=0 2024-06-06T07:15:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:26 smithi073 ceph-mon[32303]: pgmap v445: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 188 op/s 2024-06-06T07:15:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:26 smithi136 ceph-mon[35901]: pgmap v445: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 188 op/s 2024-06-06T07:15:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:26 smithi170 ceph-mon[36108]: pgmap v445: 129 pgs: 129 active+clean; 387 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 188 op/s 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf3aff5640 1 -- 172.21.15.73:0/3543346357 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b3a3acef10 con 0x55b3a3bdcb50 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf3aff5640 10 client.26249 handle_command_reply: tid=0 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 client.26249 shutdown 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 msgr2=0x55b3a3b74440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcf48009230 tx=0x7fcf4803f070 comp rx=0 tx=0).stop 2024-06-06T07:15:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 shutdown_connections 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55b3a3bdcb50 0x55b3a3bdef70 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b3a3b62350 0x55b3a3b74ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55b3a3b57a90 0x55b3a3b74980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 --2- 172.21.15.73:0/3543346357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55b3a3b57170 0x55b3a3b74440 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf3aff5640 0 client.26249 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 >> 172.21.15.73:0/3543346357 conn(0x55b3a3b53680 msgr2=0x55b3a3b58a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 shutdown_connections 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 1 -- 172.21.15.73:0/3543346357 wait complete. 2024-06-06T07:15:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.627+0000 7fcf601d2740 20 client.26249 trim_cache size 0 max 16384 2024-06-06T07:15:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:15:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:15:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:15:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 msgr2=0x7fcf400a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 0x7fcf400a5260 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcf48031440 tx=0x7fcf48005b90 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 msgr2=0x7fcf5819a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c00d950 tx=0x7fcf4c00dd50 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 shutdown_connections 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcf400a2da0 0x7fcf400a5260 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcf5810e380 0x7fcf5819ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcf581010f0 0x7fcf5819a840 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 --2- 172.21.15.73:0/1557242581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf581007d0 0x7fcf5819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.628+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 >> 172.21.15.73:0/1557242581 conn(0x7fcf58077a20 msgr2=0x7fcf5810f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:27.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.629+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 shutdown_connections 2024-06-06T07:15:27.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:27.629+0000 7fcf5d3e7640 1 -- 172.21.15.73:0/1557242581 wait complete. 2024-06-06T07:15:27.798 INFO:tasks.workunit.client.0.smithi073.stdout: 10 5094 13.92 MB/sec warmup 32 sec latency 207.440 ms 2024-06-06T07:15:28.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 5275 13.83 MB/sec warmup 33 sec latency 67.903 ms 2024-06-06T07:15:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:28 smithi073 ceph-mon[32303]: pgmap v446: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 341 op/s 2024-06-06T07:15:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:28 smithi136 ceph-mon[35901]: pgmap v446: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 341 op/s 2024-06-06T07:15:29.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:28 smithi170 ceph-mon[36108]: pgmap v446: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 341 op/s 2024-06-06T07:15:29.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 5479 13.82 MB/sec warmup 34 sec latency 75.328 ms 2024-06-06T07:15:30.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 5643 13.54 MB/sec warmup 35 sec latency 57.449 ms 2024-06-06T07:15:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:30 smithi073 ceph-mon[32303]: pgmap v447: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:30 smithi136 ceph-mon[35901]: pgmap v447: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:31.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:30 smithi170 ceph-mon[36108]: pgmap v447: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:31.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 5824 13.33 MB/sec warmup 36 sec latency 60.069 ms 2024-06-06T07:15:32.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:31 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:32.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:31 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:32.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:31 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:32.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6013 13.20 MB/sec warmup 37 sec latency 134.987 ms 2024-06-06T07:15:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:32 smithi073 ceph-mon[32303]: pgmap v448: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:32 smithi136 ceph-mon[35901]: pgmap v448: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:32 smithi170 ceph-mon[36108]: pgmap v448: 129 pgs: 129 active+clean; 370 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 256 op/s 2024-06-06T07:15:33.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6159 12.98 MB/sec warmup 38 sec latency 71.012 ms 2024-06-06T07:15:34.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6342 12.96 MB/sec warmup 39 sec latency 59.569 ms 2024-06-06T07:15:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:34 smithi073 ceph-mon[32303]: pgmap v449: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-06-06T07:15:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:34 smithi136 ceph-mon[35901]: pgmap v449: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-06-06T07:15:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:34 smithi170 ceph-mon[36108]: pgmap v449: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.0 MiB/s wr, 370 op/s 2024-06-06T07:15:35.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6511 12.98 MB/sec warmup 40 sec latency 143.132 ms 2024-06-06T07:15:36.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6707 12.96 MB/sec warmup 41 sec latency 67.424 ms 2024-06-06T07:15:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:36 smithi073 ceph-mon[32303]: pgmap v450: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-06-06T07:15:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:36 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:36 smithi136 ceph-mon[35901]: pgmap v450: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-06-06T07:15:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:36 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:36 smithi170 ceph-mon[36108]: pgmap v450: 129 pgs: 129 active+clean; 381 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 266 op/s 2024-06-06T07:15:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:36 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:15:37.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 6916 13.04 MB/sec warmup 42 sec latency 163.940 ms 2024-06-06T07:15:38.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7144 13.01 MB/sec warmup 43 sec latency 68.800 ms 2024-06-06T07:15:39.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:38 smithi073 ceph-mon[32303]: pgmap v451: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2024-06-06T07:15:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:38 smithi136 ceph-mon[35901]: pgmap v451: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2024-06-06T07:15:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:38 smithi170 ceph-mon[36108]: pgmap v451: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 391 op/s 2024-06-06T07:15:39.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7286 12.86 MB/sec warmup 44 sec latency 93.428 ms 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 -- 172.21.15.73:0/2171531264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60106940 msgr2=0x7f7d60106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 --2- 172.21.15.73:0/2171531264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60106940 0x7f7d60106d20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009900 tx=0x7f7d5002a3f0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 -- 172.21.15.73:0/2171531264 shutdown_connections 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 --2- 172.21.15.73:0/2171531264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d601014b0 0x7f7d6010d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 --2- 172.21.15.73:0/2171531264 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d60100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 --2- 172.21.15.73:0/2171531264 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60106940 0x7f7d60106d20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 -- 172.21.15.73:0/2171531264 >> 172.21.15.73:0/2171531264 conn(0x7f7d60077780 msgr2=0x7f7d60077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 -- 172.21.15.73:0/2171531264 shutdown_connections 2024-06-06T07:15:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.764+0000 7f7d6507b640 1 -- 172.21.15.73:0/2171531264 wait complete. 2024-06-06T07:15:40.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 Processor -- start 2024-06-06T07:15:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 -- start start 2024-06-06T07:15:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d601014b0 0x7f7d60109e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 0x7f7d6010a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010ee50 con 0x7f7d60106940 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010ecd0 con 0x7f7d60100b10 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d6507b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6010efd0 con 0x7f7d601014b0 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d5ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 0x7f7d6010a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d5ed9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:38564/0 (socket says 172.21.15.73:38564) 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d5ed9d640 1 -- 172.21.15.73:0/3989328463 learned_addr learned my addr 172.21.15.73:0/3989328463 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.765+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d601014b0 0x7f7d60109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 -- 172.21.15.73:0/3989328463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d601014b0 msgr2=0x7f7d60109e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d601014b0 0x7f7d60109e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 -- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 msgr2=0x7f7d6010a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 0x7f7d6010a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 -- 172.21.15.73:0/3989328463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601aa940 con 0x7f7d60100b10 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 0x7f7d6010a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d5002a9e0 tx=0x7f7d50003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d5cd99640 1 -- 172.21.15.73:0/3989328463 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5003f070 con 0x7f7d60100b10 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.766+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d601aabd0 con 0x7f7d60100b10 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.767+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601ab110 con 0x7f7d60100b10 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.767+0000 7f7d5cd99640 1 -- 172.21.15.73:0/3989328463 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d50031b50 con 0x7f7d60100b10 2024-06-06T07:15:40.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.767+0000 7f7d5cd99640 1 -- 172.21.15.73:0/3989328463 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049060 con 0x7f7d60100b10 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 Processor -- start 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 -- start start 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d5cd99640 1 -- 172.21.15.73:0/3989328463 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d50042070 con 0x7f7d60100b10 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3be740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55719a3154f0 con 0x55719a3bc350 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55719a2b3870 con 0x55719a3b1170 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d67d82740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55719a3114b0 con 0x55719a3b1a90 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:38568/0 (socket says 172.21.15.73:38568) 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.770+0000 7f7d5f59e640 1 -- 172.21.15.73:0/432038945 learned_addr learned my addr 172.21.15.73:0/432038945 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d57fff640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3be740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 -- 172.21.15.73:0/432038945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 msgr2=0x55719a3bbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 -- 172.21.15.73:0/432038945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 msgr2=0x55719a3be740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3be740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 -- 172.21.15.73:0/432038945 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719a2cf9b0 con 0x55719a3b1170 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d57fff640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3be740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d58011790 tx=0x7f7d58011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c9d39807a14f5c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d5cd99640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 0x7f7d4c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d557fa640 1 -- 172.21.15.73:0/432038945 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003c30 con 0x55719a3b1170 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d557fa640 1 -- 172.21.15.73:0/432038945 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d5800fe50 con 0x55719a3b1170 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.771+0000 7f7d557fa640 1 -- 172.21.15.73:0/432038945 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d580193e0 con 0x55719a3b1170 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.772+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 0x7f7d4c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.772+0000 7f7d5cd99640 1 -- 172.21.15.73:0/3989328463 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7f7d500efc40 con 0x7f7d60100b10 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- 172.21.15.73:0/432038945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 msgr2=0x55719a3b1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d5ed9d640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 0x7f7d4c0a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ed030 tx=0x7f7d50009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7d58011790 tx=0x7f7d58011c50 comp rx=0 tx=0).stop 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- 172.21.15.73:0/432038945 shutdown_connections 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3be740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 --2- 172.21.15.73:0/432038945 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b1550 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- 172.21.15.73:0/432038945 >> 172.21.15.73:0/432038945 conn(0x55719a3ad680 msgr2=0x55719a3ada60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- 172.21.15.73:0/432038945 shutdown_connections 2024-06-06T07:15:40.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- 172.21.15.73:0/432038945 wait complete. 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 -- start start 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.773+0000 7f7d67d82740 1 Processor -- start 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55719a3bef80 con 0x55719a3b1170 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55719a29eae0 con 0x55719a3bc350 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55719a3bf100 con 0x55719a3b1a90 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d67d82740 10 client.0 _ll_register_callbacks cb 0x55719a2d79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35702/0 (socket says 172.21.15.73:35702) 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3086104394 learned_addr learned my addr 172.21.15.73:0/3086104394 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:40.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 msgr2=0x55719a3b98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3086104394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 msgr2=0x55719a3b9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3b9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.774+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3086104394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719a029210 con 0x55719a3b1170 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005f40 tx=0x7f7d58005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6a622c1cfff3cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802f060 con 0x55719a3b1170 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5800fe50 con 0x55719a3b1170 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7f7d58003e40 con 0x55719a3b1170 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f7d5801f040 con 0x55719a3b1170 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d67d82740 10 client.17022 fetch_fsmap learned FSMap version 57 2024-06-06T07:15:40.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.775+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55719a0f5620 con 0x55719a3b1170 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d5800fe50 con 0x55719a3b1170 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f7d58018070 con 0x55719a3b1170 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 10 client.17022 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 10 client.17022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 20 client.17022 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a3ce8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 4 client.17022 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:15:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.776+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55719a328f10 con 0x55719a3cc500 2024-06-06T07:15:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.778+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a3ce8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.778+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a3ce8f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2613b9de99d1402c server_cookie=f5a5c7dad734a46a in_seq=0 out_seq=0 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.778+0000 7f7d467fc640 10 client.17022 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.779+0000 7f7d467fc640 1 -- 172.21.15.73:0/3086104394 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55719a328f10 con 0x55719a3cc500 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.779+0000 7f7d467fc640 10 client.17022 handle_command_reply: tid=0 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.779+0000 7f7d67d82740 1 client.17022 shutdown 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 msgr2=0x55719a3b9370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f7d58005f40 tx=0x7f7d58005ac0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 shutdown_connections 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a3ce8f0 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3b9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d467fc640 0 client.17022 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3086104394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3b9370 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 >> 172.21.15.73:0/3086104394 conn(0x55719a3ad680 msgr2=0x55719a3b33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 shutdown_connections 2024-06-06T07:15:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 1 -- 172.21.15.73:0/3086104394 wait complete. 2024-06-06T07:15:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.780+0000 7f7d67d82740 20 client.17022 trim_cache size 0 max 16384 2024-06-06T07:15:40.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7404 12.66 MB/sec warmup 45 sec latency 102.529 ms 2024-06-06T07:15:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 Processor -- start 2024-06-06T07:15:40.807 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 -- start start 2024-06-06T07:15:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.808 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3d4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b54f0 con 0x55719a3bc350 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b7240 con 0x55719a3b1170 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d67d82740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b5c80 con 0x55719a3b1a90 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3d4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:38584/0 (socket says 172.21.15.73:38584) 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d5f59e640 1 -- 172.21.15.73:0/4055767475 learned_addr learned my addr 172.21.15.73:0/4055767475 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.807+0000 7f7d5f59e640 1 -- 172.21.15.73:0/4055767475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 msgr2=0x55719a3b9a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b9a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d5f59e640 1 -- 172.21.15.73:0/4055767475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 msgr2=0x55719a3d4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3d4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.809 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d5f59e640 1 -- 172.21.15.73:0/4055767475 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55719a3a7ae0 con 0x55719a3b1170 2024-06-06T07:15:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d580113d0 tx=0x7f7d58011c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956ffb45fff5d910 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d54ff9640 1 -- 172.21.15.73:0/4055767475 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802f070 con 0x55719a3b1170 2024-06-06T07:15:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d54ff9640 1 -- 172.21.15.73:0/4055767475 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d58027440 con 0x55719a3b1170 2024-06-06T07:15:40.810 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.808+0000 7f7d54ff9640 1 -- 172.21.15.73:0/4055767475 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58003ea0 con 0x55719a3b1170 2024-06-06T07:15:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- 172.21.15.73:0/4055767475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 msgr2=0x55719a3b84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d580113d0 tx=0x7f7d58011c50 comp rx=0 tx=0).stop 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- 172.21.15.73:0/4055767475 shutdown_connections 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3bc350 0x55719a3d4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3b9a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 --2- 172.21.15.73:0/4055767475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3b1170 0x55719a3b84d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- 172.21.15.73:0/4055767475 >> 172.21.15.73:0/4055767475 conn(0x55719a3ad680 msgr2=0x55719a3b9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- 172.21.15.73:0/4055767475 shutdown_connections 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- 172.21.15.73:0/4055767475 wait complete. 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.809+0000 7f7d67d82740 1 -- start start 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 Processor -- start 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3bfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b6b70 con 0x55719a3b1170 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b63e0 con 0x55719a3bc350 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b34f0 con 0x55719a3b1a90 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d67d82740 10 client.0 _ll_register_callbacks cb 0x7f7d58011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35716/0 (socket says 172.21.15.73:35716) 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3019950665 learned_addr learned my addr 172.21.15.73:0/3019950665 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d57fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3bfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 msgr2=0x55719a3bf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3019950665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 msgr2=0x55719a3bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3bfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d5f59e640 1 -- 172.21.15.73:0/3019950665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55719a435110 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.810+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d5f59e640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ffd0 tx=0x7f7d58000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b8e954a4db011e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d467fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5802f040 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7f7d5801a650 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d58027440 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f7d5804f050 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d67d82740 10 client.17028 fetch_fsmap learned FSMap version 57 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.811+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55719a436300 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d58027440 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f7d58018070 con 0x55719a3b1170 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 10 client.17028 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 10 client.17028 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:40.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 20 client.17028 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a438a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 4 client.17028 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55719a322c20 con 0x55719a3cc500 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.812+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a438a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.813+0000 7f7d467fc640 10 client.17028 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.813+0000 7f7d57fff640 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a438a80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a162569f78c319b0 server_cookie=66e997ee4846669a in_seq=0 out_seq=0 2024-06-06T07:15:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d467fc640 1 -- 172.21.15.73:0/3019950665 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55719a322c20 con 0x55719a3cc500 2024-06-06T07:15:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d467fc640 10 client.17028 handle_command_reply: tid=0 2024-06-06T07:15:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 client.17028 shutdown 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 msgr2=0x55719a3bf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800ffd0 tx=0x7f7d58000f40 comp rx=0 tx=0).stop 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 shutdown_connections 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55719a3cc500 0x55719a438a80 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55719a3bc350 0x55719a3bfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d467fc640 0 client.17028 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55719a3b1a90 0x55719a3bf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 --2- 172.21.15.73:0/3019950665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55719a3b1170 0x55719a3bf310 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.879+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 >> 172.21.15.73:0/3019950665 conn(0x55719a3ad680 msgr2=0x55719a3b8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 shutdown_connections 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d67d82740 1 -- 172.21.15.73:0/3019950665 wait complete. 2024-06-06T07:15:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d67d82740 20 client.17028 trim_cache size 0 max 16384 2024-06-06T07:15:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:15:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:15:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 msgr2=0x7f7d4c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 0x7f7d4c0a5260 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ed030 tx=0x7f7d50009260 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 msgr2=0x7f7d601098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.880+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7d5002a9e0 tx=0x7f7d50003c10 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 shutdown_connections 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106940 0x7f7d6010a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2da0 0x7f7d4c0a5260 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d601014b0 0x7f7d60109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 --2- 172.21.15.73:0/3989328463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60100b10 0x7f7d601098f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 >> 172.21.15.73:0/3989328463 conn(0x7f7d60077780 msgr2=0x7f7d600fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 shutdown_connections 2024-06-06T07:15:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:40.881+0000 7f7d6507b640 1 -- 172.21.15.73:0/3989328463 wait complete. 2024-06-06T07:15:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:41 smithi073 ceph-mon[32303]: pgmap v452: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:41 smithi136 ceph-mon[35901]: pgmap v452: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:41 smithi170 ceph-mon[36108]: pgmap v452: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:41.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7589 12.84 MB/sec warmup 46 sec latency 173.384 ms 2024-06-06T07:15:42.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7748 12.70 MB/sec warmup 47 sec latency 74.001 ms 2024-06-06T07:15:43.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 7934 12.63 MB/sec warmup 48 sec latency 63.665 ms 2024-06-06T07:15:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:43 smithi170 ceph-mon[36108]: pgmap v453: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:44.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:43 smithi073 ceph-mon[32303]: pgmap v453: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:44.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:43 smithi136 ceph-mon[35901]: pgmap v453: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 238 op/s 2024-06-06T07:15:44.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8113 12.52 MB/sec warmup 49 sec latency 78.910 ms 2024-06-06T07:15:45.799 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8252 12.33 MB/sec warmup 50 sec latency 58.650 ms 2024-06-06T07:15:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:45 smithi170 ceph-mon[36108]: pgmap v454: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-06-06T07:15:46.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:45 smithi073 ceph-mon[32303]: pgmap v454: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-06-06T07:15:46.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:45 smithi136 ceph-mon[35901]: pgmap v454: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 383 op/s 2024-06-06T07:15:46.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8406 12.25 MB/sec warmup 51 sec latency 77.732 ms 2024-06-06T07:15:47.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8574 12.17 MB/sec warmup 52 sec latency 220.849 ms 2024-06-06T07:15:48.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:47 smithi073 ceph-mon[32303]: pgmap v455: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2024-06-06T07:15:48.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:47 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:15:48.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:47 smithi136 ceph-mon[35901]: pgmap v455: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2024-06-06T07:15:48.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:47 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:15:48.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:47 smithi170 ceph-mon[36108]: pgmap v455: 129 pgs: 129 active+clean; 395 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 269 op/s 2024-06-06T07:15:48.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:47 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:15:48.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8763 12.18 MB/sec warmup 53 sec latency 155.494 ms 2024-06-06T07:15:49.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:48 smithi073 ceph-mon[32303]: pgmap v456: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 393 op/s 2024-06-06T07:15:49.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:48 smithi136 ceph-mon[35901]: pgmap v456: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 393 op/s 2024-06-06T07:15:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:48 smithi170 ceph-mon[36108]: pgmap v456: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 393 op/s 2024-06-06T07:15:49.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 8965 12.19 MB/sec warmup 54 sec latency 77.219 ms 2024-06-06T07:15:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:49 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:15:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:49 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:15:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:49 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:15:50.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:49 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:15:50.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:49 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:15:50.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:49 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:15:50.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:49 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:15:50.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:49 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:15:50.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:49 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:15:50.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9101 11.99 MB/sec warmup 55 sec latency 63.126 ms 2024-06-06T07:15:51.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:50 smithi073 ceph-mon[32303]: pgmap v457: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:51.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:50 smithi136 ceph-mon[35901]: pgmap v457: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:51.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:50 smithi170 ceph-mon[36108]: pgmap v457: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:51.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9298 12.12 MB/sec warmup 56 sec latency 71.242 ms 2024-06-06T07:15:52.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9468 11.99 MB/sec warmup 57 sec latency 76.333 ms 2024-06-06T07:15:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:52 smithi073 ceph-mon[32303]: pgmap v458: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:52 smithi136 ceph-mon[35901]: pgmap v458: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:52 smithi170 ceph-mon[36108]: pgmap v458: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 269 op/s 2024-06-06T07:15:53.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9659 11.92 MB/sec warmup 58 sec latency 95.229 ms 2024-06-06T07:15:54.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 -- 172.21.15.73:0/3437569009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e41008d0 msgr2=0x7fe7e4100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 --2- 172.21.15.73:0/3437569009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e41008d0 0x7fe7e4100d50 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8009930 tx=0x7fe7d802a420 comp rx=0 tx=0).stop 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 -- 172.21.15.73:0/3437569009 shutdown_connections 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 --2- 172.21.15.73:0/3437569009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4101290 0x7fe7e410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 --2- 172.21.15.73:0/3437569009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e41008d0 0x7fe7e4100d50 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 --2- 172.21.15.73:0/3437569009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e4106890 0x7fe7e4106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 -- 172.21.15.73:0/3437569009 >> 172.21.15.73:0/3437569009 conn(0x7fe7e4077a20 msgr2=0x7fe7e4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 -- 172.21.15.73:0/3437569009 shutdown_connections 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.005+0000 7fe7e962c640 1 -- 172.21.15.73:0/3437569009 wait complete. 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 Processor -- start 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 -- start start 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e4101290 0x7fe7e4198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 0x7fe7e419ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4110b50 con 0x7fe7e4101290 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41109d0 con 0x7fe7e4106890 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e962c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e4110cd0 con 0x7fe7e41008d0 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34686/0 (socket says 172.21.15.73:34686) 2024-06-06T07:15:54.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e2ffd640 1 -- 172.21.15.73:0/375124615 learned_addr learned my addr 172.21.15.73:0/375124615 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.006+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e4101290 0x7fe7e4198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 0x7fe7e419ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 -- 172.21.15.73:0/375124615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 msgr2=0x7fe7e419ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 0x7fe7e419ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 -- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e4101290 msgr2=0x7fe7e4198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e4101290 0x7fe7e4198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 -- 172.21.15.73:0/375124615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7e419d190 con 0x7fe7e41008d0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 0x7fe7e419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e9f0 tx=0x7fe7d400edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7cbfff640 1 -- 172.21.15.73:0/375124615 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d400cd80 con 0x7fe7e41008d0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7e419d480 con 0x7fe7e41008d0 2024-06-06T07:15:54.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7cbfff640 1 -- 172.21.15.73:0/375124615 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d40054d0 con 0x7fe7e41008d0 2024-06-06T07:15:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7e41a4d10 con 0x7fe7e41008d0 2024-06-06T07:15:54.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.007+0000 7fe7cbfff640 1 -- 172.21.15.73:0/375124615 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d4010490 con 0x7fe7e41008d0 2024-06-06T07:15:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.010+0000 7fe7cbfff640 1 -- 172.21.15.73:0/375124615 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe7d4005020 con 0x7fe7e41008d0 2024-06-06T07:15:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 Processor -- start 2024-06-06T07:15:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 -- start start 2024-06-06T07:15:54.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465aa90 0x564254664e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564254665350 0x564254667740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5642545be4f0 con 0x56425465a170 2024-06-06T07:15:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56425455c870 con 0x564254665350 2024-06-06T07:15:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5642545ba4b0 con 0x56425465aa90 2024-06-06T07:15:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37860/0 (socket says 172.21.15.73:37860) 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 -- 172.21.15.73:0/137626468 learned_addr learned my addr 172.21.15.73:0/137626468 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465aa90 0x564254664e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564254665350 0x564254667740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 -- 172.21.15.73:0/137626468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465aa90 msgr2=0x564254664e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465aa90 0x564254664e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 -- 172.21.15.73:0/137626468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564254665350 msgr2=0x564254667740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564254665350 0x564254667740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7e37fe640 1 -- 172.21.15.73:0/137626468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642545789b0 con 0x56425465a170 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.011+0000 7fe7cbfff640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 0x7fe7cc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7cbfff640 1 -- 172.21.15.73:0/375124615 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fe7d40cfc50 con 0x7fe7e41008d0 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 0x7fe7cc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009fb0 tx=0x7fe7dc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a1a044ffbe6b78 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7c97fa640 1 -- 172.21.15.73:0/137626468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc016070 con 0x56425465a170 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7c97fa640 1 -- 172.21.15.73:0/137626468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc012bc0 con 0x56425465a170 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.012+0000 7fe7c97fa640 1 -- 172.21.15.73:0/137626468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7dc01bcb0 con 0x56425465a170 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 -- 172.21.15.73:0/137626468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 msgr2=0x56425465a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fe7dc009fb0 tx=0x7fe7dc010800 comp rx=0 tx=0).stop 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7e2ffd640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 0x7fe7cc0a52e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56425465b710 tx=0x7fe7d40d3680 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 -- 172.21.15.73:0/137626468 shutdown_connections 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564254665350 0x564254667740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465aa90 0x564254664e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/137626468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x56425465a550 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 -- 172.21.15.73:0/137626468 >> 172.21.15.73:0/137626468 conn(0x564254656680 msgr2=0x564254656a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 -- 172.21.15.73:0/137626468 shutdown_connections 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.013+0000 7fe7ec40e740 1 -- 172.21.15.73:0/137626468 wait complete. 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 -- start start 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 Processor -- start 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x564254662430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x564254662eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564254667f80 con 0x56425465a170 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564254547ae0 con 0x56425465aa90 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564254668100 con 0x564254665350 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7ec40e740 10 client.0 _ll_register_callbacks cb 0x5642545809b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:35422/0 (socket says 172.21.15.73:35422) 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7e27fc640 1 -- 172.21.15.73:0/396197389 learned_addr learned my addr 172.21.15.73:0/396197389 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.014+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x564254662430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x564254662eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 -- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 msgr2=0x564254662eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x564254662eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 -- 172.21.15.73:0/396197389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 msgr2=0x564254662430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x564254662430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 -- 172.21.15.73:0/396197389 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642542d2210 con 0x56425465aa90 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x564254662eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:15:54.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7d802a940 tx=0x7fe7d8009670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbdb7738db771a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8035040 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fe7d80317f0 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7dc012bc0 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe7d8004040 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7ec40e740 10 client.26279 fetch_fsmap learned FSMap version 57 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.015+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x56425439e620 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7dc012bc0 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fe7d80035f0 con 0x56425465aa90 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 10 client.26279 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 10 client.26279 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:54.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 20 client.26279 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546756f0 0x564254677b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 4 client.26279 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642545d1f10 con 0x5642546756f0 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.016+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546756f0 0x564254677b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.017+0000 7fe7c6ff5640 10 client.26279 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:54.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.017+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546756f0 0x564254677b20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8647fc862709e604 server_cookie=c8c5bfbaba4939d7 in_seq=0 out_seq=0 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/396197389 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5642545d1f10 con 0x5642546756f0 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7c6ff5640 10 client.26279 handle_command_reply: tid=0 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 client.26279 shutdown 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 msgr2=0x564254662970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe7d802a940 tx=0x7fe7d8009670 comp rx=0 tx=0).stop 2024-06-06T07:15:54.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 shutdown_connections 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546756f0 0x564254677b20 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x564254662eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x564254662970 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7c6ff5640 0 client.26279 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/396197389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465a170 0x564254662430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 >> 172.21.15.73:0/396197389 conn(0x564254656680 msgr2=0x56425465c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 shutdown_connections 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 1 -- 172.21.15.73:0/396197389 wait complete. 2024-06-06T07:15:54.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.019+0000 7fe7ec40e740 20 client.26279 trim_cache size 0 max 16384 2024-06-06T07:15:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.045+0000 7fe7ec40e740 1 Processor -- start 2024-06-06T07:15:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.045+0000 7fe7ec40e740 1 -- start start 2024-06-06T07:15:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.045+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465a170 0x5642546615d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564254665350 0x564254655f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41acca0 con 0x564254665350 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ae9f0 con 0x56425465aa90 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ad430 con 0x56425465a170 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564254665350 0x564254655f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:35426/0 (socket says 172.21.15.73:35426) 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 -- 172.21.15.73:0/26995148 learned_addr learned my addr 172.21.15.73:0/26995148 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465a170 0x5642546615d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 -- 172.21.15.73:0/26995148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465a170 msgr2=0x5642546615d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465a170 0x5642546615d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 -- 172.21.15.73:0/26995148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564254665350 msgr2=0x564254655f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564254665350 0x564254655f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.046+0000 7fe7e27fc640 1 -- 172.21.15.73:0/26995148 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564254651f70 con 0x56425465aa90 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.047+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8003a90 tx=0x7fe7d8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a2a57201ad3e1ec server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.047+0000 7fe7c8ff9640 1 -- 172.21.15.73:0/26995148 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8035030 con 0x56425465aa90 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.047+0000 7fe7c8ff9640 1 -- 172.21.15.73:0/26995148 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d80054f0 con 0x56425465aa90 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.047+0000 7fe7c8ff9640 1 -- 172.21.15.73:0/26995148 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d80041e0 con 0x56425465aa90 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- 172.21.15.73:0/26995148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 msgr2=0x56425465cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8003a90 tx=0x7fe7d8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- 172.21.15.73:0/26995148 shutdown_connections 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564254665350 0x564254655f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465aa90 0x56425465cc90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/26995148 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56425465a170 0x5642546615d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- 172.21.15.73:0/26995148 >> 172.21.15.73:0/26995148 conn(0x564254656680 msgr2=0x564254663c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- 172.21.15.73:0/26995148 shutdown_connections 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- 172.21.15.73:0/26995148 wait complete. 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 -- start start 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 Processor -- start 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465a170 0x56425466d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.048+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x56425465f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41ae320 con 0x56425465aa90 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41adb90 con 0x56425465a170 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7e41aaca0 con 0x564254665350 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 10 client.0 _ll_register_callbacks cb 0x7fe7d8031c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37882/0 (socket says 172.21.15.73:37882) 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465a170 0x56425466d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 -- 172.21.15.73:0/3654576319 learned_addr learned my addr 172.21.15.73:0/3654576319 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x56425465f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 -- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 msgr2=0x56425465f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x56425465f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 -- 172.21.15.73:0/3654576319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465a170 msgr2=0x56425466d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:15:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465a170 0x56425466d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 -- 172.21.15.73:0/3654576319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642546711e0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7e27fc640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8004010 tx=0x7fe7d8031b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1250425edd2a20e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7d8035070 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7d80054f0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.049+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(121..121 src has 1..121) ==== 9554+0+0 (secure 0 0 0) 0x7fe7d805a040 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe7d80043f0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 10 client.17046 fetch_fsmap learned FSMap version 57 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5642546dfeb0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7d80054f0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fe7d80464f0 con 0x56425465aa90 2024-06-06T07:15:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 10 client.17046 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 10 client.17046 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 20 client.17046 populate_metadata read hostname 'smithi073' 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546e04b0 0x5642546e28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 4 client.17046 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5642545cbc20 con 0x5642546e04b0 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.050+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546e04b0 0x5642546e28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.051+0000 7fe7c6ff5640 10 client.17046 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.051+0000 7fe7e37fe640 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546e04b0 0x5642546e28d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=134b78b13293cb9b server_cookie=c60d1cf3f59e962c in_seq=0 out_seq=0 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7c6ff5640 1 -- 172.21.15.73:0/3654576319 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5642545cbc20 con 0x5642546e04b0 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7c6ff5640 10 client.17046 handle_command_reply: tid=0 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7ec40e740 1 client.17046 shutdown 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 msgr2=0x56425465ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe7d8004010 tx=0x7fe7d8031b30 comp rx=0 tx=0).stop 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 shutdown_connections 2024-06-06T07:15:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.108+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5642546e04b0 0x5642546e28d0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564254665350 0x56425465f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56425465aa90 0x56425465ee10 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 --2- 172.21.15.73:0/3654576319 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56425465a170 0x56425466d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 >> 172.21.15.73:0/3654576319 conn(0x564254656680 msgr2=0x564254670a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7c6ff5640 0 client.17046 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 shutdown_connections 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 1 -- 172.21.15.73:0/3654576319 wait complete. 2024-06-06T07:15:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.109+0000 7fe7ec40e740 20 client.17046 trim_cache size 0 max 16384 2024-06-06T07:15:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:15:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:15:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 msgr2=0x7fe7cc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 0x7fe7cc0a52e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x56425465b710 tx=0x7fe7d40d3680 comp rx=0 tx=0).stop 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 msgr2=0x7fe7e41981f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe7d400e9f0 tx=0x7fe7d400edf0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 shutdown_connections 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe7e4106890 0x7fe7e419ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7e4101290 0x7fe7e4198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7cc0a2e20 0x7fe7cc0a52e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 --2- 172.21.15.73:0/375124615 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe7e41008d0 0x7fe7e41981f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:15:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 >> 172.21.15.73:0/375124615 conn(0x7fe7e4077a20 msgr2=0x7fe7e4069430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:15:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 shutdown_connections 2024-06-06T07:15:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:15:54.110+0000 7fe7e962c640 1 -- 172.21.15.73:0/375124615 wait complete. 2024-06-06T07:15:54.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9833 11.80 MB/sec warmup 59 sec latency 125.114 ms 2024-06-06T07:15:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:54 smithi073 ceph-mon[32303]: pgmap v459: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 397 op/s 2024-06-06T07:15:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:54 smithi136 ceph-mon[35901]: pgmap v459: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 397 op/s 2024-06-06T07:15:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:54 smithi170 ceph-mon[36108]: pgmap v459: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 397 op/s 2024-06-06T07:15:55.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 9994 11.73 MB/sec warmup 60 sec latency 113.109 ms 2024-06-06T07:15:56.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 10152 11.65 MB/sec warmup 61 sec latency 60.321 ms 2024-06-06T07:15:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:56 smithi073 ceph-mon[32303]: pgmap v460: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-06T07:15:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:56 smithi136 ceph-mon[35901]: pgmap v460: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-06T07:15:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:56 smithi170 ceph-mon[36108]: pgmap v460: 129 pgs: 129 active+clean; 403 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-06T07:15:57.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 10326 11.62 MB/sec warmup 62 sec latency 103.222 ms 2024-06-06T07:15:58.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 10530 11.65 MB/sec warmup 63 sec latency 221.709 ms 2024-06-06T07:15:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:15:58 smithi073 ceph-mon[32303]: pgmap v461: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-06-06T07:15:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:15:58 smithi136 ceph-mon[35901]: pgmap v461: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-06-06T07:15:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:15:58 smithi170 ceph-mon[36108]: pgmap v461: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 400 op/s 2024-06-06T07:15:59.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 10772 11.76 MB/sec warmup 64 sec latency 80.774 ms 2024-06-06T07:16:00.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 10934 11.67 MB/sec warmup 65 sec latency 108.854 ms 2024-06-06T07:16:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:00 smithi073 ceph-mon[32303]: pgmap v462: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:00 smithi136 ceph-mon[35901]: pgmap v462: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:01.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:00 smithi170 ceph-mon[36108]: pgmap v462: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:01.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11094 11.65 MB/sec warmup 66 sec latency 63.326 ms 2024-06-06T07:16:02.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11271 11.64 MB/sec warmup 67 sec latency 62.571 ms 2024-06-06T07:16:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:02 smithi073 ceph-mon[32303]: pgmap v463: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:03.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:02 smithi136 ceph-mon[35901]: pgmap v463: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:03.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:02 smithi170 ceph-mon[36108]: pgmap v463: 129 pgs: 129 active+clean; 406 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 275 op/s 2024-06-06T07:16:03.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11474 11.70 MB/sec warmup 68 sec latency 65.769 ms 2024-06-06T07:16:04.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11630 11.64 MB/sec warmup 69 sec latency 177.272 ms 2024-06-06T07:16:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:04 smithi073 ceph-mon[32303]: pgmap v464: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 411 op/s 2024-06-06T07:16:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:04 smithi073 ceph-mon[32303]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T07:16:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:04 smithi136 ceph-mon[35901]: pgmap v464: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 411 op/s 2024-06-06T07:16:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:04 smithi136 ceph-mon[35901]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T07:16:05.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:04 smithi170 ceph-mon[36108]: pgmap v464: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.3 MiB/s wr, 411 op/s 2024-06-06T07:16:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:04 smithi170 ceph-mon[36108]: osdmap e122: 12 total, 12 up, 12 in 2024-06-06T07:16:05.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11783 11.62 MB/sec warmup 70 sec latency 1177.300 ms 2024-06-06T07:16:06.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 11940 11.56 MB/sec warmup 71 sec latency 2177.343 ms 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 -- 172.21.15.73:0/662530000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 msgr2=0x7fbfd0074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 --2- 172.21.15.73:0/662530000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd0074120 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009ca0 tx=0x7fbfc002a950 comp rx=0 tx=0).stop 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 -- 172.21.15.73:0/662530000 shutdown_connections 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 --2- 172.21.15.73:0/662530000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 0x7fbfd0114d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 --2- 172.21.15.73:0/662530000 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd0112450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 --2- 172.21.15.73:0/662530000 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd0074120 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 -- 172.21.15.73:0/662530000 >> 172.21.15.73:0/662530000 conn(0x7fbfd00fd450 msgr2=0x7fbfd00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 -- 172.21.15.73:0/662530000 shutdown_connections 2024-06-06T07:16:07.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.246+0000 7fbfd4963640 1 -- 172.21.15.73:0/662530000 wait complete. 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.247+0000 7fbfd4963640 1 Processor -- start 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.247+0000 7fbfd4963640 1 -- start start 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.247+0000 7fbfd4963640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfd4963640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfd4963640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 0x7fbfd01a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfd4963640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0118280 con 0x7fbfd0112990 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfd4963640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0118100 con 0x7fbfd0073d40 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfd4963640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd0118400 con 0x7fbfd0074660 2024-06-06T07:16:07.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53060/0 (socket says 172.21.15.73:53060) 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 -- 172.21.15.73:0/1722747517 learned_addr learned my addr 172.21.15.73:0/1722747517 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 0x7fbfd01a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfce59c640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 -- 172.21.15.73:0/1722747517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 msgr2=0x7fbfd019e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd019e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 -- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 msgr2=0x7fbfd01a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 0x7fbfd01a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.248+0000 7fbfc5d9b640 1 -- 172.21.15.73:0/1722747517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd01a3750 con 0x7fbfd0074660 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e9f0 tx=0x7fbfb800edf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfc7fff640 1 -- 172.21.15.73:0/1722747517 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800cd80 con 0x7fbfd0074660 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfc7fff640 1 -- 172.21.15.73:0/1722747517 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfb80054d0 con 0x7fbfd0074660 2024-06-06T07:16:07.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfc7fff640 1 -- 172.21.15.73:0/1722747517 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8010490 con 0x7fbfd0074660 2024-06-06T07:16:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd01a3a40 con 0x7fbfd0074660 2024-06-06T07:16:07.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.249+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01ab2d0 con 0x7fbfd0074660 2024-06-06T07:16:07.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.253+0000 7fbfc7fff640 1 -- 172.21.15.73:0/1722747517 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbfb8005020 con 0x7fbfd0074660 2024-06-06T07:16:07.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.253+0000 7fbfd7676740 1 Processor -- start 2024-06-06T07:16:07.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.253+0000 7fbfd7676740 1 -- start start 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2a90 0x559cb3ddce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3ddf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559cb3d364f0 con 0x559cb3ddd350 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cb3cd4870 con 0x559cb3dd2170 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfd7676740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559cb3d324b0 con 0x559cb3dd2a90 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:44314/0 (socket says 172.21.15.73:44314) 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.254+0000 7fbfced9d640 1 -- 172.21.15.73:0/520836918 learned_addr learned my addr 172.21.15.73:0/520836918 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfce59c640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2a90 0x559cb3ddce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfc7fff640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 0x7fbfbc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 -- 172.21.15.73:0/520836918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2a90 msgr2=0x559cb3ddce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2a90 0x559cb3ddce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 -- 172.21.15.73:0/520836918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 msgr2=0x559cb3ddf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3ddf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3ddf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 -- 172.21.15.73:0/520836918 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3cf09b0 con 0x559cb3dd2170 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3ddf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfc7fff640 1 -- 172.21.15.73:0/1722747517 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fbfb80cfb90 con 0x7fbfd0074660 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 0x7fbfbc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.255+0000 7fbfced9d640 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80045e0 tx=0x7fbfc8004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fc2f59a520c73d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.256+0000 7fbfc4d99640 1 -- 172.21.15.73:0/520836918 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8022070 con 0x559cb3dd2170 2024-06-06T07:16:07.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.256+0000 7fbfc4d99640 1 -- 172.21.15.73:0/520836918 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfc8015070 con 0x559cb3dd2170 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.256+0000 7fbfc4d99640 1 -- 172.21.15.73:0/520836918 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc80106a0 con 0x559cb3dd2170 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 0x7fbfbc0a52e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bfd0 tx=0x7fbfc8020000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- 172.21.15.73:0/520836918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 msgr2=0x559cb3dd2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbfc80045e0 tx=0x7fbfc8004610 comp rx=0 tx=0).stop 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- 172.21.15.73:0/520836918 shutdown_connections 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3ddf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2a90 0x559cb3ddce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 --2- 172.21.15.73:0/520836918 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dd2550 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- 172.21.15.73:0/520836918 >> 172.21.15.73:0/520836918 conn(0x559cb3dce680 msgr2=0x559cb3dcea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- 172.21.15.73:0/520836918 shutdown_connections 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- 172.21.15.73:0/520836918 wait complete. 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 -- start start 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.257+0000 7fbfd7676740 1 Processor -- start 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 0x559cb3dda830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3ddad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x559cb3ddff80 con 0x559cb3dd2a90 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x559cb3cbfae0 con 0x559cb3dd2170 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x559cb3de0100 con 0x559cb3ddd350 2024-06-06T07:16:07.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 0x559cb3dda830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfd7676740 10 client.0 _ll_register_callbacks cb 0x559cb3cf89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:44316/0 (socket says 172.21.15.73:44316) 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 -- 172.21.15.73:0/936324786 learned_addr learned my addr 172.21.15.73:0/936324786 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfc5d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 0x559cb3dda830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55990/0 (socket says 172.21.15.73:55990) 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3ddad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 -- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 msgr2=0x559cb3ddad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3ddad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 -- 172.21.15.73:0/936324786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 msgr2=0x559cb3dda830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 0x559cb3dda830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.258+0000 7fbfce59c640 1 -- 172.21.15.73:0/936324786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb3a4a210 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfce59c640 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbfc002ae70 tx=0x7fbfc0003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5d13a781f6a1cf8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0049060 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fbfc002fa70 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc8015070 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0050050 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfd7676740 10 client.26297 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.259+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559cb3b16620 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfc8015070 con 0x559cb3dd2170 2024-06-06T07:16:07.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fbfc0030440 con 0x559cb3dd2170 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 10 client.26297 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 10 client.26297 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 20 client.26297 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3def950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 4 client.26297 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cb3d49f10 con 0x559cb3ded490 2024-06-06T07:16:07.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.260+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3def950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.261+0000 7fbfb5ffb640 10 client.26297 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.261+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3def950 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e60d8e65efcd1999 server_cookie=5c39e2fac7e372c1 in_seq=0 out_seq=0 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.262+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/936324786 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559cb3d49f10 con 0x559cb3ded490 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.262+0000 7fbfb5ffb640 10 client.26297 handle_command_reply: tid=0 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.262+0000 7fbfd7676740 1 client.26297 shutdown 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.262+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 msgr2=0x559cb3dda2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.262+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbfc002ae70 tx=0x7fbfc0003ba0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 shutdown_connections 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3def950 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3ddad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2a90 0x559cb3dda830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfb5ffb640 0 client.26297 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 --2- 172.21.15.73:0/936324786 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2170 0x559cb3dda2f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 >> 172.21.15.73:0/936324786 conn(0x559cb3dce680 msgr2=0x559cb3dd4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 shutdown_connections 2024-06-06T07:16:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 1 -- 172.21.15.73:0/936324786 wait complete. 2024-06-06T07:16:07.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.263+0000 7fbfd7676740 20 client.26297 trim_cache size 0 max 16384 2024-06-06T07:16:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:06 smithi073 ceph-mon[32303]: pgmap v466: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 338 op/s 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 Processor -- start 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 -- start start 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 0x559cb3dcdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3dd9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b56a0 con 0x559cb3ddd350 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b73f0 con 0x559cb3dd2a90 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfd7676740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b5e30 con 0x559cb3dd2170 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55992/0 (socket says 172.21.15.73:55992) 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 -- 172.21.15.73:0/1727173080 learned_addr learned my addr 172.21.15.73:0/1727173080 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3dd9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfce59c640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 0x559cb3dcdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 -- 172.21.15.73:0/1727173080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 msgr2=0x559cb3dcdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 0x559cb3dcdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 -- 172.21.15.73:0/1727173080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 msgr2=0x559cb3dd9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3dd9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfced9d640 1 -- 172.21.15.73:0/1727173080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3ddfab0 con 0x559cb3ddd350 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfce59c640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 0x559cb3dcdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.281+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3dd9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfced9d640 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bfa0 tx=0x7fbfc8027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3a6c6cc4618947 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfb7fff640 1 -- 172.21.15.73:0/1727173080 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8022040 con 0x559cb3ddd350 2024-06-06T07:16:07.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfb7fff640 1 -- 172.21.15.73:0/1727173080 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfc8015040 con 0x559cb3ddd350 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfb7fff640 1 -- 172.21.15.73:0/1727173080 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8027660 con 0x559cb3ddd350 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 -- 172.21.15.73:0/1727173080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 msgr2=0x559cb3dd4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bfa0 tx=0x7fbfc8027000 comp rx=0 tx=0).stop 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 -- 172.21.15.73:0/1727173080 shutdown_connections 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3ddd350 0x559cb3dd4280 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3dd9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 --2- 172.21.15.73:0/1727173080 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3dd2170 0x559cb3dcdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.282+0000 7fbfd7676740 1 -- 172.21.15.73:0/1727173080 >> 172.21.15.73:0/1727173080 conn(0x559cb3dce680 msgr2=0x559cb3dd3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- 172.21.15.73:0/1727173080 shutdown_connections 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- 172.21.15.73:0/1727173080 wait complete. 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- start start 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 Processor -- start 2024-06-06T07:16:07.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3de5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3dd6b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b6d20 con 0x559cb3dd2170 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b6590 con 0x559cb3dd2a90 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd01b36a0 con 0x559cb3ddd350 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfd7676740 10 client.0 _ll_register_callbacks cb 0x7fbfc0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55996/0 (socket says 172.21.15.73:55996) 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 -- 172.21.15.73:0/2814815588 learned_addr learned my addr 172.21.15.73:0/2814815588 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3de5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfced9d640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3dd6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 -- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 msgr2=0x559cb3dd6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3dd6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 -- 172.21.15.73:0/2814815588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 msgr2=0x559cb3de5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3de5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfce59c640 1 -- 172.21.15.73:0/2814815588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb3dd7250 con 0x559cb3dd2170 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.283+0000 7fbfced9d640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3dd6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfce59c640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009430 tx=0x7fbfc0030670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6087fc897dbe514 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0058020 con 0x559cb3dd2170 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(122..122 src has 1..122) ==== 9642+0+0 (secure 0 0 0) 0x7fbfc002f870 con 0x559cb3dd2170 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfc8015040 con 0x559cb3dd2170 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fbfc0049040 con 0x559cb3dd2170 2024-06-06T07:16:07.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 10 client.17061 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x559cb3e57b50 con 0x559cb3dd2170 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfc8015040 con 0x559cb3dd2170 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fbfc0004510 con 0x559cb3dd2170 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 10 client.17061 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 20 client.17061 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3e5a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 4 client.17061 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.284+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559cb3d43c20 con 0x559cb3ded490 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.285+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3e5a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.285+0000 7fbfb5ffb640 10 client.17061 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:07.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.285+0000 7fbfc5d9b640 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3e5a420 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29dabbfa076bf27f server_cookie=3a1e25fe3a28f5fb in_seq=0 out_seq=0 2024-06-06T07:16:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:06 smithi136 ceph-mon[35901]: pgmap v466: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 338 op/s 2024-06-06T07:16:07.349 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:06 smithi170 ceph-mon[36108]: pgmap v466: 129 pgs: 129 active+clean; 423 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 338 op/s 2024-06-06T07:16:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfb5ffb640 1 -- 172.21.15.73:0/2814815588 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559cb3d43c20 con 0x559cb3ded490 2024-06-06T07:16:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfb5ffb640 10 client.17061 handle_command_reply: tid=0 2024-06-06T07:16:07.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfd7676740 1 client.17061 shutdown 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 msgr2=0x559cb3de5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0009430 tx=0x7fbfc0030670 comp rx=0 tx=0).stop 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.446+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 shutdown_connections 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x559cb3ded490 0x559cb3e5a420 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x559cb3ddd350 0x559cb3dd6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfb5ffb640 0 client.17061 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x559cb3dd2a90 0x559cb3de5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 --2- 172.21.15.73:0/2814815588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cb3dd2170 0x559cb3de5540 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 >> 172.21.15.73:0/2814815588 conn(0x559cb3dce680 msgr2=0x559cb3dd4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 shutdown_connections 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 1 -- 172.21.15.73:0/2814815588 wait complete. 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.447+0000 7fbfd7676740 20 client.17061 trim_cache size 0 max 16384 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:16:07.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 msgr2=0x7fbfbc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 0x7fbfbc0a52e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800bfd0 tx=0x7fbfc8020000 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 msgr2=0x7fbfd019eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e9f0 tx=0x7fbfb800edf0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 shutdown_connections 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbfd0112990 0x7fbfd01a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbfbc0a2e20 0x7fbfbc0a52e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbfd0074660 0x7fbfd019eca0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 --2- 172.21.15.73:0/1722747517 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbfd0073d40 0x7fbfd019e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.448+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 >> 172.21.15.73:0/1722747517 conn(0x7fbfd00fd450 msgr2=0x7fbfd00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.449+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 shutdown_connections 2024-06-06T07:16:07.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:07.449+0000 7fbfd4963640 1 -- 172.21.15.73:0/1722747517 wait complete. 2024-06-06T07:16:07.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12082 11.50 MB/sec warmup 72 sec latency 3177.399 ms 2024-06-06T07:16:08.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12240 11.44 MB/sec warmup 73 sec latency 4177.436 ms 2024-06-06T07:16:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:08 smithi073 ceph-mon[32303]: pgmap v467: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:08 smithi136 ceph-mon[35901]: pgmap v467: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:09.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:08 smithi170 ceph-mon[36108]: pgmap v467: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:09.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12422 11.45 MB/sec warmup 74 sec latency 4458.380 ms 2024-06-06T07:16:10.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12572 11.37 MB/sec warmup 75 sec latency 66.363 ms 2024-06-06T07:16:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:10 smithi073 ceph-mon[32303]: pgmap v468: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:10 smithi136 ceph-mon[35901]: pgmap v468: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:11.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:10 smithi170 ceph-mon[36108]: pgmap v468: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:11.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12719 11.29 MB/sec warmup 76 sec latency 90.143 ms 2024-06-06T07:16:12.800 INFO:tasks.workunit.client.0.smithi073.stdout: 10 12887 11.30 MB/sec warmup 77 sec latency 107.417 ms 2024-06-06T07:16:13.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13085 11.26 MB/sec warmup 78 sec latency 57.976 ms 2024-06-06T07:16:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:13 smithi073 ceph-mon[32303]: pgmap v469: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:13 smithi073 ceph-mon[32303]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T07:16:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:13 smithi136 ceph-mon[35901]: pgmap v469: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:13 smithi136 ceph-mon[35901]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T07:16:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:13 smithi170 ceph-mon[36108]: pgmap v469: 129 pgs: 129 active+clean; 439 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.6 MiB/s wr, 339 op/s 2024-06-06T07:16:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:13 smithi170 ceph-mon[36108]: osdmap e123: 12 total, 12 up, 12 in 2024-06-06T07:16:14.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13242 11.20 MB/sec warmup 79 sec latency 68.236 ms 2024-06-06T07:16:15.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13394 11.11 MB/sec warmup 80 sec latency 84.639 ms 2024-06-06T07:16:16.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:15 smithi073 ceph-mon[32303]: pgmap v471: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-06-06T07:16:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:15 smithi136 ceph-mon[35901]: pgmap v471: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-06-06T07:16:16.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:15 smithi170 ceph-mon[36108]: pgmap v471: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 352 op/s 2024-06-06T07:16:16.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13594 11.15 MB/sec warmup 81 sec latency 61.870 ms 2024-06-06T07:16:17.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:16 smithi073 ceph-mon[32303]: pgmap v472: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 350 op/s 2024-06-06T07:16:17.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:16 smithi136 ceph-mon[35901]: pgmap v472: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 350 op/s 2024-06-06T07:16:17.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:16 smithi170 ceph-mon[36108]: pgmap v472: 129 pgs: 129 active+clean; 448 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.2 MiB/s wr, 350 op/s 2024-06-06T07:16:17.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13774 11.11 MB/sec warmup 82 sec latency 132.093 ms 2024-06-06T07:16:18.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 13924 11.06 MB/sec warmup 83 sec latency 62.601 ms 2024-06-06T07:16:19.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:18 smithi073 ceph-mon[32303]: pgmap v473: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:18 smithi136 ceph-mon[35901]: pgmap v473: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:18 smithi170 ceph-mon[36108]: pgmap v473: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:19.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 14096 11.04 MB/sec warmup 84 sec latency 108.917 ms 2024-06-06T07:16:20.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.578+0000 7fe5b535a640 1 -- 172.21.15.73:0/232175562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b010c5d0 msgr2=0x7fe5b010c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.578+0000 7fe5b535a640 1 --2- 172.21.15.73:0/232175562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b010c5d0 0x7fe5b010c9b0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe5a0009860 tx=0x7fe5a002a340 comp rx=0 tx=0).stop 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 -- 172.21.15.73:0/232175562 shutdown_connections 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 --2- 172.21.15.73:0/232175562 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b0119270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 --2- 172.21.15.73:0/232175562 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010cf80 0x7fe5b0110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 --2- 172.21.15.73:0/232175562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b010c5d0 0x7fe5b010c9b0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 -- 172.21.15.73:0/232175562 >> 172.21.15.73:0/232175562 conn(0x7fe5b006cc00 msgr2=0x7fe5b010a9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 -- 172.21.15.73:0/232175562 shutdown_connections 2024-06-06T07:16:20.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 -- 172.21.15.73:0/232175562 wait complete. 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.579+0000 7fe5b535a640 1 Processor -- start 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 -- start start 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 0x7fe5b006d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 0x7fe5b01b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b011c600 con 0x7fe5b006e040 2024-06-06T07:16:20.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b011c480 con 0x7fe5b01116a0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5b535a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b011c780 con 0x7fe5b010c5d0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5ae7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 0x7fe5b01b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5ae7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53146/0 (socket says 172.21.15.73:53146) 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/2638378179 learned_addr learned my addr 172.21.15.73:0/2638378179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.580+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 0x7fe5b006d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/2638378179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 msgr2=0x7fe5b006d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 0x7fe5b006d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 msgr2=0x7fe5b01b30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 0x7fe5b01b30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/2638378179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5b01b3630 con 0x7fe5b01116a0 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 0x7fe5b006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:20.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 0x7fe5b01b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.581+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400e9f0 tx=0x7fe5a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.583+0000 7fe597fff640 1 -- 172.21.15.73:0/2638378179 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a400ccc0 con 0x7fe5b01116a0 2024-06-06T07:16:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.583+0000 7fe597fff640 1 -- 172.21.15.73:0/2638378179 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a40054d0 con 0x7fe5b01116a0 2024-06-06T07:16:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.583+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5b01b38c0 con 0x7fe5b01116a0 2024-06-06T07:16:20.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.584+0000 7fe597fff640 1 -- 172.21.15.73:0/2638378179 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4010430 con 0x7fe5b01116a0 2024-06-06T07:16:20.586 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.584+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5b01b3d80 con 0x7fe5b01116a0 2024-06-06T07:16:20.587 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.587+0000 7fe597fff640 1 -- 172.21.15.73:0/2638378179 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe5a4004680 con 0x7fe5b01116a0 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.587+0000 7fe597fff640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 0x7fe5980a5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 Processor -- start 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe597fff640 1 -- 172.21.15.73:0/2638378179 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fe5a40cf690 con 0x7fe5b01116a0 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 0x7fe5980a5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 -- start start 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e67f3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67fde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.590 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5626e67574f0 con 0x5626e67fe350 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626e66f5870 con 0x5626e67f3170 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5b8067740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5626e67534b0 con 0x5626e67f3a90 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36174/0 (socket says 172.21.15.73:36174) 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5af7fe640 1 -- 172.21.15.73:0/2009583699 learned_addr learned my addr 172.21.15.73:0/2009583699 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 0x7fe5980a5210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002a930 tx=0x7fe5a00058d0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e67f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.588+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67fde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 -- 172.21.15.73:0/2009583699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 msgr2=0x5626e67fde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67fde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 -- 172.21.15.73:0/2009583699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 msgr2=0x5626e67f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e67f3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.591 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 -- 172.21.15.73:0/2009583699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e67119b0 con 0x5626e67fe350 2024-06-06T07:16:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009fc0 tx=0x7fe5a800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50259b60e962e5d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5957fa640 1 -- 172.21.15.73:0/2009583699 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a801d070 con 0x5626e67fe350 2024-06-06T07:16:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5957fa640 1 -- 172.21.15.73:0/2009583699 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a8003e20 con 0x5626e67fe350 2024-06-06T07:16:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.589+0000 7fe5957fa640 1 -- 172.21.15.73:0/2009583699 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a80046a0 con 0x5626e67fe350 2024-06-06T07:16:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.590+0000 7fe5b8067740 1 -- 172.21.15.73:0/2009583699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 msgr2=0x5626e6800740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.590+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe5a8009fc0 tx=0x7fe5a800ec30 comp rx=0 tx=0).stop 2024-06-06T07:16:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 -- 172.21.15.73:0/2009583699 shutdown_connections 2024-06-06T07:16:20.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e6800740 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67fde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2009583699 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e67f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 -- 172.21.15.73:0/2009583699 >> 172.21.15.73:0/2009583699 conn(0x5626e67ef680 msgr2=0x5626e67efa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 -- 172.21.15.73:0/2009583699 shutdown_connections 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 -- 172.21.15.73:0/2009583699 wait complete. 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 -- start start 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.591+0000 7fe5b8067740 1 Processor -- start 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 0x5626e67fb870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5626e6800f80 con 0x5626e67f3a90 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5ae7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 0x5626e67fb870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36598/0 (socket says 172.21.15.73:36598) 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 -- 172.21.15.73:0/3923774188 learned_addr learned my addr 172.21.15.73:0/3923774188 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5626e66e0ae0 con 0x5626e67fe350 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5626e6801100 con 0x5626e67f3170 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5b8067740 10 client.0 _ll_register_callbacks cb 0x5626e67199b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 -- 172.21.15.73:0/3923774188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 msgr2=0x5626e67fbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67fbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 -- 172.21.15.73:0/3923774188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 msgr2=0x5626e67fb870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 0x5626e67fb870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5aeffd640 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e646b210 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.592+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 0x5626e67fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002a860 tx=0x7fe5a0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c903845967c16d0d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a004e070 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fe5a00329f0 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a8003e20 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe5a005a050 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe5b8067740 10 client.26491 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.593+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5626e6537620 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a8003e20 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fe5a003e480 con 0x5626e67f3170 2024-06-06T07:16:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 10 client.26491 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 10 client.26491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 20 client.26491 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e680e770 0x5626e6810c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 4 client.26491 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5626e676af10 con 0x5626e680e770 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.594+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e680e770 0x5626e6810c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.595+0000 7fe592ff5640 10 client.26491 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.595+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e680e770 0x5626e6810c30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57945ce3bc6ce657 server_cookie=6aa461a5717d9ff9 in_seq=0 out_seq=0 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.595+0000 7fe592ff5640 1 -- 172.21.15.73:0/3923774188 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5626e676af10 con 0x5626e680e770 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.595+0000 7fe592ff5640 10 client.26491 handle_command_reply: tid=0 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.595+0000 7fe5b8067740 1 client.26491 shutdown 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 msgr2=0x5626e67fb330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002a860 tx=0x7fe5a0005f90 comp rx=0 tx=0).stop 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 shutdown_connections 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e680e770 0x5626e6810c30 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67fbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3a90 0x5626e67fb870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 --2- 172.21.15.73:0/3923774188 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3170 0x5626e67fb330 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 >> 172.21.15.73:0/3923774188 conn(0x5626e67ef680 msgr2=0x5626e67f53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe592ff5640 0 client.26491 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 shutdown_connections 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 1 -- 172.21.15.73:0/3923774188 wait complete. 2024-06-06T07:16:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.596+0000 7fe5b8067740 20 client.26491 trim_cache size 0 max 16384 2024-06-06T07:16:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 Processor -- start 2024-06-06T07:16:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 -- start start 2024-06-06T07:16:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67f5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01bd980 con 0x5626e67f3170 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01bf6d0 con 0x5626e67fe350 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5b8067740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01be110 con 0x5626e67f3a90 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36198/0 (socket says 172.21.15.73:36198) 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 -- 172.21.15.73:0/2743365084 learned_addr learned my addr 172.21.15.73:0/2743365084 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5af7fe640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 -- 172.21.15.73:0/2743365084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 msgr2=0x5626e67f5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67f5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 -- 172.21.15.73:0/2743365084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 msgr2=0x5626e67f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5aeffd640 1 -- 172.21.15.73:0/2743365084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626e67f98c0 con 0x5626e67f3170 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.620+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.621+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe5a003e820 tx=0x7fe5a0030fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b26b80329950bf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.621+0000 7fe594ff9640 1 -- 172.21.15.73:0/2743365084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a0063070 con 0x5626e67f3170 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.621+0000 7fe594ff9640 1 -- 172.21.15.73:0/2743365084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a0043040 con 0x5626e67f3170 2024-06-06T07:16:20.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.621+0000 7fe594ff9640 1 -- 172.21.15.73:0/2743365084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a004e040 con 0x5626e67f3170 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 -- 172.21.15.73:0/2743365084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 msgr2=0x5626e67fac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fe5a003e820 tx=0x7fe5a0030fc0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 -- 172.21.15.73:0/2743365084 shutdown_connections 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67fe350 0x5626e67f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e67f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 --2- 172.21.15.73:0/2743365084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67f3170 0x5626e67fac50 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 -- 172.21.15.73:0/2743365084 >> 172.21.15.73:0/2743365084 conn(0x5626e67ef680 msgr2=0x5626e67f4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 -- 172.21.15.73:0/2743365084 shutdown_connections 2024-06-06T07:16:20.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.622+0000 7fe5b8067740 1 -- 172.21.15.73:0/2743365084 wait complete. 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 -- start start 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 Processor -- start 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e6806350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e67f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01bf000 con 0x5626e67fe350 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01be870 con 0x5626e67f3170 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe5b01bb980 con 0x5626e67f3a90 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5b8067740 10 client.0 _ll_register_callbacks cb 0x7fe5a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e67f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5ae7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e6806350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5aeffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e6806350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53170/0 (socket says 172.21.15.73:53170) 2024-06-06T07:16:20.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5af7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e67f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36208/0 (socket says 172.21.15.73:36208) 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5aeffd640 1 -- 172.21.15.73:0/4043208880 learned_addr learned my addr 172.21.15.73:0/4043208880 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.623+0000 7fe5ae7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36606/0 (socket says 172.21.15.73:36606) 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/4043208880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 msgr2=0x5626e6806350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e6806350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/4043208880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 msgr2=0x5626e67f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e67f7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 -- 172.21.15.73:0/4043208880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626e67f7fd0 con 0x5626e67f3a90 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5ae7fc640 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40cc9d0 tx=0x7fe5a400cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa99fa00bb749a7b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe592ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5a40b8070 con 0x5626e67f3a90 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7fe5a40cf200 con 0x5626e67f3a90 2024-06-06T07:16:20.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5a0043040 con 0x5626e67f3a90 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7fe5a40a4410 con 0x5626e67f3a90 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5b8067740 10 client.26497 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.624+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x5626e6878de0 con 0x5626e67f3a90 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5a0043040 con 0x5626e67f3a90 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7fe5a40974f0 con 0x5626e67f3a90 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 10 client.26497 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 10 client.26497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 20 client.26497 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e6879620 0x5626e687ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 4 client.26497 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5626e6764c20 con 0x5626e6879620 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.625+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e6879620 0x5626e687ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:20.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.626+0000 7fe592ff5640 10 client.26497 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:20.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.626+0000 7fe5aeffd640 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e6879620 0x5626e687ba40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1028c35607bdcb16 server_cookie=769ff7a825fa9222 in_seq=0 out_seq=0 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.781+0000 7fe592ff5640 1 -- 172.21.15.73:0/4043208880 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5626e6764c20 con 0x5626e6879620 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.781+0000 7fe592ff5640 10 client.26497 handle_command_reply: tid=0 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.781+0000 7fe5b8067740 1 client.26497 shutdown 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.781+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 msgr2=0x5626e6806890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.781+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40cc9d0 tx=0x7fe5a400cf10 comp rx=0 tx=0).stop 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 shutdown_connections 2024-06-06T07:16:20.783 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5626e6879620 0x5626e687ba40 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5626e67fe350 0x5626e67f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5626e67f3a90 0x5626e6806890 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe592ff5640 0 client.26497 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 --2- 172.21.15.73:0/4043208880 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5626e67f3170 0x5626e6806350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 >> 172.21.15.73:0/4043208880 conn(0x5626e67ef680 msgr2=0x5626e67f53b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 shutdown_connections 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 1 -- 172.21.15.73:0/4043208880 wait complete. 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b8067740 20 client.26497 trim_cache size 0 max 16384 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:16:20.784 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:16:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:16:20.785 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 msgr2=0x7fe5980a5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 0x7fe5980a5210 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe5a002a930 tx=0x7fe5a00058d0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 msgr2=0x7fe5b006db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.782+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400e9f0 tx=0x7fe5a400eeb0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 shutdown_connections 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe5b006e040 0x7fe5b01b30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe5b01116a0 0x7fe5b006db00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe5980a2d50 0x7fe5980a5210 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 --2- 172.21.15.73:0/2638378179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe5b010c5d0 0x7fe5b006d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 >> 172.21.15.73:0/2638378179 conn(0x7fe5b006cc00 msgr2=0x7fe5b010f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:20.786 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 shutdown_connections 2024-06-06T07:16:20.787 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:20.783+0000 7fe5b535a640 1 -- 172.21.15.73:0/2638378179 wait complete. 2024-06-06T07:16:20.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 14283 11.04 MB/sec warmup 85 sec latency 70.843 ms 2024-06-06T07:16:21.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:20 smithi073 ceph-mon[32303]: pgmap v474: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:21.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:20 smithi136 ceph-mon[35901]: pgmap v474: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:21.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:20 smithi170 ceph-mon[36108]: pgmap v474: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:21.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 14477 11.00 MB/sec warmup 86 sec latency 57.547 ms 2024-06-06T07:16:22.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 14677 11.02 MB/sec warmup 87 sec latency 85.508 ms 2024-06-06T07:16:23.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:22 smithi073 ceph-mon[32303]: pgmap v475: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:22 smithi136 ceph-mon[35901]: pgmap v475: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:23.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:22 smithi170 ceph-mon[36108]: pgmap v475: 129 pgs: 129 active+clean; 454 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 347 op/s 2024-06-06T07:16:23.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 14811 10.96 MB/sec warmup 88 sec latency 124.128 ms 2024-06-06T07:16:24.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15024 11.06 MB/sec warmup 89 sec latency 61.485 ms 2024-06-06T07:16:25.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:24 smithi073 ceph-mon[32303]: pgmap v476: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 471 op/s 2024-06-06T07:16:25.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:24 smithi136 ceph-mon[35901]: pgmap v476: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 471 op/s 2024-06-06T07:16:25.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:24 smithi170 ceph-mon[36108]: pgmap v476: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.3 MiB/s wr, 471 op/s 2024-06-06T07:16:25.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15213 11.06 MB/sec warmup 90 sec latency 62.914 ms 2024-06-06T07:16:26.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15380 11.04 MB/sec warmup 91 sec latency 78.106 ms 2024-06-06T07:16:27.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:26 smithi073 ceph-mon[32303]: pgmap v477: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-06-06T07:16:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:26 smithi136 ceph-mon[35901]: pgmap v477: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-06-06T07:16:27.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:26 smithi170 ceph-mon[36108]: pgmap v477: 129 pgs: 129 active+clean; 463 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-06-06T07:16:27.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15532 11.01 MB/sec warmup 92 sec latency 57.541 ms 2024-06-06T07:16:28.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15705 10.99 MB/sec warmup 93 sec latency 63.796 ms 2024-06-06T07:16:29.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:28 smithi073 ceph-mon[32303]: pgmap v478: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 433 op/s 2024-06-06T07:16:29.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:28 smithi136 ceph-mon[35901]: pgmap v478: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 433 op/s 2024-06-06T07:16:29.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:28 smithi170 ceph-mon[36108]: pgmap v478: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 433 op/s 2024-06-06T07:16:29.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 15877 10.98 MB/sec warmup 94 sec latency 66.426 ms 2024-06-06T07:16:30.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16034 10.93 MB/sec warmup 95 sec latency 70.654 ms 2024-06-06T07:16:31.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:30 smithi073 ceph-mon[32303]: pgmap v479: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:31.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:30 smithi136 ceph-mon[35901]: pgmap v479: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:30 smithi170 ceph-mon[36108]: pgmap v479: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:31.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16192 10.91 MB/sec warmup 96 sec latency 70.808 ms 2024-06-06T07:16:32.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16369 10.90 MB/sec warmup 97 sec latency 68.949 ms 2024-06-06T07:16:33.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:33 smithi073 ceph-mon[32303]: pgmap v480: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:33.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:33 smithi136 ceph-mon[35901]: pgmap v480: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:33 smithi170 ceph-mon[36108]: pgmap v480: 129 pgs: 129 active+clean; 476 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.5 MiB/s wr, 288 op/s 2024-06-06T07:16:33.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16558 10.91 MB/sec warmup 98 sec latency 60.683 ms 2024-06-06T07:16:33.911 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 -- 172.21.15.73:0/3192654561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e006a440 msgr2=0x7f09e006a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3192654561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e006a440 0x7f09e006a820 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009900 tx=0x7f09dc02a410 comp rx=0 tx=0).stop 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 -- 172.21.15.73:0/3192654561 shutdown_connections 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3192654561 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e01116c0 0x7f09e0119290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3192654561 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e006adf0 0x7f09e0110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3192654561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e006a440 0x7f09e006a820 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 -- 172.21.15.73:0/3192654561 >> 172.21.15.73:0/3192654561 conn(0x7f09e00731a0 msgr2=0x7f09e006f410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 -- 172.21.15.73:0/3192654561 shutdown_connections 2024-06-06T07:16:33.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.911+0000 7f09e7f22640 1 -- 172.21.15.73:0/3192654561 wait complete. 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 Processor -- start 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 -- start start 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e006a440 0x7f09e0119170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e0116750 0x7f09e0116be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09e011c630 con 0x7f09e0116750 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f09e011c4b0 con 0x7f09e01116c0 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e7f22640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09e011c7b0 con 0x7f09e006a440 2024-06-06T07:16:33.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e54bd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e54bd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:51314/0 (socket says 172.21.15.73:51314) 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e0116750 0x7f09e0116be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e54bd640 1 -- 172.21.15.73:0/3078599178 learned_addr learned my addr 172.21.15.73:0/3078599178 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.912+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e006a440 0x7f09e0119170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 -- 172.21.15.73:0/3078599178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e006a440 msgr2=0x7f09e0119170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e006a440 0x7f09e0119170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 -- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e0116750 msgr2=0x7f09e0116be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e0116750 0x7f09e0116be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 -- 172.21.15.73:0/3078599178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e01b35b0 con 0x7f09e01116c0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e54bd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9c0 tx=0x7f09d000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09d6ffd640 1 -- 172.21.15.73:0/3078599178 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d000cd80 con 0x7f09e01116c0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e01b38a0 con 0x7f09e01116c0 2024-06-06T07:16:33.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09d6ffd640 1 -- 172.21.15.73:0/3078599178 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d00054d0 con 0x7f09e01116c0 2024-06-06T07:16:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09d6ffd640 1 -- 172.21.15.73:0/3078599178 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0010430 con 0x7f09e01116c0 2024-06-06T07:16:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.913+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01b3e40 con 0x7f09e01116c0 2024-06-06T07:16:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.916+0000 7f09d6ffd640 1 -- 172.21.15.73:0/3078599178 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f09d0004fd0 con 0x7f09e01116c0 2024-06-06T07:16:33.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.916+0000 7f09eac32740 1 Processor -- start 2024-06-06T07:16:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.916+0000 7f09eac32740 1 -- start start 2024-06-06T07:16:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e5cbe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:51330/0 (socket says 172.21.15.73:51330) 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 -- 172.21.15.73:0/3878450999 learned_addr learned my addr 172.21.15.73:0/3878450999 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09d6ffd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 0x7f09cc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5ea740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55883d5414f0 con 0x55883d5dda90 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55883d4df870 con 0x55883d5dd170 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55883d53d4b0 con 0x55883d5e8350 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e54bd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 0x7f09cc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09d6ffd640 1 -- 172.21.15.73:0/3078599178 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f09d00cfc20 con 0x7f09e01116c0 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5ea740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 -- 172.21.15.73:0/3878450999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 msgr2=0x55883d5ea740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5ea740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 -- 172.21.15.73:0/3878450999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 msgr2=0x55883d5e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.917+0000 7f09e64bf640 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09e64bf640 1 -- 172.21.15.73:0/3878450999 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883d4fb9b0 con 0x55883d5dd170 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09e54bd640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 0x7f09cc0a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09d00cd010 tx=0x7f09d0098050 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09e64bf640 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09d80049d0 tx=0x7f09d8009c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8814c854f887beb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09cbfff640 1 -- 172.21.15.73:0/3878450999 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d801a030 con 0x55883d5dd170 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09cbfff640 1 -- 172.21.15.73:0/3878450999 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d8011110 con 0x55883d5dd170 2024-06-06T07:16:33.920 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.918+0000 7f09cbfff640 1 -- 172.21.15.73:0/3878450999 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8005280 con 0x55883d5dd170 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 msgr2=0x55883d5dd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09d80049d0 tx=0x7f09d8009c80 comp rx=0 tx=0).stop 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 shutdown_connections 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5ea740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 --2- 172.21.15.73:0/3878450999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5dd550 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 >> 172.21.15.73:0/3878450999 conn(0x55883d5d9680 msgr2=0x55883d5d9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 shutdown_connections 2024-06-06T07:16:33.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- 172.21.15.73:0/3878450999 wait complete. 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 -- start start 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.919+0000 7f09eac32740 1 Processor -- start 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46576/0 (socket says 172.21.15.73:46576) 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 -- 172.21.15.73:0/1395391129 learned_addr learned my addr 172.21.15.73:0/1395391129 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e64bf640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5e5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55883d5eaf80 con 0x55883d5dda90 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55883d4caae0 con 0x55883d5dd170 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55883d5eb100 con 0x55883d5e8350 2024-06-06T07:16:33.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09eac32740 10 client.0 _ll_register_callbacks cb 0x55883d5039b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e64bf640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5e5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 -- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 msgr2=0x55883d5e5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5e5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 -- 172.21.15.73:0/1395391129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 msgr2=0x55883d5e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.920+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09e5cbe640 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883d255210 con 0x55883d5dda90 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a930 tx=0x7f09dc032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c90963046a654b81 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc035040 con 0x55883d5dda90 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f09dc0316a0 con 0x55883d5dda90 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d8011110 con 0x55883d5dda90 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f09dc0318a0 con 0x55883d5dda90 2024-06-06T07:16:33.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09eac32740 10 client.17091 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.921+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55883d321620 con 0x55883d5dda90 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09d8011110 con 0x55883d5dda90 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f09dc004750 con 0x55883d5dda90 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 10 client.17091 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 10 client.17091 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 20 client.17091 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d5f84a0 0x55883d5fa920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 4 client.17091 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55883d554f10 con 0x55883d5f84a0 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.922+0000 7f09e64bf640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d5f84a0 0x55883d5fa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.923+0000 7f09c97fa640 10 client.17091 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.923+0000 7f09e64bf640 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d5f84a0 0x55883d5fa920 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40b6b786b4bc295b server_cookie=2a3510b5dae4a534 in_seq=0 out_seq=0 2024-06-06T07:16:33.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09c97fa640 1 -- 172.21.15.73:0/1395391129 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55883d554f10 con 0x55883d5f84a0 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09c97fa640 10 client.17091 handle_command_reply: tid=0 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 client.17091 shutdown 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 msgr2=0x55883d5e5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f09dc02a930 tx=0x7f09dc032d30 comp rx=0 tx=0).stop 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 shutdown_connections 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d5f84a0 0x55883d5fa920 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5e8350 0x55883d5e5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d5e5810 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09c97fa640 0 client.17091 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 --2- 172.21.15.73:0/1395391129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5dd170 0x55883d5e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 >> 172.21.15.73:0/1395391129 conn(0x55883d5d9680 msgr2=0x55883d5df400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 shutdown_connections 2024-06-06T07:16:33.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.924+0000 7f09eac32740 1 -- 172.21.15.73:0/1395391129 wait complete. 2024-06-06T07:16:33.930 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.925+0000 7f09eac32740 20 client.17091 trim_cache size 0 max 16384 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.956+0000 7f09eac32740 1 Processor -- start 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.956+0000 7f09eac32740 1 -- start start 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.956+0000 7f09eac32740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 0x55883d5dfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.956+0000 7f09eac32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d600230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09eac32740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d600770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d600230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 0x55883d5dfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 0x55883d5dfb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:58292/0 (socket says 172.21.15.73:58292) 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e64bf640 1 -- 172.21.15.73:0/2528462691 learned_addr learned my addr 172.21.15.73:0/2528462691 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:33.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01bddd0 con 0x55883d5dda90 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01bfb20 con 0x55883d5e8350 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01be560 con 0x55883d5dd170 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e64bf640 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d600770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 -- 172.21.15.73:0/2528462691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 msgr2=0x55883d5dfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 0x55883d5dfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 -- 172.21.15.73:0/2528462691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 msgr2=0x55883d600770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d600770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.957+0000 7f09e5cbe640 1 -- 172.21.15.73:0/2528462691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55883d5e3820 con 0x55883d5dda90 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.958+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d600230 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009900 tx=0x7f09dc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e9481e3559e03a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.958+0000 7f09cb7fe640 1 -- 172.21.15.73:0/2528462691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc056020 con 0x55883d5dda90 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.958+0000 7f09cb7fe640 1 -- 172.21.15.73:0/2528462691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09dc005850 con 0x55883d5dda90 2024-06-06T07:16:33.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.958+0000 7f09cb7fe640 1 -- 172.21.15.73:0/2528462691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc035070 con 0x55883d5dda90 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 msgr2=0x55883d600230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d600230 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f09dc009900 tx=0x7f09dc009780 comp rx=0 tx=0).stop 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 shutdown_connections 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d600770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dda90 0x55883d600230 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 --2- 172.21.15.73:0/2528462691 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dd170 0x55883d5dfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 >> 172.21.15.73:0/2528462691 conn(0x55883d5d9680 msgr2=0x55883d5e7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 shutdown_connections 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- 172.21.15.73:0/2528462691 wait complete. 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.960+0000 7f09eac32740 1 -- start start 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.961+0000 7f09eac32740 1 Processor -- start 2024-06-06T07:16:33.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.961+0000 7f09eac32740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.961+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.961+0000 7f09e64bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46588/0 (socket says 172.21.15.73:46588) 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.961+0000 7f09e64bf640 1 -- 172.21.15.73:0/2055347816 learned_addr learned my addr 172.21.15.73:0/2055347816 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dda90 0x55883d5e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d5e25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01bf450 con 0x55883d5dd170 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01becc0 con 0x55883d5e8350 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01bbdd0 con 0x55883d5dda90 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09eac32740 10 client.0 _ll_register_callbacks cb 0x7f09d80093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dda90 0x55883d5e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d5e25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 -- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dda90 msgr2=0x55883d5e2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dda90 0x55883d5e2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 -- 172.21.15.73:0/2055347816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 msgr2=0x55883d5e25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d5e25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:33.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55883d5e2e00 con 0x55883d5dd170 2024-06-06T07:16:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09e64bf640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000f80 tx=0x7f09d8022040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d88d7be9000811 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.962+0000 7f09c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:16:33.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8004a00 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f09d8004ba0 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09dc005850 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f09d8004da0 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09eac32740 10 client.17097 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.963+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55883d663270 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09dc005850 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f09d8005040 con 0x55883d5dd170 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 10 client.17097 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 20 client.17097 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d663930 0x55883d665d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:33.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 4 client.17097 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:16:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55883d54ec20 con 0x55883d663930 2024-06-06T07:16:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d663930 0x55883d665d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09e5cbe640 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d663930 0x55883d665d50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475a1310b7cc440e server_cookie=e0eb7c7e04ca2f45 in_seq=0 out_seq=0 2024-06-06T07:16:33.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:33.964+0000 7f09c97fa640 10 client.17097 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09c97fa640 1 -- 172.21.15.73:0/2055347816 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55883d54ec20 con 0x55883d663930 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09c97fa640 10 client.17097 handle_command_reply: tid=0 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 client.17097 shutdown 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 msgr2=0x55883d5f0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f09d8000f80 tx=0x7f09d8022040 comp rx=0 tx=0).stop 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 shutdown_connections 2024-06-06T07:16:34.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55883d663930 0x55883d665d50 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55883d5e8350 0x55883d5e25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55883d5dda90 0x55883d5e2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 --2- 172.21.15.73:0/2055347816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55883d5dd170 0x55883d5f0af0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 >> 172.21.15.73:0/2055347816 conn(0x55883d5d9680 msgr2=0x55883d5e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09c97fa640 0 client.17097 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 shutdown_connections 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 1 -- 172.21.15.73:0/2055347816 wait complete. 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.103+0000 7f09eac32740 20 client.17097 trim_cache size 0 max 16384 2024-06-06T07:16:34.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:16:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:16:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:16:34.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.104+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 msgr2=0x7f09cc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.104+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 0x7f09cc0a52e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f09d00cd010 tx=0x7f09d0098050 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.104+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 msgr2=0x7f09e0116210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.104+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f09d000e9c0 tx=0x7f09d000ee80 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 shutdown_connections 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09e0116750 0x7f09e0116be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f09e01116c0 0x7f09e0116210 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f09cc0a2e20 0x7f09cc0a52e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 --2- 172.21.15.73:0/3078599178 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f09e006a440 0x7f09e0119170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 >> 172.21.15.73:0/3078599178 conn(0x7f09e00731a0 msgr2=0x7f09e006d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 shutdown_connections 2024-06-06T07:16:34.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:34.105+0000 7f09e7f22640 1 -- 172.21.15.73:0/3078599178 wait complete. 2024-06-06T07:16:34.801 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16732 10.86 MB/sec warmup 99 sec latency 106.608 ms 2024-06-06T07:16:35.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:35 smithi073 ceph-mon[32303]: pgmap v481: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 430 op/s 2024-06-06T07:16:35.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:35 smithi136 ceph-mon[35901]: pgmap v481: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 430 op/s 2024-06-06T07:16:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:35 smithi170 ceph-mon[36108]: pgmap v481: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.7 MiB/s wr, 430 op/s 2024-06-06T07:16:35.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 16900 10.87 MB/sec warmup 100 sec latency 67.576 ms 2024-06-06T07:16:36.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17055 10.80 MB/sec warmup 101 sec latency 65.933 ms 2024-06-06T07:16:37.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:37 smithi073 ceph-mon[32303]: pgmap v482: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 275 op/s 2024-06-06T07:16:37.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:37 smithi136 ceph-mon[35901]: pgmap v482: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 275 op/s 2024-06-06T07:16:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:37 smithi170 ceph-mon[36108]: pgmap v482: 129 pgs: 129 active+clean; 486 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.7 MiB/s wr, 275 op/s 2024-06-06T07:16:37.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17234 10.81 MB/sec warmup 102 sec latency 61.595 ms 2024-06-06T07:16:38.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17394 10.75 MB/sec warmup 103 sec latency 54.686 ms 2024-06-06T07:16:39.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:39 smithi073 ceph-mon[32303]: pgmap v483: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 445 op/s 2024-06-06T07:16:39.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:39 smithi136 ceph-mon[35901]: pgmap v483: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 445 op/s 2024-06-06T07:16:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:39 smithi170 ceph-mon[36108]: pgmap v483: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 445 op/s 2024-06-06T07:16:39.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17579 10.71 MB/sec warmup 104 sec latency 72.125 ms 2024-06-06T07:16:40.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17784 10.71 MB/sec warmup 105 sec latency 74.449 ms 2024-06-06T07:16:41.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:41 smithi073 ceph-mon[32303]: pgmap v484: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:41.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:41 smithi136 ceph-mon[35901]: pgmap v484: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:41 smithi170 ceph-mon[36108]: pgmap v484: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:41.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 17957 10.66 MB/sec warmup 106 sec latency 68.082 ms 2024-06-06T07:16:42.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 18130 10.70 MB/sec warmup 107 sec latency 118.328 ms 2024-06-06T07:16:43.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:43 smithi073 ceph-mon[32303]: pgmap v485: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:43.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:43 smithi136 ceph-mon[35901]: pgmap v485: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:43.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:43 smithi170 ceph-mon[36108]: pgmap v485: 129 pgs: 129 active+clean; 481 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 312 op/s 2024-06-06T07:16:43.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 18327 10.74 MB/sec warmup 108 sec latency 114.226 ms 2024-06-06T07:16:44.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 18507 10.74 MB/sec warmup 109 sec latency 72.400 ms 2024-06-06T07:16:45.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:45 smithi073 ceph-mon[32303]: pgmap v486: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-06T07:16:45.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:45 smithi136 ceph-mon[35901]: pgmap v486: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-06T07:16:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:45 smithi170 ceph-mon[36108]: pgmap v486: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 451 op/s 2024-06-06T07:16:45.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 18701 10.74 MB/sec warmup 110 sec latency 55.559 ms 2024-06-06T07:16:46.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 18911 10.72 MB/sec warmup 111 sec latency 55.168 ms 2024-06-06T07:16:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 -- 172.21.15.73:0/3288561565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8106890 msgr2=0x7f4ec8106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3288561565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8106890 0x7f4ec8106c70 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec402a3f0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 -- 172.21.15.73:0/3288561565 shutdown_connections 2024-06-06T07:16:47.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3288561565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec8101290 0x7f4ec810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3288561565 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec81008d0 0x7f4ec8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3288561565 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8106890 0x7f4ec8106c70 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 -- 172.21.15.73:0/3288561565 >> 172.21.15.73:0/3288561565 conn(0x7f4ec8077a20 msgr2=0x7f4ec8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 -- 172.21.15.73:0/3288561565 shutdown_connections 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.250+0000 7f4ed0610640 1 -- 172.21.15.73:0/3288561565 wait complete. 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 Processor -- start 2024-06-06T07:16:47.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 -- start start 2024-06-06T07:16:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 0x7f4ec819a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec8106890 0x7f4ec819eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8110b60 con 0x7f4ec81008d0 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81109e0 con 0x7f4ec8101290 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ed0610640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec8110ce0 con 0x7f4ec8106890 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 0x7f4ec819a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ecdbab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ecdbab640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:58552/0 (socket says 172.21.15.73:58552) 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.251+0000 7f4ecdbab640 1 -- 172.21.15.73:0/3368047132 learned_addr learned my addr 172.21.15.73:0/3368047132 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecebad640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec8106890 0x7f4ec819eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 -- 172.21.15.73:0/3368047132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec8106890 msgr2=0x7f4ec819eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec8106890 0x7f4ec819eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 -- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 msgr2=0x7f4ec819a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 0x7f4ec819a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 -- 172.21.15.73:0/3368047132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec819f2c0 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 0x7f4ec819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9f0 tx=0x7f4eb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ebf7fe640 1 -- 172.21.15.73:0/3368047132 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb800cd80 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ebf7fe640 1 -- 172.21.15.73:0/3368047132 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb80108e0 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.252+0000 7f4ebf7fe640 1 -- 172.21.15.73:0/3368047132 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8018920 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.253+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec819f5b0 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.253+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec81a6e40 con 0x7f4ec8101290 2024-06-06T07:16:47.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.256+0000 7f4ebf7fe640 1 -- 172.21.15.73:0/3368047132 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4eb8010a90 con 0x7f4ec8101290 2024-06-06T07:16:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.256+0000 7f4ed33f4740 1 Processor -- start 2024-06-06T07:16:47.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 -- start start 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4bb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ebf7fe640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 0x7f4eb40a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 0x55c96c4c8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c96c41f4f0 con 0x55c96c4bba90 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c96c3bd870 con 0x55c96c4bb170 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c96c41b4b0 con 0x55c96c4c6350 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55630/0 (socket says 172.21.15.73:55630) 2024-06-06T07:16:47.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.257+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3138392116 learned_addr learned my addr 172.21.15.73:0/3138392116 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:47.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 0x55c96c4c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ebf7fe640 1 -- 172.21.15.73:0/3368047132 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f4eb80cfab0 con 0x7f4ec8101290 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3138392116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 msgr2=0x55c96c4c8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 0x55c96c4c8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ecebad640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 0x7f4eb40a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3138392116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 msgr2=0x55c96c4bb550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4bb550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3138392116 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96c3d99b0 con 0x55c96c4bba90 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 0x55c96c4c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec4004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe6382e874db38e server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.258+0000 7f4ecebad640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 0x7f4eb40a53c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005fd0 tx=0x7f4ec0009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.259+0000 7f4ebcbfe640 1 -- 172.21.15.73:0/3138392116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4035040 con 0x55c96c4bba90 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.259+0000 7f4ebcbfe640 1 -- 172.21.15.73:0/3138392116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ec4004760 con 0x55c96c4bba90 2024-06-06T07:16:47.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.259+0000 7f4ebcbfe640 1 -- 172.21.15.73:0/3138392116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4003ab0 con 0x55c96c4bba90 2024-06-06T07:16:47.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.259+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3138392116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 msgr2=0x55c96c4c5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.259+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009900 tx=0x7f4ec4004480 comp rx=0 tx=0).stop 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3138392116 shutdown_connections 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4c6350 0x55c96c4c8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bba90 0x55c96c4c5e10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3138392116 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4bb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3138392116 >> 172.21.15.73:0/3138392116 conn(0x55c96c4b7680 msgr2=0x55c96c4b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3138392116 shutdown_connections 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3138392116 wait complete. 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 -- start start 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.260+0000 7f4ed33f4740 1 Processor -- start 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4c3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4c3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c96c4c8f20 con 0x55c96c4bb170 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c96c3a8ae0 con 0x55c96c4c6350 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c96c4c90a0 con 0x55c96c4bba90 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ed33f4740 10 client.0 _ll_register_callbacks cb 0x55c96c3e19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55646/0 (socket says 172.21.15.73:55646) 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3640617950 learned_addr learned my addr 172.21.15.73:0/3640617950 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ecebad640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4c3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 msgr2=0x55c96c4c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4c3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3640617950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 msgr2=0x55c96c4c3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4c3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 -- 172.21.15.73:0/3640617950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96c133210 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402a9e0 tx=0x7f4ec4004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b954c0b6f1202531 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.261+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4053060 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f4ec40050c0 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ec4004760 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4ec4035050 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 10 client.17109 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c96c1ff620 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ec4004760 con 0x55c96c4bb170 2024-06-06T07:16:47.266 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f4ec403c440 con 0x55c96c4bb170 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 10 client.17109 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 10 client.17109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 20 client.17109 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c4e0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 4 client.17109 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c96c432f10 con 0x55c96c4deaf0 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.262+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c4e0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.263+0000 7f4eb1ffb640 10 client.17109 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:47.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.263+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c4e0ee0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5f0d3b013f3acd0 server_cookie=1b38c3406a52cfb8 in_seq=0 out_seq=0 2024-06-06T07:16:47.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/3640617950 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c96c432f10 con 0x55c96c4deaf0 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4eb1ffb640 10 client.17109 handle_command_reply: tid=0 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 client.17109 shutdown 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 msgr2=0x55c96c4c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402a9e0 tx=0x7f4ec4004f60 comp rx=0 tx=0).stop 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 shutdown_connections 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c4e0ee0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4c3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4eb1ffb640 0 client.17109 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/3640617950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c32c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 >> 172.21.15.73:0/3640617950 conn(0x55c96c4b7680 msgr2=0x55c96c4bd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 shutdown_connections 2024-06-06T07:16:47.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 1 -- 172.21.15.73:0/3640617950 wait complete. 2024-06-06T07:16:47.270 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.264+0000 7f4ed33f4740 20 client.17109 trim_cache size 0 max 16384 2024-06-06T07:16:47.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:47 smithi073 ceph-mon[32303]: pgmap v487: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-06-06T07:16:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 Processor -- start 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 -- start start 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81acbe0 con 0x55c96c4bb170 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ae930 con 0x55c96c4c6350 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ad370 con 0x55c96c4bba90 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55654/0 (socket says 172.21.15.73:55654) 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ece3ac640 1 -- 172.21.15.73:0/1213627700 learned_addr learned my addr 172.21.15.73:0/1213627700 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.296 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.294+0000 7f4ecebad640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 -- 172.21.15.73:0/1213627700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 msgr2=0x55c96c4e6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4e6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 -- 172.21.15.73:0/1213627700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 msgr2=0x55c96c4c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 -- 172.21.15.73:0/1213627700 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c96c4c1810 con 0x55c96c4bba90 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bdb80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80ccd90 tx=0x7f4eb8097f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6983aaaae0668ae8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4eb3fff640 1 -- 172.21.15.73:0/1213627700 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8026040 con 0x55c96c4bba90 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4eb3fff640 1 -- 172.21.15.73:0/1213627700 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb80a47d0 con 0x55c96c4bba90 2024-06-06T07:16:47.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.295+0000 7f4eb3fff640 1 -- 172.21.15.73:0/1213627700 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80cd010 con 0x55c96c4bba90 2024-06-06T07:16:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1213627700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 msgr2=0x55c96c4bdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bdb80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4eb80ccd90 tx=0x7f4eb8097f30 comp rx=0 tx=0).stop 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1213627700 shutdown_connections 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4c6350 0x55c96c4e6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bdb80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1213627700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4bb170 0x55c96c4c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1213627700 >> 172.21.15.73:0/1213627700 conn(0x55c96c4b7680 msgr2=0x55c96c4c5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1213627700 shutdown_connections 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1213627700 wait complete. 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 -- start start 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.296+0000 7f4ed33f4740 1 Processor -- start 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 0x55c96c4c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81ae260 con 0x55c96c4c6350 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:58602/0 (socket says 172.21.15.73:58602) 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 -- 172.21.15.73:0/1609936705 learned_addr learned my addr 172.21.15.73:0/1609936705 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81adad0 con 0x55c96c4bb170 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec81aac60 con 0x55c96c4bba90 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ed33f4740 10 client.0 _ll_register_callbacks cb 0x7f4eb80056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ecebad640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 0x55c96c4c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 -- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 msgr2=0x55c96c4bfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.297+0000 7f4ece3ac640 1 -- 172.21.15.73:0/1609936705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 msgr2=0x55c96c4c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 0x55c96c4c0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ece3ac640 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c96c4c0980 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ecebad640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 0x55c96c4c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ece3ac640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402a9e0 tx=0x7f4ec403cc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51f6d2c0ff330976 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4eb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4053040 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f4ec40051e0 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4eb80a47d0 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) ==== 24+0+0 (secure 0 0 0) 0x7f4ec4004760 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ed33f4740 10 client.26348 fetch_fsmap learned FSMap version 57 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.298+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) -- 0x55c96c549410 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4eb80a47d0 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 57) ==== 8418+0+0 (secure 0 0 0) 0x7f4ec4003aa0 con 0x55c96c4bb170 2024-06-06T07:16:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 10 client.26348 fetch_fsmap finished waiting for FSMap version 57 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 10 client.26348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 20 client.26348 populate_metadata read hostname 'smithi073' 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c54bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 4 client.26348 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c96c42cc20 con 0x55c96c4deaf0 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c54bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.299+0000 7f4eb1ffb640 10 client.26348 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.300+0000 7f4ecdbab640 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c54bbf0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab8aa9168a9e2127 server_cookie=3e55d071ee795a31 in_seq=0 out_seq=0 2024-06-06T07:16:47.304 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:47 smithi136 ceph-mon[35901]: pgmap v487: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-06-06T07:16:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:47 smithi170 ceph-mon[36108]: pgmap v487: 129 pgs: 129 active+clean; 487 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 308 op/s 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.353+0000 7f4eb1ffb640 1 -- 172.21.15.73:0/1609936705 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c96c42cc20 con 0x55c96c4deaf0 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.353+0000 7f4eb1ffb640 10 client.26348 handle_command_reply: tid=0 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.353+0000 7f4ed33f4740 1 client.26348 shutdown 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.353+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 msgr2=0x55c96c4d7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.353+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f4ec402a9e0 tx=0x7f4ec403cc80 comp rx=0 tx=0).stop 2024-06-06T07:16:47.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 shutdown_connections 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c96c4deaf0 0x55c96c54bbf0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4eb1ffb640 0 client.26348 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c96c4c6350 0x55c96c4c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c96c4bba90 0x55c96c4bfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 --2- 172.21.15.73:0/1609936705 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c96c4bb170 0x55c96c4d7160 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 >> 172.21.15.73:0/1609936705 conn(0x55c96c4b7680 msgr2=0x55c96c4c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 shutdown_connections 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 1 -- 172.21.15.73:0/1609936705 wait complete. 2024-06-06T07:16:47.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed33f4740 20 client.26348 trim_cache size 0 max 16384 2024-06-06T07:16:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:16:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:16:47.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 msgr2=0x7f4eb40a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 0x7f4eb40a53c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0005fd0 tx=0x7f4ec0009500 comp rx=0 tx=0).stop 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 msgr2=0x7f4ec819a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.354+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb800e9f0 tx=0x7f4eb800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 shutdown_connections 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f4eb40a2f00 0x7f4eb40a53c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4ec8106890 0x7f4ec819eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.358 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4ec8101290 0x7f4ec819a810 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 --2- 172.21.15.73:0/3368047132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4ec81008d0 0x7f4ec819a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:16:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 >> 172.21.15.73:0/3368047132 conn(0x7f4ec8077a20 msgr2=0x7f4ec8069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:16:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 shutdown_connections 2024-06-06T07:16:47.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:16:47.355+0000 7f4ed0610640 1 -- 172.21.15.73:0/3368047132 wait complete. 2024-06-06T07:16:47.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19054 10.67 MB/sec warmup 112 sec latency 88.824 ms 2024-06-06T07:16:48.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19208 10.64 MB/sec warmup 113 sec latency 63.597 ms 2024-06-06T07:16:49.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:48 smithi073 ceph-mon[32303]: pgmap v488: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-06-06T07:16:49.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:48 smithi136 ceph-mon[35901]: pgmap v488: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-06-06T07:16:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:48 smithi170 ceph-mon[36108]: pgmap v488: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.2 MiB/s wr, 447 op/s 2024-06-06T07:16:49.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19385 10.67 MB/sec warmup 114 sec latency 111.807 ms 2024-06-06T07:16:50.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:49 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:16:50.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:49 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:16:50.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:49 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:16:50.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19537 10.62 MB/sec warmup 115 sec latency 118.677 ms 2024-06-06T07:16:51.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:50 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:16:51.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:50 smithi073 ceph-mon[32303]: pgmap v489: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:51.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:50 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:16:51.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:50 smithi136 ceph-mon[35901]: pgmap v489: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:51.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:50 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:16:51.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:50 smithi170 ceph-mon[36108]: pgmap v489: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:51.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19762 10.73 MB/sec warmup 116 sec latency 73.109 ms 2024-06-06T07:16:52.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 19934 10.69 MB/sec warmup 117 sec latency 117.047 ms 2024-06-06T07:16:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:52 smithi073 ceph-mon[32303]: pgmap v490: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:52 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:16:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:52 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:16:53.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:52 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:16:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:52 smithi136 ceph-mon[35901]: pgmap v490: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:52 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:16:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:52 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:16:53.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:52 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:16:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:52 smithi170 ceph-mon[36108]: pgmap v490: 129 pgs: 129 active+clean; 495 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 277 op/s 2024-06-06T07:16:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:52 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:16:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:52 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:16:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:52 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:16:53.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 20078 10.69 MB/sec warmup 118 sec latency 94.350 ms 2024-06-06T07:16:54.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 20219 10.63 MB/sec warmup 119 sec latency 60.091 ms 2024-06-06T07:16:55.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:54 smithi073 ceph-mon[32303]: pgmap v491: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 418 op/s 2024-06-06T07:16:55.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:54 smithi136 ceph-mon[35901]: pgmap v491: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 418 op/s 2024-06-06T07:16:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:54 smithi170 ceph-mon[36108]: pgmap v491: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.8 MiB/s wr, 418 op/s 2024-06-06T07:16:56.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 20567 5.50 MB/sec execute 1 sec latency 80.895 ms 2024-06-06T07:16:57.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:56 smithi073 ceph-mon[32303]: pgmap v492: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-06T07:16:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:56 smithi136 ceph-mon[35901]: pgmap v492: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-06T07:16:57.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:56 smithi170 ceph-mon[36108]: pgmap v492: 129 pgs: 129 active+clean; 505 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-06T07:16:57.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 20710 5.11 MB/sec execute 2 sec latency 150.656 ms 2024-06-06T07:16:58.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 20877 6.24 MB/sec execute 3 sec latency 123.883 ms 2024-06-06T07:16:59.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:16:59 smithi073 ceph-mon[32303]: pgmap v493: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-06-06T07:16:59.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:16:59 smithi136 ceph-mon[35901]: pgmap v493: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-06-06T07:16:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:16:59 smithi170 ceph-mon[36108]: pgmap v493: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.6 MiB/s wr, 428 op/s 2024-06-06T07:16:59.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21059 7.34 MB/sec execute 4 sec latency 70.629 ms 2024-06-06T07:17:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2407590863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a44100690 msgr2=0x7f2a44100b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2407590863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a44100690 0x7f2a44100b10 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2a30009930 tx=0x7f2a3002a420 comp rx=0 tx=0).stop 2024-06-06T07:17:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2407590863 shutdown_connections 2024-06-06T07:17:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2407590863 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4410d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2407590863 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a44100690 0x7f2a44100b10 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2407590863 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a441066a0 0x7f2a44106a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2407590863 >> 172.21.15.73:0/2407590863 conn(0x7f2a440fc1c0 msgr2=0x7f2a440fe5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2407590863 shutdown_connections 2024-06-06T07:17:00.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.483+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2407590863 wait complete. 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 Processor -- start 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 -- start start 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a44100690 0x7f2a440734f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441109b0 con 0x7f2a441066a0 2024-06-06T07:17:00.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44110830 con 0x7f2a44101050 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a49fd6640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a44110b30 con 0x7f2a44100690 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a497d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.484+0000 7f2a497d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38636/0 (socket says 172.21.15.73:38636) 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 -- 172.21.15.73:0/2565985013 learned_addr learned my addr 172.21.15.73:0/2565985013 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a44100690 0x7f2a440734f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 -- 172.21.15.73:0/2565985013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a44100690 msgr2=0x7f2a440734f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a44100690 0x7f2a440734f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 -- 172.21.15.73:0/2565985013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 msgr2=0x7f2a4406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4406b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 -- 172.21.15.73:0/2565985013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a4406c2b0 con 0x7f2a441066a0 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4406b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:00.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00eb10 tx=0x7f2a3c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.485+0000 7f2a41ffb640 1 -- 172.21.15.73:0/2565985013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c004b20 con 0x7f2a441066a0 2024-06-06T07:17:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.486+0000 7f2a41ffb640 1 -- 172.21.15.73:0/2565985013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a3c004cc0 con 0x7f2a441066a0 2024-06-06T07:17:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.486+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a441ace60 con 0x7f2a441066a0 2024-06-06T07:17:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.486+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a441ad2d0 con 0x7f2a441066a0 2024-06-06T07:17:00.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.487+0000 7f2a41ffb640 1 -- 172.21.15.73:0/2565985013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c010690 con 0x7f2a441066a0 2024-06-06T07:17:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.489+0000 7f2a4e01e740 1 Processor -- start 2024-06-06T07:17:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a41ffb640 1 -- 172.21.15.73:0/2565985013 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a3c010830 con 0x7f2a441066a0 2024-06-06T07:17:00.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 -- start start 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a327208720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327215910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a32716c4f0 con 0x55a327208c60 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a32710a870 con 0x55a327208340 2024-06-06T07:17:00.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.490+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a3271684b0 con 0x55a327213520 2024-06-06T07:17:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a327208720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327215910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38648/0 (socket says 172.21.15.73:38648) 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a48fd4640 1 -- 172.21.15.73:0/472947213 learned_addr learned my addr 172.21.15.73:0/472947213 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.491+0000 7f2a48fd4640 1 -- 172.21.15.73:0/472947213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 msgr2=0x55a327215910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327215910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a48fd4640 1 -- 172.21.15.73:0/472947213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 msgr2=0x55a327208720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a327208720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a48fd4640 1 -- 172.21.15.73:0/472947213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3271269b0 con 0x55a327208c60 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a41ffb640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 0x7f2a340a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a43fff640 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a327208720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a41ffb640 1 -- 172.21.15.73:0/2565985013 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f2a3c0cfe60 con 0x7f2a441066a0 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 0x7f2a340a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.492+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009c70 tx=0x7f2a38011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c20b5898e5a0fd1c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.493+0000 7f2a2effd640 1 -- 172.21.15.73:0/472947213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38004230 con 0x55a327208c60 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.493+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 0x7f2a340a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a30031440 tx=0x7f2a30000bc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.493+0000 7f2a2effd640 1 -- 172.21.15.73:0/472947213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a380043d0 con 0x55a327208c60 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.493+0000 7f2a2effd640 1 -- 172.21.15.73:0/472947213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380197a0 con 0x55a327208c60 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- 172.21.15.73:0/472947213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 msgr2=0x55a327212fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f2a38009c70 tx=0x7f2a38011810 comp rx=0 tx=0).stop 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- 172.21.15.73:0/472947213 shutdown_connections 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327215910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212fe0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/472947213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a327208720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- 172.21.15.73:0/472947213 >> 172.21.15.73:0/472947213 conn(0x55a327204850 msgr2=0x55a327204c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- 172.21.15.73:0/472947213 shutdown_connections 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- 172.21.15.73:0/472947213 wait complete. 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.494+0000 7f2a4e01e740 1 -- start start 2024-06-06T07:17:00.496 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 Processor -- start 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 0x55a3272104c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327210a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 0x55a327210f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55a327216150 con 0x55a327208c60 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55a3270f5ae0 con 0x55a327213520 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55a3272162d0 con 0x55a327208340 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a4e01e740 10 client.0 _ll_register_callbacks cb 0x55a32712e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 0x55a3272104c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a43fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327210a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 0x55a3272104c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:37888/0 (socket says 172.21.15.73:37888) 2024-06-06T07:17:00.499 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a48fd4640 1 -- 172.21.15.73:0/2084929073 learned_addr learned my addr 172.21.15.73:0/2084929073 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.495+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 0x55a327210f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 -- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 msgr2=0x55a3272104c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 0x55a3272104c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 -- 172.21.15.73:0/2084929073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 msgr2=0x55a327210a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327210a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 -- 172.21.15.73:0/2084929073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a326e80210 con 0x55a327213520 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 0x55a327210f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0cd140 tx=0x7f2a3c09b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f63aadcd33f3f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a23fff640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a3c0a48f0 con 0x55a327213520 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a380043d0 con 0x55a327213520 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.496+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f2a3c0ba800 con 0x55a327213520 2024-06-06T07:17:00.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2a3c080050 con 0x55a327213520 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 10 client.26360 fetch_fsmap learned FSMap version 58 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a326f4c620 con 0x55a327213520 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a380043d0 con 0x55a327213520 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7f2a3c0ba4d0 con 0x55a327213520 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 10 client.26360 fetch_fsmap finished waiting for FSMap version 58 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 10 client.26360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 20 client.26360 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a3272237a0 0x55a327225c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 4 client.26360 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a32717ff10 con 0x55a3272237a0 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.497+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a3272237a0 0x55a327225c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.498+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a3272237a0 0x55a327225c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba81399c09d84797 server_cookie=5019c4abd718367b in_seq=0 out_seq=0 2024-06-06T07:17:00.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.498+0000 7f2a23fff640 10 client.26360 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.499+0000 7f2a23fff640 1 -- 172.21.15.73:0/2084929073 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a32717ff10 con 0x55a3272237a0 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.499+0000 7f2a23fff640 10 client.26360 handle_command_reply: tid=0 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.499+0000 7f2a4e01e740 1 client.26360 shutdown 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 msgr2=0x55a327210f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 0x55a327210f40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c0cd140 tx=0x7f2a3c09b930 comp rx=0 tx=0).stop 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 shutdown_connections 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a3272237a0 0x55a327225c00 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327213520 0x55a327210f40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327210a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2084929073 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327208340 0x55a3272104c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 >> 172.21.15.73:0/2084929073 conn(0x55a327204850 msgr2=0x55a32720a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a23fff640 0 client.26360 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:00.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 shutdown_connections 2024-06-06T07:17:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2084929073 wait complete. 2024-06-06T07:17:00.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.500+0000 7f2a4e01e740 20 client.26360 trim_cache size 0 max 16384 2024-06-06T07:17:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 Processor -- start 2024-06-06T07:17:00.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 -- start start 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a32722b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b5400 con 0x55a327208c60 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b70f0 con 0x55a327208340 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.528+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b5b90 con 0x55a327213520 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:51850/0 (socket says 172.21.15.73:51850) 2024-06-06T07:17:00.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 -- 172.21.15.73:0/863081689 learned_addr learned my addr 172.21.15.73:0/863081689 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a497d5640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a32722b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a43fff640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 -- 172.21.15.73:0/863081689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 msgr2=0x55a32722b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a32722b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 -- 172.21.15.73:0/863081689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 msgr2=0x55a327212400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 -- 172.21.15.73:0/863081689 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3272092d0 con 0x55a327208340 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a43fff640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a38003ee0 tx=0x7f2a38009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29d86715d54f3ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a2e7fc640 1 -- 172.21.15.73:0/863081689 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a38003f80 con 0x55a327208340 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.529+0000 7f2a2e7fc640 1 -- 172.21.15.73:0/863081689 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a380035f0 con 0x55a327208340 2024-06-06T07:17:00.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.530+0000 7f2a2e7fc640 1 -- 172.21.15.73:0/863081689 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a380051a0 con 0x55a327208340 2024-06-06T07:17:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 -- 172.21.15.73:0/863081689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 msgr2=0x55a32720ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2a38003ee0 tx=0x7f2a38009ee0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 -- 172.21.15.73:0/863081689 shutdown_connections 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a32722b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327212400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/863081689 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a32720ad00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 -- 172.21.15.73:0/863081689 >> 172.21.15.73:0/863081689 conn(0x55a327204850 msgr2=0x55a327211cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 -- 172.21.15.73:0/863081689 shutdown_connections 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.531+0000 7f2a4e01e740 1 -- 172.21.15.73:0/863081689 wait complete. 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.532+0000 7f2a4e01e740 1 -- start start 2024-06-06T07:17:00.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.532+0000 7f2a4e01e740 1 Processor -- start 2024-06-06T07:17:00.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a3272867e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a3272867e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327286d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327216ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b6a20 con 0x55a327208c60 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b6290 con 0x55a327208340 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a441b3400 con 0x55a327213520 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a4e01e740 10 client.0 _ll_register_callbacks cb 0x7f2a3c0057a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a48fd4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a3272867e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:51852/0 (socket says 172.21.15.73:51852) 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a48fd4640 1 -- 172.21.15.73:0/2547490753 learned_addr learned my addr 172.21.15.73:0/2547490753 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327286d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.535 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a497d5640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327216ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 -- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 msgr2=0x55a327216ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327216ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 -- 172.21.15.73:0/2547490753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 msgr2=0x55a3272867e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a3272867e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.533+0000 7f2a43fff640 1 -- 172.21.15.73:0/2547490753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3272171a0 con 0x55a327208c60 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.534+0000 7f2a43fff640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327286d20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002a940 tx=0x7f2a30005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcc09d576034b4b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.534+0000 7f2a23fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.534+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30058060 con 0x55a327208c60 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.534+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a380035f0 con 0x55a327208c60 2024-06-06T07:17:00.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.534+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(123..123 src has 1..123) ==== 9554+0+0 (secure 0 0 0) 0x7f2a300035f0 con 0x55a327208c60 2024-06-06T07:17:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) ==== 24+0+0 (secure 0 0 0) 0x7f2a3005f050 con 0x55a327208c60 2024-06-06T07:17:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 10 client.17130 fetch_fsmap learned FSMap version 58 2024-06-06T07:17:00.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) -- 0x55a3272966a0 con 0x55a327208c60 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a380035f0 con 0x55a327208c60 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) ==== 8410+0+0 (secure 0 0 0) 0x7f2a300327e0 con 0x55a327208c60 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 10 client.17130 fetch_fsmap finished waiting for FSMap version 58 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 20 client.17130 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a327296f10 0x55a327299330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.535+0000 7f2a4e01e740 4 client.17130 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.536+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a327179c20 con 0x55a327296f10 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.536+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a327296f10 0x55a327299330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.537+0000 7f2a23fff640 10 client.17130 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:00.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:00.537+0000 7f2a48fd4640 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a327296f10 0x55a327299330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40c948ed3a81a48f server_cookie=4ae4e40046d73b6b in_seq=0 out_seq=0 2024-06-06T07:17:00.802 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21189 7.41 MB/sec execute 5 sec latency 297.110 ms 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a23fff640 1 -- 172.21.15.73:0/2547490753 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a327179c20 con 0x55a327296f10 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a23fff640 10 client.17130 handle_command_reply: tid=0 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a4e01e740 1 client.17130 shutdown 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 msgr2=0x55a327286d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327286d20 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002a940 tx=0x7f2a30005a20 comp rx=0 tx=0).stop 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 shutdown_connections 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.022+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55a327296f10 0x55a327299330 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55a327213520 0x55a327216ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.024 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55a327208c60 0x55a327286d20 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 --2- 172.21.15.73:0/2547490753 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55a327208340 0x55a3272867e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 >> 172.21.15.73:0/2547490753 conn(0x55a327204850 msgr2=0x55a327210d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a23fff640 0 client.17130 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 shutdown_connections 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 1 -- 172.21.15.73:0/2547490753 wait complete. 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.023+0000 7f2a4e01e740 20 client.17130 trim_cache size 0 max 16384 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:17:01.025 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 msgr2=0x7f2a340a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 0x7f2a340a51b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2a30031440 tx=0x7f2a30000bc0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 msgr2=0x7f2a4406b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00eb10 tx=0x7f2a3c00c2d0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 shutdown_connections 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a441066a0 0x7f2a4406b9f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a44101050 0x7f2a4406b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a340a2cf0 0x7f2a340a51b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 --2- 172.21.15.73:0/2565985013 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a44100690 0x7f2a440734f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 >> 172.21.15.73:0/2565985013 conn(0x7f2a440fc1c0 msgr2=0x7f2a44069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 shutdown_connections 2024-06-06T07:17:01.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:01.024+0000 7f2a49fd6640 1 -- 172.21.15.73:0/2565985013 wait complete. 2024-06-06T07:17:01.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:01 smithi073 ceph-mon[32303]: pgmap v494: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:01.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-mon[35901]: pgmap v494: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:01.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:01 smithi170 ceph-mon[36108]: pgmap v494: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:01.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21325 7.32 MB/sec execute 6 sec latency 552.260 ms 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: *** Caught signal (Segmentation fault) ** 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: in thread 7f2edcea7700 thread_name:md_log_replay 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2eede4bd20] 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2: /usr/bin/ceph-mds(+0x549a12) [0x55909ac25a12] 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55909ac291c3] 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 4: (MDLog::_replay_thread()+0x617) [0x55909ac83ff7] 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55909ac8d861] 2024-06-06T07:17:02.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2eede411ca] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 7: clone() 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:01.862+0000 7f2edcea7700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: in thread 7f2edcea7700 thread_name:md_log_replay 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2eede4bd20] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2: /usr/bin/ceph-mds(+0x549a12) [0x55909ac25a12] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55909ac291c3] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 4: (MDLog::_replay_thread()+0x617) [0x55909ac83ff7] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55909ac8d861] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2eede411ca] 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 7: clone() 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.303 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 0> 2024-06-06T07:17:01.862+0000 7f2edcea7700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: in thread 7f2edcea7700 thread_name:md_log_replay 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2eede4bd20] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2: /usr/bin/ceph-mds(+0x549a12) [0x55909ac25a12] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55909ac291c3] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 4: (MDLog::_replay_thread()+0x617) [0x55909ac83ff7] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55909ac8d861] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2eede411ca] 2024-06-06T07:17:02.304 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 7: clone() 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 0> 2024-06-06T07:17:01.862+0000 7f2edcea7700 -1 *** Caught signal (Segmentation fault) ** 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: in thread 7f2edcea7700 thread_name:md_log_replay 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 1: /lib64/libpthread.so.0(+0x12d20) [0x7f2eede4bd20] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2: /usr/bin/ceph-mds(+0x549a12) [0x55909ac25a12] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x55909ac291c3] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 4: (MDLog::_replay_thread()+0x617) [0x55909ac83ff7] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 5: (MDLog::ReplayThread::entry()+0x11) [0x55909ac8d861] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7f2eede411ca] 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 7: clone() 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2024-06-06T07:17:02.305 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[69745]: 2024-06-06T07:17:02.306 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:01 smithi136 podman[73675]: 2024-06-06 07:17:01.933015653 +0000 UTC m=+0.023290871 container died 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-06T07:17:02.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21520 7.62 MB/sec execute 7 sec latency 184.757 ms 2024-06-06T07:17:03.115 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:02 smithi136 podman[73675]: 2024-06-06 07:17:02.815134996 +0000 UTC m=+0.905410209 container remove 2185d951072ca97e88f900c3741730f860560a23862e105691b34cc108024344 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-06T07:17:03.115 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:02 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Main process exited, code=exited, status=139/n/a 2024-06-06T07:17:03.115 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:03 smithi136 ceph-mon[35901]: pgmap v495: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:03.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:03 smithi073 ceph-mon[32303]: pgmap v495: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:03.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:03 smithi170 ceph-mon[36108]: pgmap v495: 129 pgs: 129 active+clean; 507 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 289 op/s 2024-06-06T07:17:03.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:03 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Failed with result 'exit-code'. 2024-06-06T07:17:03.551 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:03 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Consumed 7.607s CPU time. 2024-06-06T07:17:03.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21713 8.28 MB/sec execute 8 sec latency 61.221 ms 2024-06-06T07:17:04.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 21866 7.99 MB/sec execute 9 sec latency 96.997 ms 2024-06-06T07:17:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:05 smithi073 ceph-mon[32303]: pgmap v496: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 453 op/s 2024-06-06T07:17:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:05 smithi073 ceph-mon[32303]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T07:17:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:05 smithi136 ceph-mon[35901]: pgmap v496: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 453 op/s 2024-06-06T07:17:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:05 smithi136 ceph-mon[35901]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T07:17:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:05 smithi170 ceph-mon[36108]: pgmap v496: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.4 MiB/s wr, 453 op/s 2024-06-06T07:17:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:05 smithi170 ceph-mon[36108]: osdmap e124: 12 total, 12 up, 12 in 2024-06-06T07:17:05.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 22043 8.05 MB/sec execute 10 sec latency 67.130 ms 2024-06-06T07:17:06.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:06 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:06.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:06 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:06 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:06.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 22205 7.64 MB/sec execute 11 sec latency 52.483 ms 2024-06-06T07:17:07.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:07 smithi136 ceph-mon[35901]: pgmap v498: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-06-06T07:17:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:07 smithi170 ceph-mon[36108]: pgmap v498: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-06-06T07:17:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:07 smithi073 ceph-mon[32303]: pgmap v498: 129 pgs: 129 active+clean; 515 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 373 op/s 2024-06-06T07:17:07.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 22403 8.13 MB/sec execute 12 sec latency 100.381 ms 2024-06-06T07:17:08.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 22591 8.36 MB/sec execute 13 sec latency 86.047 ms 2024-06-06T07:17:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:09 smithi136 ceph-mon[35901]: pgmap v499: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:09 smithi170 ceph-mon[36108]: pgmap v499: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:09 smithi073 ceph-mon[32303]: pgmap v499: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:09.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 22800 8.56 MB/sec execute 14 sec latency 59.462 ms 2024-06-06T07:17:10.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23004 8.99 MB/sec execute 15 sec latency 71.301 ms 2024-06-06T07:17:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:11 smithi136 ceph-mon[35901]: pgmap v500: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:11 smithi170 ceph-mon[36108]: pgmap v500: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:11 smithi073 ceph-mon[32303]: pgmap v500: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:11.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23210 9.04 MB/sec execute 16 sec latency 70.705 ms 2024-06-06T07:17:12.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23389 9.47 MB/sec execute 17 sec latency 68.053 ms 2024-06-06T07:17:13.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:13 smithi136 ceph-mon[35901]: pgmap v501: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:13.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:13 smithi136 ceph-mon[35901]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T07:17:13.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:13 smithi170 ceph-mon[36108]: pgmap v501: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:13.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:13 smithi170 ceph-mon[36108]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T07:17:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:13 smithi073 ceph-mon[32303]: pgmap v501: 129 pgs: 129 active+clean; 532 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-06T07:17:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:13 smithi073 ceph-mon[32303]: osdmap e125: 12 total, 12 up, 12 in 2024-06-06T07:17:13.583 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Scheduled restart job, restart counter is at 2. 2024-06-06T07:17:13.583 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 systemd[1]: Stopped Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T07:17:13.583 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service: Consumed 7.607s CPU time. 2024-06-06T07:17:13.583 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 systemd[1]: Starting Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:17:13.583 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 podman[73777]: 2024-06-06 07:17:13.45163139 +0000 UTC m=+0.023265863 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:17:13.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23539 9.27 MB/sec execute 18 sec latency 110.767 ms 2024-06-06T07:17:13.837 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 podman[73777]: 2024-06-06 07:17:13.583046732 +0000 UTC m=+0.154681184 container create 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T07:17:14.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.160+0000 7f142927b640 1 -- 172.21.15.73:0/4126996692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1424101030 msgr2=0x7f1424101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.160+0000 7f142927b640 1 --2- 172.21.15.73:0/4126996692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1424101030 0x7f1424101490 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f1418009900 tx=0x7f141802a3e0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 -- 172.21.15.73:0/4126996692 shutdown_connections 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 --2- 172.21.15.73:0/4126996692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f14241019d0 0x7f142410d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 --2- 172.21.15.73:0/4126996692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1424101030 0x7f1424101490 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 --2- 172.21.15.73:0/4126996692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f1424105c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 -- 172.21.15.73:0/4126996692 >> 172.21.15.73:0/4126996692 conn(0x7f1424076e30 msgr2=0x7f14240fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.161+0000 7f142927b640 1 -- 172.21.15.73:0/4126996692 shutdown_connections 2024-06-06T07:17:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.162+0000 7f142927b640 1 -- 172.21.15.73:0/4126996692 wait complete. 2024-06-06T07:17:14.163 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.162+0000 7f142927b640 1 Processor -- start 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 -- start start 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1424101030 0x7f14241a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f142410c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1424111d90 con 0x7f14241019d0 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1424111c10 con 0x7f1424105890 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f142927b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1424111f10 con 0x7f1424101030 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f14227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f14227fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35954/0 (socket says 172.21.15.73:35954) 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f14227fc640 1 -- 172.21.15.73:0/1862424637 learned_addr learned my addr 172.21.15.73:0/1862424637 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:14.165 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1424101030 0x7f14241a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.163+0000 7f14237fe640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f142410c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14227fc640 1 -- 172.21.15.73:0/1862424637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1424101030 msgr2=0x7f14241a4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14227fc640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1424101030 0x7f14241a4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.166 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14227fc640 1 -- 172.21.15.73:0/1862424637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 msgr2=0x7f142410c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14227fc640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f142410c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14227fc640 1 -- 172.21.15.73:0/1862424637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142410cf90 con 0x7f14241019d0 2024-06-06T07:17:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.164+0000 7f14237fe640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f142410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:17:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.165+0000 7f14227fc640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f141802a9d0 tx=0x7f1418003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.167 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.166+0000 7f140bfff640 1 -- 172.21.15.73:0/1862424637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141803f070 con 0x7f14241019d0 2024-06-06T07:17:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.167+0000 7f140bfff640 1 -- 172.21.15.73:0/1862424637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f141803bb20 con 0x7f14241019d0 2024-06-06T07:17:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.167+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142410d1c0 con 0x7f14241019d0 2024-06-06T07:17:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.167+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142406b3c0 con 0x7f14241019d0 2024-06-06T07:17:14.169 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.167+0000 7f140bfff640 1 -- 172.21.15.73:0/1862424637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418035030 con 0x7f14241019d0 2024-06-06T07:17:14.172 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.171+0000 7f140bfff640 1 -- 172.21.15.73:0/1862424637 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1418042070 con 0x7f14241019d0 2024-06-06T07:17:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.173+0000 7f140bfff640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 0x7f140c0a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.173+0000 7f140bfff640 1 -- 172.21.15.73:0/1862424637 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f14180f0d70 con 0x7f14241019d0 2024-06-06T07:17:14.173 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.173+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 0x7f140c0a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.173+0000 7f142bf81740 1 Processor -- start 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 -- start start 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315e2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2a90 0x55d3315ece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315ef740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3315464f0 con 0x55d3315e2170 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d3314e4870 con 0x55d3315ed350 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f142bf81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d3315424b0 con 0x55d3315e2a90 2024-06-06T07:17:14.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f14237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315ef740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f14237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315ef740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:34968/0 (socket says 172.21.15.73:34968) 2024-06-06T07:17:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f14237fe640 1 -- 172.21.15.73:0/658621824 learned_addr learned my addr 172.21.15.73:0/658621824 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:14.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 0x7f140c0a53c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f141400bbe0 tx=0x7f141400b390 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.174+0000 7f1422ffd640 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315e2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 -- 172.21.15.73:0/658621824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2a90 msgr2=0x55d3315ece10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2a90 0x55d3315ece10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 -- 172.21.15.73:0/658621824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 msgr2=0x55d3315ef740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315ef740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 -- 172.21.15.73:0/658621824 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3315009b0 con 0x55d3315e2170 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.175+0000 7f1422ffd640 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315e2550 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1414010f00 tx=0x7f1414013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e0736b05f3c0d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.176+0000 7f14097fa640 1 -- 172.21.15.73:0/658621824 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141401e620 con 0x55d3315e2170 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.176+0000 7f14097fa640 1 -- 172.21.15.73:0/658621824 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1414011e50 con 0x55d3315e2170 2024-06-06T07:17:14.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.176+0000 7f14097fa640 1 -- 172.21.15.73:0/658621824 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141401dc10 con 0x55d3315e2170 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 -- 172.21.15.73:0/658621824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 msgr2=0x55d3315e2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315e2550 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1414010f00 tx=0x7f1414013000 comp rx=0 tx=0).stop 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 -- 172.21.15.73:0/658621824 shutdown_connections 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315ef740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2a90 0x55d3315ece10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 --2- 172.21.15.73:0/658621824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315e2550 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 -- 172.21.15.73:0/658621824 >> 172.21.15.73:0/658621824 conn(0x55d3315de680 msgr2=0x55d3315dea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 -- 172.21.15.73:0/658621824 shutdown_connections 2024-06-06T07:17:14.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.177+0000 7f142bf81740 1 -- 172.21.15.73:0/658621824 wait complete. 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 -- start start 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 Processor -- start 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 0x55d3315ea390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315ea8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315eae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3315eff80 con 0x55d3315e2a90 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d3314cfae0 con 0x55d3315e2170 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d3315f0100 con 0x55d3315ed350 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f142bf81740 10 client.0 _ll_register_callbacks cb 0x55d3315089b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f1422ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 0x55d3315ea390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.178+0000 7f1422ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 0x55d3315ea390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:34978/0 (socket says 172.21.15.73:34978) 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f1422ffd640 1 -- 172.21.15.73:0/908544607 learned_addr learned my addr 172.21.15.73:0/908544607 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315ea8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 -- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 msgr2=0x55d3315eae10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315eae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 -- 172.21.15.73:0/908544607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 msgr2=0x55d3315ea390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 0x55d3315ea390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 -- 172.21.15.73:0/908544607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33125a210 con 0x55d3315e2a90 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f14227fc640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315ea8d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f14180ee110 tx=0x7f14180bc840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c8466cddef2e03 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f1406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:17:14.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.179+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180c1290 con 0x55d3315e2a90 2024-06-06T07:17:14.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f14180b9550 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1414011e50 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f14180c5b50 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 10 client.17148 fetch_fsmap learned FSMap version 61 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d331326620 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1414011e50 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 61) ==== 8406+0+0 (secure 0 0 0) 0x7f14180bc330 con 0x55d3315e2a90 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 10 client.17148 fetch_fsmap finished waiting for FSMap version 61 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 10 client.17148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 20 client.17148 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3315fd7a0 0x55d3315ffbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 4 client.17148 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.180+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d331559f10 con 0x55d3315fd7a0 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.181+0000 7f1422ffd640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3315fd7a0 0x55d3315ffbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.181+0000 7f1406ff5640 10 client.17148 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.181+0000 7f1422ffd640 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3315fd7a0 0x55d3315ffbc0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68763175ad7b1fdc server_cookie=bb3bc10eb3134895 in_seq=0 out_seq=0 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f1406ff5640 1 -- 172.21.15.73:0/908544607 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d331559f10 con 0x55d3315fd7a0 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f1406ff5640 10 client.17148 handle_command_reply: tid=0 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 client.17148 shutdown 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 msgr2=0x55d3315ea8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315ea8d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f14180ee110 tx=0x7f14180bc840 comp rx=0 tx=0).stop 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 shutdown_connections 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3315fd7a0 0x55d3315ffbc0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315eae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315ea8d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 --2- 172.21.15.73:0/908544607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2170 0x55d3315ea390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 >> 172.21.15.73:0/908544607 conn(0x55d3315de680 msgr2=0x55d3315e4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.182+0000 7f1406ff5640 0 client.17148 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.183+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 shutdown_connections 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.183+0000 7f142bf81740 1 -- 172.21.15.73:0/908544607 wait complete. 2024-06-06T07:17:14.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.183+0000 7f142bf81740 20 client.17148 trim_cache size 0 max 16384 2024-06-06T07:17:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.213+0000 7f142bf81740 1 Processor -- start 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.213+0000 7f142bf81740 1 -- start start 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315d99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2a90 0x55d3315e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f141000fe80 con 0x55d3315e2170 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1410011b70 con 0x55d3315e2a90 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f142bf81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1410010610 con 0x55d3315ed350 2024-06-06T07:17:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f14227fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2a90 0x55d3315e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f14237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f14237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:46194/0 (socket says 172.21.15.73:46194) 2024-06-06T07:17:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.214+0000 7f14237fe640 1 -- 172.21.15.73:0/1552132964 learned_addr learned my addr 172.21.15.73:0/1552132964 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 -- 172.21.15.73:0/1552132964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2a90 msgr2=0x55d3315e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2a90 0x55d3315e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 -- 172.21.15.73:0/1552132964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 msgr2=0x55d3315d99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315d99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 -- 172.21.15.73:0/1552132964 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3315f9ec0 con 0x55d3315ed350 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f14237fe640 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f141c005fd0 tx=0x7f141c012820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc7370ccac1aa2d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.215+0000 7f1408ff9640 1 -- 172.21.15.73:0/1552132964 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c004d20 con 0x55d3315ed350 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.216+0000 7f1408ff9640 1 -- 172.21.15.73:0/1552132964 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f141c0054d0 con 0x55d3315ed350 2024-06-06T07:17:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.216+0000 7f1408ff9640 1 -- 172.21.15.73:0/1552132964 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141c010a30 con 0x55d3315ed350 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- 172.21.15.73:0/1552132964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 msgr2=0x55d3315dd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f141c005fd0 tx=0x7f141c012820 comp rx=0 tx=0).stop 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- 172.21.15.73:0/1552132964 shutdown_connections 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315ed350 0x55d3315dd660 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315e2a90 0x55d3315e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 --2- 172.21.15.73:0/1552132964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2170 0x55d3315d99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- 172.21.15.73:0/1552132964 >> 172.21.15.73:0/1552132964 conn(0x55d3315de680 msgr2=0x55d3315dcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- 172.21.15.73:0/1552132964 shutdown_connections 2024-06-06T07:17:14.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- 172.21.15.73:0/1552132964 wait complete. 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.217+0000 7f142bf81740 1 -- start start 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 Processor -- start 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 0x55d3315f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315f0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315f1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f14100114a0 con 0x55d3315e2a90 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1410010d10 con 0x55d3315ed350 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f141000de60 con 0x55d3315e2170 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f142bf81740 10 client.0 _ll_register_callbacks cb 0x7f1418005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f1422ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 0x55d3315f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f14237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315f1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f14237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315f1190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:34994/0 (socket says 172.21.15.73:34994) 2024-06-06T07:17:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.218+0000 7f14237fe640 1 -- 172.21.15.73:0/1710897408 learned_addr learned my addr 172.21.15.73:0/1710897408 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f14227fc640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315f0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 -- 172.21.15.73:0/1710897408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 msgr2=0x55d3315f1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315f1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 -- 172.21.15.73:0/1710897408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 msgr2=0x55d3315f0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315f0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 -- 172.21.15.73:0/1710897408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3315f1750 con 0x55d3315e2170 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f14227fc640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1422ffd640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 0x55d3315f58e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55d331600430 tx=0x7f141401d440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=622648a6763f9ee7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1406ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1414005510 con 0x55d3315e2170 2024-06-06T07:17:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.219+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f141401d820 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141c0054d0 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f141401da40 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f142bf81740 10 client.26566 fetch_fsmap learned FSMap version 61 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d3316792b0 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f141c0054d0 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.220+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) ==== 8406+0+0 (secure 0 0 0) 0x7f1414032430 con 0x55d3315e2170 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 10 client.26566 fetch_fsmap finished waiting for FSMap version 61 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 10 client.26566 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 20 client.26566 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3316797a0 0x55d33167bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 4 client.26566 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d331553c20 con 0x55d3316797a0 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f14227fc640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3316797a0 0x55d33167bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f1406ff5640 10 client.26566 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.221+0000 7f14227fc640 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3316797a0 0x55d33167bbc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da0c59ad597924ce server_cookie=efc0187f1a13d9c5 in_seq=0 out_seq=0 2024-06-06T07:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f1406ff5640 1 -- 172.21.15.73:0/1710897408 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d331553c20 con 0x55d3316797a0 2024-06-06T07:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f1406ff5640 10 client.26566 handle_command_reply: tid=0 2024-06-06T07:17:14.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 client.26566 shutdown 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 msgr2=0x55d3315f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 0x55d3315f58e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55d331600430 tx=0x7f141401d440 comp rx=0 tx=0).stop 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 shutdown_connections 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3316797a0 0x55d33167bbc0 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3315ed350 0x55d3315f1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3315e2a90 0x55d3315f0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 --2- 172.21.15.73:0/1710897408 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3315e2170 0x55d3315f58e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f1406ff5640 0 client.26566 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.286+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 >> 172.21.15.73:0/1710897408 conn(0x55d3315de680 msgr2=0x55d3315ec450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 shutdown_connections 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142bf81740 1 -- 172.21.15.73:0/1710897408 wait complete. 2024-06-06T07:17:14.288 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142bf81740 20 client.26566 trim_cache size 0 max 16384 2024-06-06T07:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:17:14.289 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 msgr2=0x7f140c0a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 0x7f140c0a53c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f141400bbe0 tx=0x7f141400b390 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 msgr2=0x7f14241a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f141802a9d0 tx=0x7f1418003c10 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 shutdown_connections 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1424105890 0x7f142410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14241019d0 0x7f14241a4990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f140c0a2f00 0x7f140c0a53c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 --2- 172.21.15.73:0/1862424637 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1424101030 0x7f14241a4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.287+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 >> 172.21.15.73:0/1862424637 conn(0x7f1424076e30 msgr2=0x7f14241034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:14.290 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.288+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 shutdown_connections 2024-06-06T07:17:14.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:14.288+0000 7f142927b640 1 -- 172.21.15.73:0/1862424637 wait complete. 2024-06-06T07:17:14.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:14 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:17:14.301 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 podman[73777]: 2024-06-06 07:17:13.837245395 +0000 UTC m=+0.408879868 container init 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T07:17:14.301 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 podman[73777]: 2024-06-06 07:17:13.842797326 +0000 UTC m=+0.414431782 container start 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T07:17:14.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 ceph-mds[73797]: -- 172.21.15.136:0/1652894885 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557adef33a40 con 0x557adef06800 2024-06-06T07:17:14.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: starting mds.e at 2024-06-06T07:17:14.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 bash[73777]: 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 2024-06-06T07:17:14.302 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:17:13 smithi136 systemd[1]: Started Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T07:17:14.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:14 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:17:14.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:14 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:17:14.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23713 9.47 MB/sec execute 19 sec latency 90.089 ms 2024-06-06T07:17:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: pgmap v503: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-06-06T07:17:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T07:17:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/2817666505,v1:172.21.15.136:6835/2817666505] up:boot 2024-06-06T07:17:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:17:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:17:15.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:15 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:15.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: pgmap v503: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-06-06T07:17:15.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T07:17:15.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/2817666505,v1:172.21.15.136:6835/2817666505] up:boot 2024-06-06T07:17:15.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:17:15.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:17:15.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:15 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:15.424 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: pgmap v503: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.4 MiB/s wr, 321 op/s 2024-06-06T07:17:15.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: osdmap e126: 12 total, 12 up, 12 in 2024-06-06T07:17:15.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/2817666505,v1:172.21.15.136:6835/2817666505] up:boot 2024-06-06T07:17:15.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:17:15.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:17:15.425 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:15 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:15.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 23874 9.33 MB/sec execute 20 sec latency 63.971 ms 2024-06-06T07:17:16.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24050 9.28 MB/sec execute 21 sec latency 108.995 ms 2024-06-06T07:17:17.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:17 smithi136 ceph-mon[35901]: pgmap v505: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-06-06T07:17:17.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:17 smithi170 ceph-mon[36108]: pgmap v505: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-06-06T07:17:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:17 smithi073 ceph-mon[32303]: pgmap v505: 129 pgs: 129 active+clean; 549 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 198 op/s 2024-06-06T07:17:17.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24205 9.11 MB/sec execute 22 sec latency 110.779 ms 2024-06-06T07:17:18.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24357 9.03 MB/sec execute 23 sec latency 61.171 ms 2024-06-06T07:17:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:19 smithi136 ceph-mon[35901]: pgmap v506: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:19.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:19 smithi170 ceph-mon[36108]: pgmap v506: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:19 smithi073 ceph-mon[32303]: pgmap v506: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:19.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24512 8.89 MB/sec execute 24 sec latency 105.308 ms 2024-06-06T07:17:20.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24690 9.16 MB/sec execute 25 sec latency 56.314 ms 2024-06-06T07:17:21.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:21 smithi136 ceph-mon[35901]: pgmap v507: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:21 smithi170 ceph-mon[36108]: pgmap v507: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:21 smithi073 ceph-mon[32303]: pgmap v507: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.6 MiB/s wr, 433 op/s 2024-06-06T07:17:21.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 24837 9.06 MB/sec execute 26 sec latency 71.457 ms 2024-06-06T07:17:22.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25009 9.13 MB/sec execute 27 sec latency 66.697 ms 2024-06-06T07:17:23.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:23 smithi136 ceph-mon[35901]: pgmap v508: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-06-06T07:17:23.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:23 smithi170 ceph-mon[36108]: pgmap v508: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-06-06T07:17:23.527 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:23 smithi073 ceph-mon[32303]: pgmap v508: 129 pgs: 129 active+clean; 560 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.5 MiB/s wr, 371 op/s 2024-06-06T07:17:23.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25177 9.16 MB/sec execute 28 sec latency 78.703 ms 2024-06-06T07:17:24.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25330 9.04 MB/sec execute 29 sec latency 164.455 ms 2024-06-06T07:17:25.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:25 smithi136 ceph-mon[35901]: pgmap v509: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-06-06T07:17:25.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:25 smithi170 ceph-mon[36108]: pgmap v509: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-06-06T07:17:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:25 smithi073 ceph-mon[32303]: pgmap v509: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 372 op/s 2024-06-06T07:17:25.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25479 8.82 MB/sec execute 30 sec latency 79.198 ms 2024-06-06T07:17:26.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25634 8.65 MB/sec execute 31 sec latency 63.171 ms 2024-06-06T07:17:27.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:27 smithi136 ceph-mon[35901]: pgmap v510: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 312 op/s 2024-06-06T07:17:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:27 smithi170 ceph-mon[36108]: pgmap v510: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 312 op/s 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 -- 172.21.15.73:0/3952517528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9840102bb0 msgr2=0x7f9840102f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 --2- 172.21.15.73:0/3952517528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9840102bb0 0x7f9840102f90 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f9834009900 tx=0x7f983402a480 comp rx=0 tx=0).stop 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 -- 172.21.15.73:0/3952517528 shutdown_connections 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 --2- 172.21.15.73:0/3952517528 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f984010bf80 0x7f984010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 --2- 172.21.15.73:0/3952517528 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 0x7f984010b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 --2- 172.21.15.73:0/3952517528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9840102bb0 0x7f9840102f90 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 -- 172.21.15.73:0/3952517528 >> 172.21.15.73:0/3952517528 conn(0x7f98400fc370 msgr2=0x7f98400fe790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.419+0000 7f9845d1b640 1 -- 172.21.15.73:0/3952517528 shutdown_connections 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.420+0000 7f9845d1b640 1 -- 172.21.15.73:0/3952517528 wait complete. 2024-06-06T07:17:27.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.420+0000 7f9845d1b640 1 Processor -- start 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 -- start start 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 0x7f9840100d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f984010bf80 0x7f9840101280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9840111c70 con 0x7f984010bf80 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9840111af0 con 0x7f98401034d0 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f9845d1b640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9840111df0 con 0x7f9840102bb0 2024-06-06T07:17:27.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f983f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f983f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55706/0 (socket says 172.21.15.73:55706) 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f983f7fe640 1 -- 172.21.15.73:0/292639981 learned_addr learned my addr 172.21.15.73:0/292639981 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f983ffff640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f984010bf80 0x7f9840101280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.421+0000 7f983effd640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 0x7f9840100d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 -- 172.21.15.73:0/292639981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 msgr2=0x7f9840100d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 0x7f9840100d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 -- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f984010bf80 msgr2=0x7f9840101280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f984010bf80 0x7f9840101280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 -- 172.21.15.73:0/292639981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98401a6630 con 0x7f9840102bb0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983f7fe640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9834005940 tx=0x7f98340042f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983cff9640 1 -- 172.21.15.73:0/292639981 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983403f030 con 0x7f9840102bb0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983cff9640 1 -- 172.21.15.73:0/292639981 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9834003cb0 con 0x7f9840102bb0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f983cff9640 1 -- 172.21.15.73:0/292639981 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9834035070 con 0x7f9840102bb0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.422+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98401a68c0 con 0x7f9840102bb0 2024-06-06T07:17:27.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.423+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a6ca0 con 0x7f9840102bb0 2024-06-06T07:17:27.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 Processor -- start 2024-06-06T07:17:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 -- start start 2024-06-06T07:17:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f983cff9640 1 -- 172.21.15.73:0/292639981 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9834042030 con 0x7f9840102bb0 2024-06-06T07:17:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72a90 0x55d268f7ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 0x55d268f7f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d268ed64f0 con 0x55d268f7d350 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d268e74870 con 0x55d268f72a90 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f9848a2b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d268ed24b0 con 0x55d268f72170 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f983ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f983ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55712/0 (socket says 172.21.15.73:55712) 2024-06-06T07:17:27.438 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.426+0000 7f983ffff640 1 -- 172.21.15.73:0/3433552684 learned_addr learned my addr 172.21.15.73:0/3433552684 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983effd640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 0x55d268f7f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983f7fe640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72a90 0x55d268f7ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 -- 172.21.15.73:0/3433552684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72a90 msgr2=0x55d268f7ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72a90 0x55d268f7ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 -- 172.21.15.73:0/3433552684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 msgr2=0x55d268f7f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.439 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 0x55d268f7f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 -- 172.21.15.73:0/3433552684 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d268e909b0 con 0x55d268f72170 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983effd640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 0x55d268f7f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983ffff640 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9838009fd0 tx=0x7f98380118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3887d173d7afb3c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f983cff9640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 0x7f98280a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.427+0000 7f9825ffb640 1 -- 172.21.15.73:0/3433552684 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838004d20 con 0x55d268f72170 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.428+0000 7f983cff9640 1 -- 172.21.15.73:0/292639981 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f98340efb40 con 0x7f9840102bb0 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.428+0000 7f9825ffb640 1 -- 172.21.15.73:0/3433552684 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9838004050 con 0x55d268f72170 2024-06-06T07:17:27.440 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.428+0000 7f983f7fe640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 0x7f98280a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.428+0000 7f9825ffb640 1 -- 172.21.15.73:0/3433552684 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838005280 con 0x55d268f72170 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- 172.21.15.73:0/3433552684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 msgr2=0x55d268f72550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f983f7fe640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 0x7f98280a5310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98340ecf90 tx=0x7f98340f3690 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9838009fd0 tx=0x7f98380118d0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- 172.21.15.73:0/3433552684 shutdown_connections 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f7d350 0x55d268f7f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72a90 0x55d268f7ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3433552684 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f72550 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- 172.21.15.73:0/3433552684 >> 172.21.15.73:0/3433552684 conn(0x55d268f6e680 msgr2=0x55d268f6ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- 172.21.15.73:0/3433552684 shutdown_connections 2024-06-06T07:17:27.441 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- 172.21.15.73:0/3433552684 wait complete. 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 -- start start 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.429+0000 7f9848a2b740 1 Processor -- start 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 0x55d268f7a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268f7ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d268f7ff80 con 0x55d268f72170 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d268e5fae0 con 0x55d268f7d350 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d268f80100 con 0x55d268f72a90 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f9848a2b740 10 client.0 _ll_register_callbacks cb 0x55d268e989b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55726/0 (socket says 172.21.15.73:55726) 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 -- 172.21.15.73:0/3722365008 learned_addr learned my addr 172.21.15.73:0/3722365008 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983ffff640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 0x55d268f7a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983ffff640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268f7ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 -- 172.21.15.73:0/3722365008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 msgr2=0x55d268f7ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268f7ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 -- 172.21.15.73:0/3722365008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 msgr2=0x55d268f7a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 0x55d268f7a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983effd640 1 -- 172.21.15.73:0/3722365008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d268bea210 con 0x55d268f72a90 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.430+0000 7f983ffff640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 0x55d268f7a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:17:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f983effd640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9830011570 tx=0x7f9830011a30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26bb7faaa487d1d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f981effd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830013670 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f9830013810 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9838004050 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9830026050 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f9848a2b740 10 client.26578 fetch_fsmap learned FSMap version 61 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d268cb6620 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.431+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9830014440 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 61) ==== 8406+0+0 (secure 0 0 0) 0x7f983001d400 con 0x55d268f72a90 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 10 client.26578 fetch_fsmap finished waiting for FSMap version 61 2024-06-06T07:17:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 10 client.26578 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 20 client.26578 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268f8fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 4 client.26578 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9838004050 con 0x55d268f8d580 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.432+0000 7f983ffff640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268f8fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.433+0000 7f983ffff640 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268f8fa20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=650b5d93862d3707 server_cookie=b62d691365e82232 in_seq=0 out_seq=0 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.433+0000 7f981effd640 10 client.26578 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.434+0000 7f981effd640 1 -- 172.21.15.73:0/3722365008 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f9838004050 con 0x55d268f8d580 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.434+0000 7f981effd640 10 client.26578 handle_command_reply: tid=0 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.434+0000 7f9848a2b740 1 client.26578 shutdown 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.434+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 msgr2=0x55d268f7a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.434+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9830011570 tx=0x7f9830011a30 comp rx=0 tx=0).stop 2024-06-06T07:17:27.445 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 shutdown_connections 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268f8fa20 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f981effd640 0 client.26578 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268f7ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72a90 0x55d268f7a7c0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 --2- 172.21.15.73:0/3722365008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72170 0x55d268f7a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 >> 172.21.15.73:0/3722365008 conn(0x55d268f6e680 msgr2=0x55d268f743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 shutdown_connections 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 1 -- 172.21.15.73:0/3722365008 wait complete. 2024-06-06T07:17:27.446 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.435+0000 7f9848a2b740 20 client.26578 trim_cache size 0 max 16384 2024-06-06T07:17:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.460+0000 7f9848a2b740 1 Processor -- start 2024-06-06T07:17:27.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 -- start start 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72170 0x55d268f79b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 0x55d268f95340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98401b0f10 con 0x55d268f72a90 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98401b2c00 con 0x55d268f72170 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f9848a2b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98401b16a0 con 0x55d268f7d350 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40302/0 (socket says 172.21.15.73:40302) 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 -- 172.21.15.73:0/1329040968 learned_addr learned my addr 172.21.15.73:0/1329040968 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983ffff640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72170 0x55d268f79b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983ffff640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 0x55d268f95340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 -- 172.21.15.73:0/1329040968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 msgr2=0x55d268f95340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 0x55d268f95340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 -- 172.21.15.73:0/1329040968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72170 msgr2=0x55d268f79b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72170 0x55d268f79b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.461+0000 7f983effd640 1 -- 172.21.15.73:0/1329040968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d268e790d0 con 0x55d268f72a90 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.462+0000 7f983ffff640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 0x55d268f95340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.462+0000 7f983effd640 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f983000f250 tx=0x7f98300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35b3dfe6119224de server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.462+0000 7f98257fa640 1 -- 172.21.15.73:0/1329040968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983002e020 con 0x55d268f72a90 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.462+0000 7f98257fa640 1 -- 172.21.15.73:0/1329040968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f983001d400 con 0x55d268f72a90 2024-06-06T07:17:27.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.462+0000 7f98257fa640 1 -- 172.21.15.73:0/1329040968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98300146a0 con 0x55d268f72a90 2024-06-06T07:17:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- 172.21.15.73:0/1329040968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 msgr2=0x55d268f7c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f983000f250 tx=0x7f98300046c0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- 172.21.15.73:0/1329040968 shutdown_connections 2024-06-06T07:17:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f7d350 0x55d268f95340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f7c340 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 --2- 172.21.15.73:0/1329040968 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f72170 0x55d268f79b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- 172.21.15.73:0/1329040968 >> 172.21.15.73:0/1329040968 conn(0x55d268f6e680 msgr2=0x55d268f6ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- 172.21.15.73:0/1329040968 shutdown_connections 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- 172.21.15.73:0/1329040968 wait complete. 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.463+0000 7f9848a2b740 1 -- start start 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 Processor -- start 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f80930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268ff6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98401b2530 con 0x55d268f72a90 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98401b1da0 con 0x55d268f7d350 2024-06-06T07:17:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98401aeef0 con 0x55d268f72170 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f9848a2b740 10 client.0 _ll_register_callbacks cb 0x7f98300095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f983ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f80930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f983effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f983effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40306/0 (socket says 172.21.15.73:40306) 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f983effd640 1 -- 172.21.15.73:0/649413903 learned_addr learned my addr 172.21.15.73:0/649413903 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.464+0000 7f983ffff640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268ff6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 -- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 msgr2=0x55d268f80930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f80930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 -- 172.21.15.73:0/649413903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 msgr2=0x55d268ff6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268ff6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 -- 172.21.15.73:0/649413903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d268ff6840 con 0x55d268f72a90 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983ffff640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268ff6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:17:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f983effd640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9830001eb0 tx=0x7f983000f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05ddba4028a3219 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f981effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983002e030 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f9830037c70 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983001d400 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.465+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f9830048050 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 10 client.17166 fetch_fsmap learned FSMap version 61 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=4+}) -- 0x55d268ff7a50 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9830013b20 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 61) ==== 8406+0+0 (secure 0 0 0) 0x7f9830014b50 con 0x55d268f72a90 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 10 client.17166 fetch_fsmap finished waiting for FSMap version 61 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:27.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 20 client.17166 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268ffa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 4 client.17166 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f983001d400 con 0x55d268f8d580 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.466+0000 7f983ffff640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268ffa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.467+0000 7f981effd640 10 client.17166 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:27.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.467+0000 7f983ffff640 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268ffa1f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62da405a4208214 server_cookie=c9a33eced2573282 in_seq=0 out_seq=0 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f981effd640 1 -- 172.21.15.73:0/649413903 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f983001d400 con 0x55d268f8d580 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f981effd640 10 client.17166 handle_command_reply: tid=0 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 client.17166 shutdown 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 msgr2=0x55d268f80e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f9830001eb0 tx=0x7f983000f040 comp rx=0 tx=0).stop 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 shutdown_connections 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d268f8d580 0x55d268ffa1f0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d268f7d350 0x55d268ff6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d268f72a90 0x55d268f80e70 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f981effd640 0 client.17166 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 --2- 172.21.15.73:0/649413903 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d268f72170 0x55d268f80930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 >> 172.21.15.73:0/649413903 conn(0x55d268f6e680 msgr2=0x55d268f743b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 shutdown_connections 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 1 -- 172.21.15.73:0/649413903 wait complete. 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.517+0000 7f9848a2b740 20 client.17166 trim_cache size 0 max 16384 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:17:27.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 msgr2=0x7f98280a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 0x7f98280a5310 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f98340ecf90 tx=0x7f98340f3690 comp rx=0 tx=0).stop 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 msgr2=0x7f9840100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9834005940 tx=0x7f98340042f0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 shutdown_connections 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f984010bf80 0x7f9840101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f98401034d0 0x7f9840100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98280a2e50 0x7f98280a5310 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 --2- 172.21.15.73:0/292639981 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9840102bb0 0x7f9840100800 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 >> 172.21.15.73:0/292639981 conn(0x7f98400fc370 msgr2=0x7f98400fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 shutdown_connections 2024-06-06T07:17:27.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:27.518+0000 7f9845d1b640 1 -- 172.21.15.73:0/292639981 wait complete. 2024-06-06T07:17:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:27 smithi073 ceph-mon[32303]: pgmap v510: 129 pgs: 129 active+clean; 559 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 312 op/s 2024-06-06T07:17:27.803 INFO:tasks.workunit.client.0.smithi073.stdout: 10 25815 8.71 MB/sec execute 32 sec latency 66.124 ms 2024-06-06T07:17:28.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26009 8.76 MB/sec execute 33 sec latency 70.455 ms 2024-06-06T07:17:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:29 smithi170 ceph-mon[36108]: pgmap v511: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 446 op/s 2024-06-06T07:17:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:29 smithi073 ceph-mon[32303]: pgmap v511: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 446 op/s 2024-06-06T07:17:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:29 smithi136 ceph-mon[35901]: pgmap v511: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.2 MiB/s wr, 446 op/s 2024-06-06T07:17:29.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26181 8.82 MB/sec execute 34 sec latency 80.852 ms 2024-06-06T07:17:30.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26364 8.91 MB/sec execute 35 sec latency 60.443 ms 2024-06-06T07:17:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:31 smithi170 ceph-mon[36108]: pgmap v512: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-06T07:17:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:31 smithi073 ceph-mon[32303]: pgmap v512: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-06T07:17:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:31 smithi136 ceph-mon[35901]: pgmap v512: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-06T07:17:31.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26572 9.10 MB/sec execute 36 sec latency 58.363 ms 2024-06-06T07:17:32.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26751 9.11 MB/sec execute 37 sec latency 71.189 ms 2024-06-06T07:17:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:33 smithi170 ceph-mon[36108]: pgmap v513: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T07:17:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:33 smithi073 ceph-mon[32303]: pgmap v513: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T07:17:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:33 smithi136 ceph-mon[35901]: pgmap v513: 129 pgs: 129 active+clean; 562 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.7 MiB/s wr, 290 op/s 2024-06-06T07:17:33.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 26964 9.16 MB/sec execute 38 sec latency 66.692 ms 2024-06-06T07:17:34.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27108 9.11 MB/sec execute 39 sec latency 64.552 ms 2024-06-06T07:17:35.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:35 smithi170 ceph-mon[36108]: pgmap v514: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 418 op/s 2024-06-06T07:17:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:35 smithi073 ceph-mon[32303]: pgmap v514: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 418 op/s 2024-06-06T07:17:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:35 smithi136 ceph-mon[35901]: pgmap v514: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.3 MiB/s wr, 418 op/s 2024-06-06T07:17:35.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27281 9.20 MB/sec execute 40 sec latency 108.633 ms 2024-06-06T07:17:36.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27475 9.26 MB/sec execute 41 sec latency 51.678 ms 2024-06-06T07:17:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:37 smithi170 ceph-mon[36108]: pgmap v515: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-06-06T07:17:37.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:37 smithi136 ceph-mon[35901]: pgmap v515: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-06-06T07:17:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:37 smithi073 ceph-mon[32303]: pgmap v515: 129 pgs: 129 active+clean; 575 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.2 MiB/s wr, 264 op/s 2024-06-06T07:17:37.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27647 9.29 MB/sec execute 42 sec latency 59.792 ms 2024-06-06T07:17:38.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27807 9.24 MB/sec execute 43 sec latency 51.231 ms 2024-06-06T07:17:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:39 smithi170 ceph-mon[36108]: pgmap v516: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-06-06T07:17:39.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:39 smithi073 ceph-mon[32303]: pgmap v516: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-06-06T07:17:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:39 smithi136 ceph-mon[35901]: pgmap v516: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.7 MiB/s wr, 396 op/s 2024-06-06T07:17:39.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 27966 9.23 MB/sec execute 44 sec latency 68.816 ms 2024-06-06T07:17:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 -- 172.21.15.73:0/660156653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00641069c0 msgr2=0x7f0064106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 --2- 172.21.15.73:0/660156653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00641069c0 0x7f0064106da0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f0054009930 tx=0x7f005402a420 comp rx=0 tx=0).stop 2024-06-06T07:17:40.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 -- 172.21.15.73:0/660156653 shutdown_connections 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 --2- 172.21.15.73:0/660156653 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064101540 0x7f006410d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 --2- 172.21.15.73:0/660156653 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0064100b80 0x7f0064101000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 --2- 172.21.15.73:0/660156653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00641069c0 0x7f0064106da0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 -- 172.21.15.73:0/660156653 >> 172.21.15.73:0/660156653 conn(0x7f00640775f0 msgr2=0x7f0064077a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 -- 172.21.15.73:0/660156653 shutdown_connections 2024-06-06T07:17:40.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.652+0000 7f006a7dc640 1 -- 172.21.15.73:0/660156653 wait complete. 2024-06-06T07:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.653+0000 7f006a7dc640 1 Processor -- start 2024-06-06T07:17:40.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.653+0000 7f006a7dc640 1 -- start start 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064100b80 0x7f0064107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 0x7f0064107c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 0x7f0064108190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0064111c90 con 0x7f0064101540 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0064111b10 con 0x7f00641069c0 2024-06-06T07:17:40.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f006a7dc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0064111e10 con 0x7f0064100b80 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 0x7f0064107c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 0x7f0064107c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42362/0 (socket says 172.21.15.73:42362) 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 0x7f0064108190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f00637fe640 1 -- 172.21.15.73:0/4247828111 learned_addr learned my addr 172.21.15.73:0/4247828111 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0063fff640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064100b80 0x7f0064107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 -- 172.21.15.73:0/4247828111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064100b80 msgr2=0x7f0064107710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064100b80 0x7f0064107710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 -- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 msgr2=0x7f0064107c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 0x7f0064107c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.654+0000 7f0068d79640 1 -- 172.21.15.73:0/4247828111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00641aa8f0 con 0x7f00641069c0 2024-06-06T07:17:40.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f0068d79640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 0x7f0064108190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f005c00ec80 tx=0x7f005c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f00617fa640 1 -- 172.21.15.73:0/4247828111 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c003b70 con 0x7f00641069c0 2024-06-06T07:17:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f00617fa640 1 -- 172.21.15.73:0/4247828111 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f005c00ee90 con 0x7f00641069c0 2024-06-06T07:17:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f00617fa640 1 -- 172.21.15.73:0/4247828111 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0107a0 con 0x7f00641069c0 2024-06-06T07:17:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00641aabe0 con 0x7f00641069c0 2024-06-06T07:17:40.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.655+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00641ab0e0 con 0x7f00641069c0 2024-06-06T07:17:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.658+0000 7f00617fa640 1 -- 172.21.15.73:0/4247828111 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f005c004020 con 0x7f00641069c0 2024-06-06T07:17:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 Processor -- start 2024-06-06T07:17:40.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 -- start start 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc17550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc24740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c54fb7b4f0 con 0x55c54fc17170 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c54fb19870 con 0x55c54fc17a90 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f006d5c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c54fb774b0 con 0x55c54fc22350 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc17550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.659+0000 7f0063fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:39068/0 (socket says 172.21.15.73:39068) 2024-06-06T07:17:40.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 -- 172.21.15.73:0/2625129229 learned_addr learned my addr 172.21.15.73:0/2625129229 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f00637fe640 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc24740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 -- 172.21.15.73:0/2625129229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 msgr2=0x55c54fc24740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc24740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 -- 172.21.15.73:0/2625129229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 msgr2=0x55c54fc17550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc17550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 -- 172.21.15.73:0/2625129229 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54fb359b0 con 0x55c54fc17a90 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f00637fe640 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.660+0000 7f0063fff640 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f0054004140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ded3e196d5d5cff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.661+0000 7f004a7fc640 1 -- 172.21.15.73:0/2625129229 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054035030 con 0x55c54fc17a90 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.661+0000 7f004a7fc640 1 -- 172.21.15.73:0/2625129229 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00540044b0 con 0x55c54fc17a90 2024-06-06T07:17:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.661+0000 7f004a7fc640 1 -- 172.21.15.73:0/2625129229 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0054004e20 con 0x55c54fc17a90 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f00617fa640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 0x7f004c0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 -- 172.21.15.73:0/2625129229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 msgr2=0x55c54fc21e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0054000c00 tx=0x7f0054004140 comp rx=0 tx=0).stop 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 -- 172.21.15.73:0/2625129229 shutdown_connections 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc24740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc21e10 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 --2- 172.21.15.73:0/2625129229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc17550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 -- 172.21.15.73:0/2625129229 >> 172.21.15.73:0/2625129229 conn(0x55c54fc13680 msgr2=0x55c54fc13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f00637fe640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 0x7f004c0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 -- 172.21.15.73:0/2625129229 shutdown_connections 2024-06-06T07:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f006d5c7740 1 -- 172.21.15.73:0/2625129229 wait complete. 2024-06-06T07:17:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.662+0000 7f00617fa640 1 -- 172.21.15.73:0/4247828111 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f005c0cfe70 con 0x7f00641069c0 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 -- start start 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 Processor -- start 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f00637fe640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 0x7f004c0a52b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f00580097f0 tx=0x7f0058009530 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc17170 0x55c54fc1f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 0x55c54fc1fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55c54fc24f20 con 0x55c54fc17a90 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55c54fb04ae0 con 0x55c54fc22350 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f006d5c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55c54fc250a0 con 0x55c54fc17170 2024-06-06T07:17:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.663+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42370/0 (socket says 172.21.15.73:42370) 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f00637fe640 1 -- 172.21.15.73:0/3420741552 learned_addr learned my addr 172.21.15.73:0/3420741552 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 0x55c54fc1fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0063fff640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc17170 0x55c54fc1f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f006d5c7740 10 client.0 _ll_register_callbacks cb 0x55c54fb3d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 -- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc17170 msgr2=0x55c54fc1f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc17170 0x55c54fc1f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 -- 172.21.15.73:0/3420741552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 msgr2=0x55c54fc1f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 -- 172.21.15.73:0/3420741552 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54f88f210 con 0x55c54fc22350 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f0068d79640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 0x55c54fc1fd30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f005c0cd210 tx=0x7f005c09bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf1a9437578a0fc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f00437fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c0b4020 con 0x55c54fc22350 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00540044b0 con 0x55c54fc22350 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.664+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f005c098650 con 0x55c54fc22350 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f005c09c6e0 con 0x55c54fc22350 2024-06-06T07:17:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 10 client.26417 fetch_fsmap learned FSMap version 62 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c54f95b620 con 0x55c54fc22350 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00540044b0 con 0x55c54fc22350 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 62) ==== 8402+0+0 (secure 0 0 0) 0x7f005c09b4c0 con 0x55c54fc22350 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 10 client.26417 fetch_fsmap finished waiting for FSMap version 62 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 10 client.26417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 20 client.26417 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc32480 0x55c54fc348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 4 client.26417 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:17:40.672 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c54fb8ef10 con 0x55c54fc32480 2024-06-06T07:17:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.665+0000 7f0063fff640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc32480 0x55c54fc348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.666+0000 7f00437fe640 10 client.26417 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.666+0000 7f0063fff640 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc32480 0x55c54fc348d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67feb7efe7c2ae4b server_cookie=ab22a20f30f34211 in_seq=0 out_seq=0 2024-06-06T07:17:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f00437fe640 1 -- 172.21.15.73:0/3420741552 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c54fb8ef10 con 0x55c54fc32480 2024-06-06T07:17:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f00437fe640 10 client.26417 handle_command_reply: tid=0 2024-06-06T07:17:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 client.26417 shutdown 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 msgr2=0x55c54fc1fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 0x55c54fc1fd30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f005c0cd210 tx=0x7f005c09bea0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 shutdown_connections 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc32480 0x55c54fc348d0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc22350 0x55c54fc1fd30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f00437fe640 0 client.26417 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3420741552 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc17170 0x55c54fc1f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 >> 172.21.15.73:0/3420741552 conn(0x55c54fc13680 msgr2=0x55c54fc19400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 shutdown_connections 2024-06-06T07:17:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 1 -- 172.21.15.73:0/3420741552 wait complete. 2024-06-06T07:17:40.676 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.667+0000 7f006d5c7740 20 client.26417 trim_cache size 0 max 16384 2024-06-06T07:17:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 Processor -- start 2024-06-06T07:17:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 -- start start 2024-06-06T07:17:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc18b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b5290 con 0x55c54fc17170 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b6f80 con 0x55c54fc17a90 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.695+0000 7f006d5c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b5a20 con 0x55c54fc22350 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f0063fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f0063fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42374/0 (socket says 172.21.15.73:42374) 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f0063fff640 1 -- 172.21.15.73:0/1567817207 learned_addr learned my addr 172.21.15.73:0/1567817207 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f0068d79640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc18b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 -- 172.21.15.73:0/1567817207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 msgr2=0x55c54fc18b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc18b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 -- 172.21.15.73:0/1567817207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 msgr2=0x55c54fc3a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 -- 172.21.15.73:0/1567817207 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54fc0ef70 con 0x55c54fc17a90 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f0063fff640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.696+0000 7f00637fe640 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc3a720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55c54fc20880 tx=0x7f0058018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb921f55fe62835 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.697+0000 7f0049ffb640 1 -- 172.21.15.73:0/1567817207 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058005020 con 0x55c54fc17a90 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.697+0000 7f0049ffb640 1 -- 172.21.15.73:0/1567817207 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x55c54fc17a90 2024-06-06T07:17:40.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.697+0000 7f0049ffb640 1 -- 172.21.15.73:0/1567817207 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005802b670 con 0x55c54fc17a90 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- 172.21.15.73:0/1567817207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 msgr2=0x55c54fc3a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc3a720 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55c54fc20880 tx=0x7f0058018000 comp rx=0 tx=0).stop 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- 172.21.15.73:0/1567817207 shutdown_connections 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc18b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17a90 0x55c54fc3a720 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 --2- 172.21.15.73:0/1567817207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17170 0x55c54fc3a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- 172.21.15.73:0/1567817207 >> 172.21.15.73:0/1567817207 conn(0x55c54fc13680 msgr2=0x55c54fc13a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- 172.21.15.73:0/1567817207 shutdown_connections 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- 172.21.15.73:0/1567817207 wait complete. 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 -- start start 2024-06-06T07:17:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.698+0000 7f006d5c7740 1 Processor -- start 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 0x55c54fc2aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc1c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b68b0 con 0x55c54fc17a90 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b6120 con 0x55c54fc17170 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f00641b3270 con 0x55c54fc22350 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f006d5c7740 10 client.0 _ll_register_callbacks cb 0x7f005c004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42382/0 (socket says 172.21.15.73:42382) 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 -- 172.21.15.73:0/3326269318 learned_addr learned my addr 172.21.15.73:0/3326269318 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f0068d79640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc1c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f0063fff640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 0x55c54fc2aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 -- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 msgr2=0x55c54fc1c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc1c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 -- 172.21.15.73:0/3326269318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 msgr2=0x55c54fc2aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 0x55c54fc2aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f00637fe640 1 -- 172.21.15.73:0/3326269318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54fc1cdb0 con 0x55c54fc17a90 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f0063fff640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 0x55c54fc2aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.699+0000 7f0068d79640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc1c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00637fe640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f005802d040 tx=0x7f0058004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f5fdf6dc1330b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0058039020 con 0x55c54fc17a90 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f0058005340 con 0x55c54fc17a90 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00580047a0 con 0x55c54fc17a90 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) ==== 24+0+0 (secure 0 0 0) 0x7f0058044050 con 0x55c54fc17a90 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f006d5c7740 10 client.17184 fetch_fsmap learned FSMap version 62 2024-06-06T07:17:40.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) -- 0x55c54fc9d1f0 con 0x55c54fc17a90 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f00580047a0 con 0x55c54fc17a90 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 62) ==== 8402+0+0 (secure 0 0 0) 0x7f005802a070 con 0x55c54fc17a90 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f006d5c7740 10 client.17184 fetch_fsmap finished waiting for FSMap version 62 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.700+0000 7f006d5c7740 10 client.17184 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f006d5c7740 20 client.17184 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc9da20 0x55c54fc9fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f006d5c7740 4 client.17184 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c54fb88c20 con 0x55c54fc9da20 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f0063fff640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc9da20 0x55c54fc9fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.701+0000 7f00437fe640 10 client.17184 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.702+0000 7f0063fff640 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc9da20 0x55c54fc9fe40 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5abeb413d147ad76 server_cookie=d641c15594f40f7a in_seq=0 out_seq=0 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.761+0000 7f00437fe640 1 -- 172.21.15.73:0/3326269318 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c54fb88c20 con 0x55c54fc9da20 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.761+0000 7f00437fe640 10 client.17184 handle_command_reply: tid=0 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.761+0000 7f006d5c7740 1 client.17184 shutdown 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 msgr2=0x55c54fc1c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f005802d040 tx=0x7f0058004da0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 shutdown_connections 2024-06-06T07:17:40.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55c54fc9da20 0x55c54fc9fe40 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55c54fc22350 0x55c54fc1c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55c54fc17a90 0x55c54fc1c070 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 --2- 172.21.15.73:0/3326269318 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55c54fc17170 0x55c54fc2aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 >> 172.21.15.73:0/3326269318 conn(0x55c54fc13680 msgr2=0x55c54fc18b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f00437fe640 0 client.17184 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 shutdown_connections 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 1 -- 172.21.15.73:0/3326269318 wait complete. 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.762+0000 7f006d5c7740 20 client.17184 trim_cache size 0 max 16384 2024-06-06T07:17:40.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:17:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:17:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:17:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 msgr2=0x7f004c0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 0x7f004c0a52b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f00580097f0 tx=0x7f0058009530 comp rx=0 tx=0).stop 2024-06-06T07:17:40.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 msgr2=0x7f0064108190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 0x7f0064108190 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f005c00ec80 tx=0x7f005c00c6a0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 shutdown_connections 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f00641069c0 0x7f0064108190 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0064101540 0x7f0064107c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f004c0a2df0 0x7f004c0a52b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 --2- 172.21.15.73:0/4247828111 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0064100b80 0x7f0064107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 >> 172.21.15.73:0/4247828111 conn(0x7f00640775f0 msgr2=0x7f006410bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.763+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 shutdown_connections 2024-06-06T07:17:40.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:40.764+0000 7f006a7dc640 1 -- 172.21.15.73:0/4247828111 wait complete. 2024-06-06T07:17:40.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28128 9.23 MB/sec execute 45 sec latency 87.525 ms 2024-06-06T07:17:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:41 smithi170 ceph-mon[36108]: pgmap v517: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:41 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:41 smithi073 ceph-mon[32303]: pgmap v517: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:41 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:41 smithi136 ceph-mon[35901]: pgmap v517: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:41 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:41.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28283 9.13 MB/sec execute 46 sec latency 61.451 ms 2024-06-06T07:17:42.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28439 9.11 MB/sec execute 47 sec latency 58.276 ms 2024-06-06T07:17:43.349 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:43 smithi170 ceph-mon[36108]: pgmap v518: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:43 smithi073 ceph-mon[32303]: pgmap v518: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:43 smithi136 ceph-mon[35901]: pgmap v518: 129 pgs: 129 active+clean; 585 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 260 op/s 2024-06-06T07:17:43.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28592 9.05 MB/sec execute 48 sec latency 132.203 ms 2024-06-06T07:17:44.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28763 9.03 MB/sec execute 49 sec latency 104.183 ms 2024-06-06T07:17:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:45 smithi170 ceph-mon[36108]: pgmap v519: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 421 op/s 2024-06-06T07:17:45.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:45 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:45.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:45 smithi073 ceph-mon[32303]: pgmap v519: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 421 op/s 2024-06-06T07:17:45.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:45 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:45 smithi136 ceph-mon[35901]: pgmap v519: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 421 op/s 2024-06-06T07:17:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:45 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:45.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 28959 9.10 MB/sec execute 50 sec latency 62.885 ms 2024-06-06T07:17:46.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:46 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:46.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:46 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:46.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:46 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:46.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29128 9.07 MB/sec execute 51 sec latency 66.305 ms 2024-06-06T07:17:47.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:47 smithi170 ceph-mon[36108]: pgmap v520: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:17:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:47 smithi073 ceph-mon[32303]: pgmap v520: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:17:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:47 smithi136 ceph-mon[35901]: pgmap v520: 129 pgs: 129 active+clean; 593 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:17:47.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29272 9.08 MB/sec execute 52 sec latency 150.913 ms 2024-06-06T07:17:48.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29426 9.06 MB/sec execute 53 sec latency 125.082 ms 2024-06-06T07:17:49.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:49 smithi170 ceph-mon[36108]: pgmap v521: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 438 op/s 2024-06-06T07:17:49.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:49 smithi073 ceph-mon[32303]: pgmap v521: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 438 op/s 2024-06-06T07:17:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:49 smithi136 ceph-mon[35901]: pgmap v521: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.5 MiB/s wr, 438 op/s 2024-06-06T07:17:49.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29610 9.07 MB/sec execute 54 sec latency 61.837 ms 2024-06-06T07:17:50.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29805 9.06 MB/sec execute 55 sec latency 58.691 ms 2024-06-06T07:17:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:51 smithi170 ceph-mon[36108]: pgmap v522: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:51 smithi073 ceph-mon[32303]: pgmap v522: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:51 smithi136 ceph-mon[35901]: pgmap v522: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:51.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 29977 9.00 MB/sec execute 56 sec latency 61.810 ms 2024-06-06T07:17:52.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30124 8.99 MB/sec execute 57 sec latency 89.532 ms 2024-06-06T07:17:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:53 smithi073 ceph-mon[32303]: pgmap v523: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:53 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:17:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:53 smithi136 ceph-mon[35901]: pgmap v523: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:53 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:17:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:53 smithi170 ceph-mon[36108]: pgmap v523: 129 pgs: 129 active+clean; 591 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-06-06T07:17:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:53 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:17:53.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30259 8.98 MB/sec execute 58 sec latency 163.754 ms 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 -- 172.21.15.73:0/3484841854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a38102360 msgr2=0x7f8a3810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 --2- 172.21.15.73:0/3484841854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a38102360 0x7f8a3810e870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009cd0 tx=0x7f8a3002ed40 comp rx=0 tx=0).stop 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 -- 172.21.15.73:0/3484841854 shutdown_connections 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 --2- 172.21.15.73:0/3484841854 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a38102360 0x7f8a3810e870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 --2- 172.21.15.73:0/3484841854 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a381019a0 0x7f8a38101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.240+0000 7f8a3f942640 1 --2- 172.21.15.73:0/3484841854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a38107d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.241+0000 7f8a3f942640 1 -- 172.21.15.73:0/3484841854 >> 172.21.15.73:0/3484841854 conn(0x7f8a380fd470 msgr2=0x7f8a380ff890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.241+0000 7f8a3f942640 1 -- 172.21.15.73:0/3484841854 shutdown_connections 2024-06-06T07:17:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.241+0000 7f8a3f942640 1 -- 172.21.15.73:0/3484841854 wait complete. 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.242+0000 7f8a3f942640 1 Processor -- start 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.242+0000 7f8a3f942640 1 -- start start 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381019a0 0x7f8a38073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 0x7f8a38073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a3806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810fa80 con 0x7f8a381079b0 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810f900 con 0x7f8a38102360 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3f942640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a3810fc00 con 0x7f8a381019a0 2024-06-06T07:17:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 0x7f8a38073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 0x7f8a38073680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55442/0 (socket says 172.21.15.73:55442) 2024-06-06T07:17:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3cedd640 1 -- 172.21.15.73:0/1898244657 learned_addr learned my addr 172.21.15.73:0/1898244657 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.243+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a3806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381019a0 0x7f8a38073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 -- 172.21.15.73:0/1898244657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381019a0 msgr2=0x7f8a38073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381019a0 0x7f8a38073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 -- 172.21.15.73:0/1898244657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 msgr2=0x7f8a38073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 0x7f8a38073680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 -- 172.21.15.73:0/1898244657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3806cc60 con 0x7f8a381079b0 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.244+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a3806c3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002f260 tx=0x7f8a30035b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.245+0000 7f8a2e7fc640 1 -- 172.21.15.73:0/1898244657 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30045070 con 0x7f8a381079b0 2024-06-06T07:17:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.245+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a3806cef0 con 0x7f8a381079b0 2024-06-06T07:17:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.245+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a38070ef0 con 0x7f8a381079b0 2024-06-06T07:17:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.245+0000 7f8a2e7fc640 1 -- 172.21.15.73:0/1898244657 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a300105a0 con 0x7f8a381079b0 2024-06-06T07:17:54.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.245+0000 7f8a2e7fc640 1 -- 172.21.15.73:0/1898244657 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a30045070 con 0x7f8a381079b0 2024-06-06T07:17:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.250+0000 7f8a2e7fc640 1 -- 172.21.15.73:0/1898244657 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8a30048070 con 0x7f8a381079b0 2024-06-06T07:17:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.252+0000 7f8a2e7fc640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 0x7f8a240a5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.253+0000 7f8a2e7fc640 1 -- 172.21.15.73:0/1898244657 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f8a300f05e0 con 0x7f8a381079b0 2024-06-06T07:17:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.253+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 0x7f8a240a5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.323 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.322+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 0x7f8a240a5560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009890 tx=0x7f8a34008040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 Processor -- start 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 -- start start 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e81d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e827e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e7814f0 con 0x55ea1e81da90 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e71f870 con 0x55ea1e828350 2024-06-06T07:17:54.326 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.324+0000 7f8a42650740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e77d4b0 con 0x55ea1e81d170 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3d6de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e81d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55450/0 (socket says 172.21.15.73:55450) 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 -- 172.21.15.73:0/42599040 learned_addr learned my addr 172.21.15.73:0/42599040 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e827e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 -- 172.21.15.73:0/42599040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 msgr2=0x55ea1e81d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e81d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 -- 172.21.15.73:0/42599040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 msgr2=0x55ea1e827e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e827e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 -- 172.21.15.73:0/42599040 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1e73b9b0 con 0x55ea1e828350 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e81d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e827e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a3003a040 tx=0x7f8a300095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d948c7aa8d396627 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.325+0000 7f8a237fe640 1 -- 172.21.15.73:0/42599040 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300bc160 con 0x55ea1e828350 2024-06-06T07:17:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.326+0000 7f8a237fe640 1 -- 172.21.15.73:0/42599040 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a300bcf00 con 0x55ea1e828350 2024-06-06T07:17:54.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.326+0000 7f8a237fe640 1 -- 172.21.15.73:0/42599040 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a3002f8e0 con 0x55ea1e828350 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- 172.21.15.73:0/42599040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 msgr2=0x55ea1e82a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8a3003a040 tx=0x7f8a300095c0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- 172.21.15.73:0/42599040 shutdown_connections 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e82a740 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e827e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 --2- 172.21.15.73:0/42599040 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e81d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- 172.21.15.73:0/42599040 >> 172.21.15.73:0/42599040 conn(0x55ea1e819680 msgr2=0x55ea1e819a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- 172.21.15.73:0/42599040 shutdown_connections 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- 172.21.15.73:0/42599040 wait complete. 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.327+0000 7f8a42650740 1 -- start start 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 Processor -- start 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e825880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e82ad40 con 0x55ea1e81da90 2024-06-06T07:17:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e70aae0 con 0x55ea1e81d170 2024-06-06T07:17:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ea1e82aec0 con 0x55ea1e828350 2024-06-06T07:17:54.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.328+0000 7f8a42650740 10 client.0 _ll_register_callbacks cb 0x55ea1e7439b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3d6de640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3d6de640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55462/0 (socket says 172.21.15.73:55462) 2024-06-06T07:17:54.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:41162/0 (socket says 172.21.15.73:41162) 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3d6de640 1 -- 172.21.15.73:0/3363759335 learned_addr learned my addr 172.21.15.73:0/3363759335 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 -- 172.21.15.73:0/3363759335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 msgr2=0x55ea1e825340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 -- 172.21.15.73:0/3363759335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 msgr2=0x55ea1e825880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e825880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 -- 172.21.15.73:0/3363759335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1e495210 con 0x55ea1e828350 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ea1e81e080 tx=0x7f8a30009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca980290d9445137 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.329+0000 7f8a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300d4070 con 0x55ea1e828350 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f8a300edba0 con 0x55ea1e828350 2024-06-06T07:17:54.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a300bcf00 con 0x55ea1e828350 2024-06-06T07:17:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8a300bc160 con 0x55ea1e828350 2024-06-06T07:17:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a42650740 10 client.26611 fetch_fsmap learned FSMap version 65 2024-06-06T07:17:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.330+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ea1e561620 con 0x55ea1e828350 2024-06-06T07:17:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a300bcf00 con 0x55ea1e828350 2024-06-06T07:17:54.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 65) ==== 8390+0+0 (secure 0 0 0) 0x7f8a300c4fe0 con 0x55ea1e828350 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 10 client.26611 fetch_fsmap finished waiting for FSMap version 65 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 10 client.26611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 20 client.26611 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8380b0 0x55ea1e83a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 4 client.26611 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.331+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1e794f10 con 0x55ea1e8380b0 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.332+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8380b0 0x55ea1e83a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.333+0000 7f8a20ff9640 10 client.26611 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:54.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.333+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8380b0 0x55ea1e83a4a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f94ddbd8e97254 server_cookie=4d14888349cbd487 in_seq=0 out_seq=0 2024-06-06T07:17:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a20ff9640 1 -- 172.21.15.73:0/3363759335 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea1e794f10 con 0x55ea1e8380b0 2024-06-06T07:17:54.335 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a20ff9640 10 client.26611 handle_command_reply: tid=0 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 client.26611 shutdown 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 msgr2=0x55ea1e825dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x55ea1e81e080 tx=0x7f8a30009780 comp rx=0 tx=0).stop 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 shutdown_connections 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8380b0 0x55ea1e83a4a0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e828350 0x55ea1e825dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e825880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a42650740 1 --2- 172.21.15.73:0/3363759335 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81d170 0x55ea1e825340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.334+0000 7f8a20ff9640 0 client.26611 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.335+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 >> 172.21.15.73:0/3363759335 conn(0x55ea1e819680 msgr2=0x55ea1e81f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.335+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 shutdown_connections 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.335+0000 7f8a42650740 1 -- 172.21.15.73:0/3363759335 wait complete. 2024-06-06T07:17:54.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.335+0000 7f8a42650740 20 client.26611 trim_cache size 0 max 16384 2024-06-06T07:17:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.364+0000 7f8a42650740 1 Processor -- start 2024-06-06T07:17:54.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 -- start start 2024-06-06T07:17:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e8149c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 0x55ea1e81faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381bdbf0 con 0x55ea1e828350 2024-06-06T07:17:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381bf880 con 0x55ea1e81da90 2024-06-06T07:17:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a42650740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381be380 con 0x55ea1e81d170 2024-06-06T07:17:54.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:55478/0 (socket says 172.21.15.73:55478) 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a3cedd640 1 -- 172.21.15.73:0/90112383 learned_addr learned my addr 172.21.15.73:0/90112383 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.365+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 0x55ea1e81faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3cedd640 1 -- 172.21.15.73:0/90112383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 msgr2=0x55ea1e8149c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e8149c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3cedd640 1 -- 172.21.15.73:0/90112383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 msgr2=0x55ea1e81faa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 0x55ea1e81faa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3cedd640 1 -- 172.21.15.73:0/90112383 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1e833ef0 con 0x55ea1e81da90 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.366+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 0x55ea1e81faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.367+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a3806da10 tx=0x7f8a28010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc4151d057c32ca8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.367+0000 7f8a22ffd640 1 -- 172.21.15.73:0/90112383 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28005020 con 0x55ea1e81da90 2024-06-06T07:17:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.367+0000 7f8a22ffd640 1 -- 172.21.15.73:0/90112383 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a280047a0 con 0x55ea1e81da90 2024-06-06T07:17:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.367+0000 7f8a22ffd640 1 -- 172.21.15.73:0/90112383 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a28019790 con 0x55ea1e81da90 2024-06-06T07:17:54.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 -- 172.21.15.73:0/90112383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 msgr2=0x55ea1e825be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8a3806da10 tx=0x7f8a28010800 comp rx=0 tx=0).stop 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 -- 172.21.15.73:0/90112383 shutdown_connections 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e828350 0x55ea1e81faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e81da90 0x55ea1e825be0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 --2- 172.21.15.73:0/90112383 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e8149c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 -- 172.21.15.73:0/90112383 >> 172.21.15.73:0/90112383 conn(0x55ea1e819680 msgr2=0x55ea1e817f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 -- 172.21.15.73:0/90112383 shutdown_connections 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.368+0000 7f8a42650740 1 -- 172.21.15.73:0/90112383 wait complete. 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.369+0000 7f8a42650740 1 -- start start 2024-06-06T07:17:54.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.369+0000 7f8a42650740 1 Processor -- start 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.369+0000 7f8a42650740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.369+0000 7f8a42650740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a42650740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e830690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3d6de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3cedd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38768/0 (socket says 172.21.15.73:38768) 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3d6de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:41184/0 (socket says 172.21.15.73:41184) 2024-06-06T07:17:54.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3cedd640 1 -- 172.21.15.73:0/789432271 learned_addr learned my addr 172.21.15.73:0/789432271 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381bf1b0 con 0x55ea1e81da90 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381bea50 con 0x55ea1e828350 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8a381bbbd0 con 0x55ea1e81d170 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e830690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a42650740 10 client.0 _ll_register_callbacks cb 0x7f8a2800a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 -- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 msgr2=0x55ea1e822f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 -- 172.21.15.73:0/789432271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 msgr2=0x55ea1e830060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1e8215b0 con 0x55ea1e828350 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3cedd640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:17:54.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a3dedf640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e830690 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a300f0350 tx=0x7f8a3003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=504340601d6314ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.370+0000 7f8a20ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a300d4070 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(126..126 src has 1..126) ==== 9466+0+0 (secure 0 0 0) 0x7f8a30010300 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a280047a0 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) ==== 24+0+0 (secure 0 0 0) 0x7f8a3009f050 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 10 client.26438 fetch_fsmap learned FSMap version 65 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) -- 0x55ea1e8215b0 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a280047a0 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 65) ==== 8390+0+0 (secure 0 0 0) 0x7f8a300b8550 con 0x55ea1e828350 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 10 client.26438 fetch_fsmap finished waiting for FSMap version 65 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 10 client.26438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 20 client.26438 populate_metadata read hostname 'smithi073' 2024-06-06T07:17:54.375 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8a2390 0x55ea1e8a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:17:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 4 client.26438 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:17:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.371+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ea1e78ec20 con 0x55ea1e8a2390 2024-06-06T07:17:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.372+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8a2390 0x55ea1e8a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:17:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.372+0000 7f8a20ff9640 10 client.26438 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:54.376 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.372+0000 7f8a3d6de640 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8a2390 0x55ea1e8a47b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4aae9a96efcb221 server_cookie=d372168c008791d9 in_seq=0 out_seq=0 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.452+0000 7f8a20ff9640 1 -- 172.21.15.73:0/789432271 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ea1e78ec20 con 0x55ea1e8a2390 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.452+0000 7f8a20ff9640 10 client.26438 handle_command_reply: tid=0 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.452+0000 7f8a42650740 1 client.26438 shutdown 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 msgr2=0x55ea1e830690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e830690 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8a300f0350 tx=0x7f8a3003d000 comp rx=0 tx=0).stop 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 shutdown_connections 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ea1e8a2390 0x55ea1e8a47b0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ea1e828350 0x55ea1e830690 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ea1e81da90 0x55ea1e830060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 --2- 172.21.15.73:0/789432271 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ea1e81d170 0x55ea1e822f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 >> 172.21.15.73:0/789432271 conn(0x55ea1e819680 msgr2=0x55ea1e833770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a20ff9640 0 client.26438 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 shutdown_connections 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 1 -- 172.21.15.73:0/789432271 wait complete. 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.453+0000 7f8a42650740 20 client.26438 trim_cache size 0 max 16384 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:17:54.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:17:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 msgr2=0x7f8a240a5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 0x7f8a240a5560 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8a34009890 tx=0x7f8a34008040 comp rx=0 tx=0).stop 2024-06-06T07:17:54.456 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 msgr2=0x7f8a3806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a3806c3a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8a3002f260 tx=0x7f8a30035b50 comp rx=0 tx=0).stop 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 shutdown_connections 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a381079b0 0x7f8a3806c3a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8a38102360 0x7f8a38073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f8a240a30a0 0x7f8a240a5560 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 --2- 172.21.15.73:0/1898244657 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8a381019a0 0x7f8a38073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 >> 172.21.15.73:0/1898244657 conn(0x7f8a380fd470 msgr2=0x7f8a380fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 shutdown_connections 2024-06-06T07:17:54.457 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:17:54.454+0000 7f8a3f942640 1 -- 172.21.15.73:0/1898244657 wait complete. 2024-06-06T07:17:54.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30412 8.93 MB/sec execute 59 sec latency 101.835 ms 2024-06-06T07:17:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:55 smithi073 ceph-mon[32303]: pgmap v524: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 428 op/s 2024-06-06T07:17:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:55 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:55 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:17:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:55 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:17:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:55 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:17:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:55 smithi136 ceph-mon[35901]: pgmap v524: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 428 op/s 2024-06-06T07:17:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:55 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:55 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:17:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:55 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:17:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:55 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:17:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:55 smithi170 ceph-mon[36108]: pgmap v524: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.9 MiB/s wr, 428 op/s 2024-06-06T07:17:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:55 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:55 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:17:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:55 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:17:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:55 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:17:55.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30613 9.14 MB/sec execute 60 sec latency 68.809 ms 2024-06-06T07:17:56.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30795 9.17 MB/sec execute 61 sec latency 74.033 ms 2024-06-06T07:17:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:57 smithi073 ceph-mon[32303]: pgmap v525: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2024-06-06T07:17:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:57 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:57 smithi136 ceph-mon[35901]: pgmap v525: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2024-06-06T07:17:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:57 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:57.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:57 smithi170 ceph-mon[36108]: pgmap v525: 129 pgs: 129 active+clean; 603 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 267 op/s 2024-06-06T07:17:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:57 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:17:57.804 INFO:tasks.workunit.client.0.smithi073.stdout: 10 30997 9.24 MB/sec execute 62 sec latency 63.527 ms 2024-06-06T07:17:58.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31123 9.14 MB/sec execute 63 sec latency 114.674 ms 2024-06-06T07:17:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:17:59 smithi073 ceph-mon[32303]: pgmap v526: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 416 op/s 2024-06-06T07:17:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:17:59 smithi136 ceph-mon[35901]: pgmap v526: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 416 op/s 2024-06-06T07:17:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:17:59 smithi170 ceph-mon[36108]: pgmap v526: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.5 MiB/s wr, 416 op/s 2024-06-06T07:17:59.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31296 9.16 MB/sec execute 64 sec latency 58.168 ms 2024-06-06T07:18:00.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31455 9.15 MB/sec execute 65 sec latency 95.548 ms 2024-06-06T07:18:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:01 smithi073 ceph-mon[32303]: pgmap v527: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:01 smithi136 ceph-mon[35901]: pgmap v527: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:01 smithi170 ceph-mon[36108]: pgmap v527: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:01.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31607 9.09 MB/sec execute 66 sec latency 55.943 ms 2024-06-06T07:18:02.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31754 9.07 MB/sec execute 67 sec latency 61.208 ms 2024-06-06T07:18:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:03 smithi073 ceph-mon[32303]: pgmap v528: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:03 smithi136 ceph-mon[35901]: pgmap v528: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:03 smithi170 ceph-mon[36108]: pgmap v528: 129 pgs: 129 active+clean; 611 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 271 op/s 2024-06-06T07:18:03.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 31916 9.02 MB/sec execute 68 sec latency 54.443 ms 2024-06-06T07:18:04.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32103 9.04 MB/sec execute 69 sec latency 100.658 ms 2024-06-06T07:18:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:05 smithi073 ceph-mon[32303]: pgmap v529: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-06-06T07:18:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:05 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:05 smithi073 ceph-mon[32303]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T07:18:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:05 smithi136 ceph-mon[35901]: pgmap v529: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-06-06T07:18:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:05 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:05 smithi136 ceph-mon[35901]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T07:18:05.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:05 smithi170 ceph-mon[36108]: pgmap v529: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.7 MiB/s wr, 425 op/s 2024-06-06T07:18:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:05 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:05 smithi170 ceph-mon[36108]: osdmap e127: 12 total, 12 up, 12 in 2024-06-06T07:18:05.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32267 9.05 MB/sec execute 70 sec latency 91.419 ms 2024-06-06T07:18:06.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32452 9.05 MB/sec execute 71 sec latency 88.635 ms 2024-06-06T07:18:07.399 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:07 smithi170 ceph-mon[36108]: pgmap v531: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-06-06T07:18:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:07 smithi073 ceph-mon[32303]: pgmap v531: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-06-06T07:18:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:07 smithi136 ceph-mon[35901]: pgmap v531: 129 pgs: 129 active+clean; 623 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.8 MiB/s wr, 364 op/s 2024-06-06T07:18:07.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 -- 172.21.15.73:0/4290133407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc101970 msgr2=0x7f91cc10d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 --2- 172.21.15.73:0/4290133407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc101970 0x7f91cc10d750 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009ca0 tx=0x7f91c402a950 comp rx=0 tx=0).stop 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 -- 172.21.15.73:0/4290133407 shutdown_connections 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 --2- 172.21.15.73:0/4290133407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc101970 0x7f91cc10d750 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 --2- 172.21.15.73:0/4290133407 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc100fb0 0x7f91cc101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 --2- 172.21.15.73:0/4290133407 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 0x7f91cc106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.612+0000 7f91d3a76640 1 -- 172.21.15.73:0/4290133407 >> 172.21.15.73:0/4290133407 conn(0x7f91cc076e30 msgr2=0x7f91cc0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.613+0000 7f91d3a76640 1 -- 172.21.15.73:0/4290133407 shutdown_connections 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.613+0000 7f91d3a76640 1 -- 172.21.15.73:0/4290133407 wait complete. 2024-06-06T07:18:07.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.613+0000 7f91d3a76640 1 Processor -- start 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.613+0000 7f91d3a76640 1 -- start start 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 0x7f91cc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 0x7f91cc19ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc10fac0 con 0x7f91cc100fb0 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc10f940 con 0x7f91cc101970 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc10fc40 con 0x7f91cc106900 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49282/0 (socket says 172.21.15.73:49282) 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 -- 172.21.15.73:0/2394249927 learned_addr learned my addr 172.21.15.73:0/2394249927 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:07.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1011640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 0x7f91cc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d2013640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 0x7f91cc19ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 -- 172.21.15.73:0/2394249927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 msgr2=0x7f91cc19ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 0x7f91cc19ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 -- 172.21.15.73:0/2394249927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 msgr2=0x7f91cc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 0x7f91cc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 -- 172.21.15.73:0/2394249927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91cc19f200 con 0x7f91cc100fb0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1011640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 0x7f91cc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d1812640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009990 tx=0x7f91c8009e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91c2ffd640 1 -- 172.21.15.73:0/2394249927 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c801b070 con 0x7f91cc100fb0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91c2ffd640 1 -- 172.21.15.73:0/2394249927 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c8016410 con 0x7f91cc100fb0 2024-06-06T07:18:07.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91c2ffd640 1 -- 172.21.15.73:0/2394249927 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8015610 con 0x7f91cc100fb0 2024-06-06T07:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91cc19f4f0 con 0x7f91cc100fb0 2024-06-06T07:18:07.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.614+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91cc1a6d80 con 0x7f91cc100fb0 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.617+0000 7f91c2ffd640 1 -- 172.21.15.73:0/2394249927 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f91c801e070 con 0x7f91cc100fb0 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.618+0000 7f91c2ffd640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 0x7f91b80a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.618+0000 7f91d6787740 1 Processor -- start 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.618+0000 7f91d1011640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 0x7f91b80a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.618+0000 7f91c2ffd640 1 -- 172.21.15.73:0/2394249927 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f91c80ceba0 con 0x7f91cc100fb0 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d6787740 1 -- start start 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d1011640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 0x7f91b80a52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91cc105e50 tx=0x7f91bc005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.621 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d6787740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d6787740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9a90 0x558fd42c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d6787740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42c6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d6787740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558fd421d4f0 con 0x558fd42b9a90 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d1812640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d1812640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53598/0 (socket says 172.21.15.73:53598) 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d1812640 1 -- 172.21.15.73:0/1170223257 learned_addr learned my addr 172.21.15.73:0/1170223257 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.619+0000 7f91d1011640 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9a90 0x558fd42c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558fd41bb870 con 0x558fd42c4350 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558fd42194b0 con 0x558fd42b9170 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d2013640 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 -- 172.21.15.73:0/1170223257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 msgr2=0x558fd42c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.622 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 -- 172.21.15.73:0/1170223257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9a90 msgr2=0x558fd42c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9a90 0x558fd42c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 -- 172.21.15.73:0/1170223257 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd41d79b0 con 0x558fd42b9170 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91d1812640 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91c8097170 tx=0x7f91c809a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f632aa91180dc69 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91b7fff640 1 -- 172.21.15.73:0/1170223257 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b3e60 con 0x558fd42b9170 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91b7fff640 1 -- 172.21.15.73:0/1170223257 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c80b0030 con 0x558fd42b9170 2024-06-06T07:18:07.623 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.620+0000 7f91b7fff640 1 -- 172.21.15.73:0/1170223257 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80165c0 con 0x558fd42b9170 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.621+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 msgr2=0x558fd42b9550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.621+0000 7f91d6787740 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f91c8097170 tx=0x7f91c809a730 comp rx=0 tx=0).stop 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.621+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 shutdown_connections 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.621+0000 7f91d6787740 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9a90 0x558fd42c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.627 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 --2- 172.21.15.73:0/1170223257 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9170 0x558fd42b9550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 >> 172.21.15.73:0/1170223257 conn(0x558fd42b5680 msgr2=0x558fd42b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 shutdown_connections 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 -- 172.21.15.73:0/1170223257 wait complete. 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 -- start start 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.622+0000 7f91d6787740 1 Processor -- start 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 0x558fd42c1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42c18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 0x558fd42c1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d1011640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42c18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d2013640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 0x558fd42c1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d1812640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 0x558fd42c1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d1011640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42c18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53602/0 (socket says 172.21.15.73:53602) 2024-06-06T07:18:07.628 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d1011640 1 -- 172.21.15.73:0/3461688325 learned_addr learned my addr 172.21.15.73:0/3461688325 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558fd42c6f80 con 0x558fd42c4350 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558fd41a6ae0 con 0x558fd42b9170 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.623+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558fd42c7100 con 0x558fd42b9a90 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 -- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 msgr2=0x558fd42c18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42c18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 -- 172.21.15.73:0/3461688325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 msgr2=0x558fd42c1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 0x558fd42c1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd3f31210 con 0x558fd42b9170 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d2013640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 0x558fd42c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d6787740 10 client.0 _ll_register_callbacks cb 0x558fd41df9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d1812640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 0x558fd42c1360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91c8098fc0 tx=0x7f91c8016b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3bc5210b271e601 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.629 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c80b0030 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91b57fa640 1 -- 172.21.15.73:0/3461688325 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b9070 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91b57fa640 1 -- 172.21.15.73:0/3461688325 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f91c8097380 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91b57fa640 1 -- 172.21.15.73:0/3461688325 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f91c80b3e60 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.624+0000 7f91d6787740 10 client.26450 fetch_fsmap learned FSMap version 67 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=67}) -- 0x558fd3ffd620 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91b57fa640 1 -- 172.21.15.73:0/3461688325 <== mon.2 v2:172.21.15.136:3300/0 4 ==== fsmap(e 67) ==== 8382+0+0 (secure 0 0 0) 0x7f91c80165c0 con 0x558fd42b9170 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 10 client.26450 fetch_fsmap finished waiting for FSMap version 67 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 10 client.26450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 20 client.26450 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd42d42b0 0x558fd42d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 4 client.26450 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:18:07.630 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.625+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fd4230f10 con 0x558fd42d42b0 2024-06-06T07:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.626+0000 7f91d1011640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd42d42b0 0x558fd42d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.626+0000 7f91b57fa640 10 client.26450 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:07.631 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.626+0000 7f91d1011640 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd42d42b0 0x558fd42d66a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80ac827778d8003 server_cookie=c44a8874e1626902 in_seq=0 out_seq=0 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91b57fa640 1 -- 172.21.15.73:0/3461688325 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558fd4230f10 con 0x558fd42d42b0 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91b57fa640 10 client.26450 handle_command_reply: tid=0 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 client.26450 shutdown 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 msgr2=0x558fd42c1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 0x558fd42c1360 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f91c8098fc0 tx=0x7f91c8016b20 comp rx=0 tx=0).stop 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 shutdown_connections 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd42d42b0 0x558fd42d66a0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.632 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42c4350 0x558fd42c1de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42c18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 --2- 172.21.15.73:0/3461688325 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9170 0x558fd42c1360 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 >> 172.21.15.73:0/3461688325 conn(0x558fd42b5680 msgr2=0x558fd42bb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.628+0000 7f91b57fa640 0 client.26450 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.629+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 shutdown_connections 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.629+0000 7f91d6787740 1 -- 172.21.15.73:0/3461688325 wait complete. 2024-06-06T07:18:07.633 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.629+0000 7f91d6787740 20 client.26450 trim_cache size 0 max 16384 2024-06-06T07:18:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.653+0000 7f91d6787740 1 Processor -- start 2024-06-06T07:18:07.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.653+0000 7f91d6787740 1 -- start start 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9a90 0x558fd42c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42c4350 0x558fd42d96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1b1140 con 0x558fd42b9170 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1b2d70 con 0x558fd42b9a90 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d6787740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1b18a0 con 0x558fd42c4350 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49308/0 (socket says 172.21.15.73:49308) 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 -- 172.21.15.73:0/1849354177 learned_addr learned my addr 172.21.15.73:0/1849354177 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d2013640 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42c4350 0x558fd42d96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1011640 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9a90 0x558fd42c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 -- 172.21.15.73:0/1849354177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42c4350 msgr2=0x558fd42d96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42c4350 0x558fd42d96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 -- 172.21.15.73:0/1849354177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9a90 msgr2=0x558fd42c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9a90 0x558fd42c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.654+0000 7f91d1812640 1 -- 172.21.15.73:0/1849354177 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fd42bcf80 con 0x558fd42b9170 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.655+0000 7f91d1812640 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f91c80cc490 tx=0x7f91c80b6000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f684138cd8356ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.655+0000 7f91b77fe640 1 -- 172.21.15.73:0/1849354177 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b9040 con 0x558fd42b9170 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.655+0000 7f91b77fe640 1 -- 172.21.15.73:0/1849354177 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91c80b0030 con 0x558fd42b9170 2024-06-06T07:18:07.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.655+0000 7f91b77fe640 1 -- 172.21.15.73:0/1849354177 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c8079e30 con 0x558fd42b9170 2024-06-06T07:18:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- 172.21.15.73:0/1849354177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 msgr2=0x558fd42bbaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f91c80cc490 tx=0x7f91c80b6000 comp rx=0 tx=0).stop 2024-06-06T07:18:07.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- 172.21.15.73:0/1849354177 shutdown_connections 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42c4350 0x558fd42d96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42b9a90 0x558fd42c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 --2- 172.21.15.73:0/1849354177 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42bbaa0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- 172.21.15.73:0/1849354177 >> 172.21.15.73:0/1849354177 conn(0x558fd42b5680 msgr2=0x558fd42c2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- 172.21.15.73:0/1849354177 shutdown_connections 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- 172.21.15.73:0/1849354177 wait complete. 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 -- start start 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.656+0000 7f91d6787740 1 Processor -- start 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42d7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1b26a0 con 0x558fd42b9170 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1b1f70 con 0x558fd42c4350 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f91cc1af140 con 0x558fd42b9a90 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d6787740 10 client.0 _ll_register_callbacks cb 0x7f91bc009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d1812640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d1011640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d1011640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53620/0 (socket says 172.21.15.73:53620) 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d1011640 1 -- 172.21.15.73:0/924997819 learned_addr learned my addr 172.21.15.73:0/924997819 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.657+0000 7f91d2013640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42d7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 -- 172.21.15.73:0/924997819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 msgr2=0x558fd42d7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42d7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 -- 172.21.15.73:0/924997819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 msgr2=0x558fd42d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:07.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 -- 172.21.15.73:0/924997819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fd42d7e60 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91d1011640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x558fd42c71e0 tx=0x7f91bc0046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=353b3551af67199b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91b57fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.658+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00f480 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(127..127 src has 1..127) ==== 9642+0+0 (secure 0 0 0) 0x7f91bc00f620 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91c80b0030 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=67) ==== 24+0+0 (secure 0 0 0) 0x7f91bc03e050 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91d6787740 10 client.26632 fetch_fsmap learned FSMap version 67 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=67,monmap=4+}) -- 0x558fd4347380 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91bc00fcd0 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.659+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 67) ==== 8382+0+0 (secure 0 0 0) 0x7f91bc01a040 con 0x558fd42b9a90 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 10 client.26632 fetch_fsmap finished waiting for FSMap version 67 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 10 client.26632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:07.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 20 client.26632 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd4347a90 0x558fd4349eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 4 client.26632 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f91c80b0030 con 0x558fd4347a90 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d1812640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd4347a90 0x558fd4349eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.660+0000 7f91d1812640 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd4347a90 0x558fd4349eb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5b68d645121c213 server_cookie=433d7ea46795d2f9 in_seq=0 out_seq=0 2024-06-06T07:18:07.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.661+0000 7f91b57fa640 10 client.26632 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.723+0000 7f91b57fa640 1 -- 172.21.15.73:0/924997819 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f91c80b0030 con 0x558fd4347a90 2024-06-06T07:18:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.723+0000 7f91b57fa640 10 client.26632 handle_command_reply: tid=0 2024-06-06T07:18:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.723+0000 7f91d6787740 1 client.26632 shutdown 2024-06-06T07:18:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 msgr2=0x558fd42d7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x558fd42c71e0 tx=0x7f91bc0046c0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 shutdown_connections 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x558fd4347a90 0x558fd4349eb0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558fd42c4350 0x558fd42d7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91b57fa640 0 client.26632 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:07.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558fd42b9a90 0x558fd42d7120 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 --2- 172.21.15.73:0/924997819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558fd42b9170 0x558fd42d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 >> 172.21.15.73:0/924997819 conn(0x558fd42b5680 msgr2=0x558fd42c1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 shutdown_connections 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 1 -- 172.21.15.73:0/924997819 wait complete. 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.724+0000 7f91d6787740 20 client.26632 trim_cache size 0 max 16384 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:18:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 msgr2=0x7f91b80a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 0x7f91b80a52e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f91cc105e50 tx=0x7f91bc005f90 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 msgr2=0x7f91cc19a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009990 tx=0x7f91c8009e50 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 shutdown_connections 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f91b80a2e20 0x7f91b80a52e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f91cc106900 0x7f91cc19ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f91cc101970 0x7f91cc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 --2- 172.21.15.73:0/2394249927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91cc100fb0 0x7f91cc19a210 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 >> 172.21.15.73:0/2394249927 conn(0x7f91cc076e30 msgr2=0x7f91cc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.726+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 shutdown_connections 2024-06-06T07:18:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:07.727+0000 7f91d3a76640 1 -- 172.21.15.73:0/2394249927 wait complete. 2024-06-06T07:18:07.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32619 9.09 MB/sec execute 72 sec latency 75.588 ms 2024-06-06T07:18:08.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32755 9.02 MB/sec execute 73 sec latency 133.064 ms 2024-06-06T07:18:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:09 smithi073 ceph-mon[32303]: pgmap v532: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:09 smithi136 ceph-mon[35901]: pgmap v532: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:09 smithi170 ceph-mon[36108]: pgmap v532: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:09.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 32948 9.08 MB/sec execute 74 sec latency 67.135 ms 2024-06-06T07:18:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:10 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:10 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:10 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:10.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 33139 9.06 MB/sec execute 75 sec latency 58.818 ms 2024-06-06T07:18:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:11 smithi073 ceph-mon[32303]: pgmap v533: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:11.554 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:11 smithi136 ceph-mon[35901]: pgmap v533: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:11 smithi170 ceph-mon[36108]: pgmap v533: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:11.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 33302 9.05 MB/sec execute 76 sec latency 63.398 ms 2024-06-06T07:18:12.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 33445 9.02 MB/sec execute 77 sec latency 60.870 ms 2024-06-06T07:18:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:13 smithi073 ceph-mon[32303]: pgmap v534: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:13 smithi073 ceph-mon[32303]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T07:18:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:13 smithi136 ceph-mon[35901]: pgmap v534: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:13 smithi136 ceph-mon[35901]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T07:18:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:13 smithi170 ceph-mon[36108]: pgmap v534: 129 pgs: 129 active+clean; 639 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 382 op/s 2024-06-06T07:18:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:13 smithi170 ceph-mon[36108]: osdmap e128: 12 total, 12 up, 12 in 2024-06-06T07:18:13.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 33605 9.02 MB/sec execute 78 sec latency 115.204 ms 2024-06-06T07:18:14.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 33829 9.11 MB/sec execute 79 sec latency 64.306 ms 2024-06-06T07:18:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:15 smithi073 ceph-mon[32303]: pgmap v536: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-06-06T07:18:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:15 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:15 smithi136 ceph-mon[35901]: pgmap v536: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-06-06T07:18:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:15 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:15 smithi170 ceph-mon[36108]: pgmap v536: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 366 op/s 2024-06-06T07:18:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:15 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:15.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34037 9.13 MB/sec execute 80 sec latency 60.204 ms 2024-06-06T07:18:16.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34168 9.09 MB/sec execute 81 sec latency 59.124 ms 2024-06-06T07:18:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:17 smithi073 ceph-mon[32303]: pgmap v537: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-06T07:18:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:17 smithi136 ceph-mon[35901]: pgmap v537: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-06T07:18:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:17 smithi170 ceph-mon[36108]: pgmap v537: 129 pgs: 129 active+clean; 651 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-06T07:18:17.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34299 9.03 MB/sec execute 82 sec latency 69.955 ms 2024-06-06T07:18:18.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34499 9.15 MB/sec execute 83 sec latency 85.142 ms 2024-06-06T07:18:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:19 smithi073 ceph-mon[32303]: pgmap v538: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:19 smithi136 ceph-mon[35901]: pgmap v538: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:19 smithi170 ceph-mon[36108]: pgmap v538: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:19.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34671 9.16 MB/sec execute 84 sec latency 67.077 ms 2024-06-06T07:18:20.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34830 9.16 MB/sec execute 85 sec latency 71.934 ms 2024-06-06T07:18:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.930+0000 7f218cd5a640 1 -- 172.21.15.73:0/2021556087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108bd0 msgr2=0x7f218810cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.930+0000 7f218cd5a640 1 --2- 172.21.15.73:0/2021556087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108bd0 0x7f218810cc10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f217c009900 tx=0x7f217c02a3f0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.931 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 -- 172.21.15.73:0/2021556087 shutdown_connections 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 --2- 172.21.15.73:0/2021556087 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f2188114ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 --2- 172.21.15.73:0/2021556087 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108bd0 0x7f218810cc10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 --2- 172.21.15.73:0/2021556087 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2188108220 0x7f2188108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 -- 172.21.15.73:0/2021556087 >> 172.21.15.73:0/2021556087 conn(0x7f2188075f20 msgr2=0x7f2188078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 -- 172.21.15.73:0/2021556087 shutdown_connections 2024-06-06T07:18:20.932 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 -- 172.21.15.73:0/2021556087 wait complete. 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.931+0000 7f218cd5a640 1 Processor -- start 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 -- start start 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f218819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21881a3000 0x7f21881a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2188118210 con 0x7f2188108220 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2188118090 con 0x7f218810d2f0 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218cd5a640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2188118390 con 0x7f21881a3000 2024-06-06T07:18:20.934 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41528/0 (socket says 172.21.15.73:41528) 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f218659c640 1 -- 172.21.15.73:0/1877955370 learned_addr learned my addr 172.21.15.73:0/1877955370 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f2185d9b640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f218819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.932+0000 7f2186d9d640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21881a3000 0x7f21881a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 -- 172.21.15.73:0/1877955370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21881a3000 msgr2=0x7f21881a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21881a3000 0x7f21881a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 -- 172.21.15.73:0/1877955370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 msgr2=0x7f218819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f218819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 -- 172.21.15.73:0/1877955370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21881a3af0 con 0x7f2188108220 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f2185d9b640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f218819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218659c640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9f0 tx=0x7f217800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.935 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f21777fe640 1 -- 172.21.15.73:0/1877955370 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217800cd80 con 0x7f2188108220 2024-06-06T07:18:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f21777fe640 1 -- 172.21.15.73:0/1877955370 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f21780108e0 con 0x7f2188108220 2024-06-06T07:18:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f21777fe640 1 -- 172.21.15.73:0/1877955370 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2178018920 con 0x7f2188108220 2024-06-06T07:18:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21881a3de0 con 0x7f2188108220 2024-06-06T07:18:20.936 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.933+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21881ab550 con 0x7f2188108220 2024-06-06T07:18:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.936+0000 7f21777fe640 1 -- 172.21.15.73:0/1877955370 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2178018ac0 con 0x7f2188108220 2024-06-06T07:18:20.937 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.936+0000 7f218fa66740 1 Processor -- start 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 -- start start 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 0x55cf79a94e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 0x55cf79a97740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf799ee4f0 con 0x55cf79a95350 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf7998c870 con 0x55cf79a8aa90 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218fa66740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cf799ea4b0 con 0x55cf79a8a170 2024-06-06T07:18:20.938 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f218659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 0x55cf79a94e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f2186d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f2185d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 0x55cf79a97740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f2186d9d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53138/0 (socket says 172.21.15.73:53138) 2024-06-06T07:18:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.937+0000 7f2186d9d640 1 -- 172.21.15.73:0/3779784985 learned_addr learned my addr 172.21.15.73:0/3779784985 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:20.939 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f21777fe640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 0x7f21700a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 -- 172.21.15.73:0/3779784985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 msgr2=0x55cf79a8a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.942 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 -- 172.21.15.73:0/3779784985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 msgr2=0x55cf79a97740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 0x55cf79a97740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 -- 172.21.15.73:0/3779784985 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf799a89b0 con 0x55cf79a8aa90 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2185d9b640 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 0x55cf79a97740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2186d9d640 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2186d9d640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 0x7f21700a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f21777fe640 1 -- 172.21.15.73:0/1877955370 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 9554+0+0 (secure 0 0 0) 0x7f21780cfbf0 con 0x7f2188108220 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f218659c640 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 0x55cf79a94e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21780cced0 tx=0x7f2178024000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8b808bcd1b1c04 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2174ff9640 1 -- 172.21.15.73:0/3779784985 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21780d3050 con 0x55cf79a8aa90 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2186d9d640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 0x7f21700a53c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21800107d0 tx=0x7f2180004a40 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2174ff9640 1 -- 172.21.15.73:0/3779784985 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2178010de0 con 0x55cf79a8aa90 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.938+0000 7f2174ff9640 1 -- 172.21.15.73:0/3779784985 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21780a0050 con 0x55cf79a8aa90 2024-06-06T07:18:20.943 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 -- 172.21.15.73:0/3779784985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 msgr2=0x55cf79a94e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 0x55cf79a94e10 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f21780cced0 tx=0x7f2178024000 comp rx=0 tx=0).stop 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 -- 172.21.15.73:0/3779784985 shutdown_connections 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a95350 0x55cf79a97740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8aa90 0x55cf79a94e10 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 --2- 172.21.15.73:0/3779784985 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a8a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 -- 172.21.15.73:0/3779784985 >> 172.21.15.73:0/3779784985 conn(0x55cf79a86680 msgr2=0x55cf79a86a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 -- 172.21.15.73:0/3779784985 shutdown_connections 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.939+0000 7f218fa66740 1 -- 172.21.15.73:0/3779784985 wait complete. 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 -- start start 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 Processor -- start 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a927c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.944 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79a92d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a97ec0 con 0x55cf79a8aa90 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53144/0 (socket says 172.21.15.73:53144) 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf79977ae0 con 0x55cf79a95350 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218659c640 1 -- 172.21.15.73:0/2662639935 learned_addr learned my addr 172.21.15.73:0/2662639935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cf79a98040 con 0x55cf79a8a170 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f2185d9b640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a927c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f218fa66740 10 client.0 _ll_register_callbacks cb 0x55cf799b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.940+0000 7f2186d9d640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79a92d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 -- 172.21.15.73:0/2662639935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 msgr2=0x55cf79a92d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79a92d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 -- 172.21.15.73:0/2662639935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 msgr2=0x55cf79a927c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.945 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a927c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 -- 172.21.15.73:0/2662639935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf79702210 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218659c640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9c0 tx=0x7f217809bb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44b6509127eec9fe server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f216dffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217809b600 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cf79a01f10 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9554+0+0 (secure 0 0 0) 0x7f2178018db0 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f217809c3a0 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218fa66740 10 client.26644 fetch_fsmap learned FSMap version 69 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.941+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55cf797ce620 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55cf79a01f10 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 69) ==== 8374+0+0 (secure 0 0 0) 0x7f2178098310 con 0x55cf79a8a170 2024-06-06T07:18:20.946 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 10 client.26644 fetch_fsmap finished waiting for FSMap version 69 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 10 client.26644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 20 client.26644 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79aa5200 0x55cf79aa75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 4 client.26644 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf799fbc20 con 0x55cf79aa5200 2024-06-06T07:18:20.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f2185d9b640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79aa5200 0x55cf79aa75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f216dffb640 10 client.26644 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.942+0000 7f2185d9b640 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79aa5200 0x55cf79aa75f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67cf75bd2b39b41 server_cookie=bb07e6a31b51648 in_seq=0 out_seq=0 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f216dffb640 1 -- 172.21.15.73:0/2662639935 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf799fbc20 con 0x55cf79aa5200 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f216dffb640 10 client.26644 handle_command_reply: tid=0 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 client.26644 shutdown 2024-06-06T07:18:20.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 msgr2=0x55cf79a92280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9c0 tx=0x7f217809bb60 comp rx=0 tx=0).stop 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 shutdown_connections 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79aa5200 0x55cf79aa75f0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79a92d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f216dffb640 0 client.26644 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a927c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 --2- 172.21.15.73:0/2662639935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a92280 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 >> 172.21.15.73:0/2662639935 conn(0x55cf79a86680 msgr2=0x55cf79a8c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 shutdown_connections 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.945+0000 7f218fa66740 1 -- 172.21.15.73:0/2662639935 wait complete. 2024-06-06T07:18:20.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.946+0000 7f218fa66740 20 client.26644 trim_cache size 0 max 16384 2024-06-06T07:18:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 Processor -- start 2024-06-06T07:18:20.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 -- start start 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a90b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79aacbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b55d0 con 0x55cf79a8aa90 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b71a0 con 0x55cf79a95350 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f218fa66740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b5d30 con 0x55cf79a8a170 2024-06-06T07:18:20.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f2185d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f2185d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41554/0 (socket says 172.21.15.73:41554) 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.972+0000 7f2185d9b640 1 -- 172.21.15.73:0/3854660382 learned_addr learned my addr 172.21.15.73:0/3854660382 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f218659c640 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a90b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 -- 172.21.15.73:0/3854660382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 msgr2=0x55cf79a90b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a90b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.974 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 -- 172.21.15.73:0/3854660382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 msgr2=0x55cf79aacbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79aacbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 -- 172.21.15.73:0/3854660382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf79a97fe0 con 0x55cf79a8aa90 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f2185d9b640 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f217c000c00 tx=0x7f217c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6633b8da028e79b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f216ffff640 1 -- 172.21.15.73:0/3854660382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c035040 con 0x55cf79a8aa90 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f216ffff640 1 -- 172.21.15.73:0/3854660382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f217c04a070 con 0x55cf79a8aa90 2024-06-06T07:18:20.975 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.973+0000 7f216ffff640 1 -- 172.21.15.73:0/3854660382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c0046c0 con 0x55cf79a8aa90 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.974+0000 7f218fa66740 1 -- 172.21.15.73:0/3854660382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 msgr2=0x55cf79a91090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.974+0000 7f218fa66740 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f217c000c00 tx=0x7f217c005b50 comp rx=0 tx=0).stop 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.974+0000 7f218fa66740 1 -- 172.21.15.73:0/3854660382 shutdown_connections 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.974+0000 7f218fa66740 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a95350 0x55cf79aacbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a91090 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 --2- 172.21.15.73:0/3854660382 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a8a170 0x55cf79a90b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 -- 172.21.15.73:0/3854660382 >> 172.21.15.73:0/3854660382 conn(0x55cf79a86680 msgr2=0x55cf79a8d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 -- 172.21.15.73:0/3854660382 shutdown_connections 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 -- 172.21.15.73:0/3854660382 wait complete. 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 -- start start 2024-06-06T07:18:20.976 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.975+0000 7f218fa66740 1 Processor -- start 2024-06-06T07:18:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 0x55cf79a9d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.978 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 0x55cf79a98790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b6ad0 con 0x55cf79a8aa90 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b6400 con 0x55cf79a8a170 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f21881b35d0 con 0x55cf79a95350 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 0x55cf79a9d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 0x55cf79a9d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:38372/0 (socket says 172.21.15.73:38372) 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41568/0 (socket says 172.21.15.73:41568) 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218659c640 1 -- 172.21.15.73:0/1201431268 learned_addr learned my addr 172.21.15.73:0/1201431268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f218fa66740 10 client.0 _ll_register_callbacks cb 0x7f21800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2186d9d640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 0x55cf79a98790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.979 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 -- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 msgr2=0x55cf79a98790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 0x55cf79a98790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 -- 172.21.15.73:0/1201431268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 msgr2=0x55cf79a9d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 0x55cf79a9d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2185d9b640 1 -- 172.21.15.73:0/1201431268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf79a98dd0 con 0x55cf79a8aa90 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.976+0000 7f2186d9d640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 0x55cf79a98790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f2185d9b640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x55cf79a81e60 tx=0x7f217c032630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf77f85a5cde51b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f216dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f217c052070 con 0x55cf79a8aa90 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(128..128 src has 1..128) ==== 9554+0+0 (secure 0 0 0) 0x7f217c004450 con 0x55cf79a8aa90 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f217c04a070 con 0x55cf79a8aa90 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f217c05b050 con 0x55cf79a8aa90 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f218fa66740 10 client.17232 fetch_fsmap learned FSMap version 69 2024-06-06T07:18:20.980 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.977+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) -- 0x55cf79b0ea30 con 0x55cf79a8aa90 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f217c04a070 con 0x55cf79a8aa90 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 69) ==== 8374+0+0 (secure 0 0 0) 0x7f217c03c4d0 con 0x55cf79a8aa90 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 10 client.17232 fetch_fsmap finished waiting for FSMap version 69 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 10 client.17232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 20 client.17232 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79b0f170 0x55cf79b11590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 4 client.17232 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cf799f5f70 con 0x55cf79b0f170 2024-06-06T07:18:20.981 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.978+0000 7f218659c640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79b0f170 0x55cf79b11590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.979+0000 7f218659c640 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79b0f170 0x55cf79b11590 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7589b0726f195968 server_cookie=54815b319f100e16 in_seq=0 out_seq=0 2024-06-06T07:18:20.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:20.979+0000 7f216dffb640 10 client.17232 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.043+0000 7f216dffb640 1 -- 172.21.15.73:0/1201431268 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cf799f5f70 con 0x55cf79b0f170 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.043+0000 7f216dffb640 10 client.17232 handle_command_reply: tid=0 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.043+0000 7f218fa66740 1 client.17232 shutdown 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 msgr2=0x55cf79a98250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x55cf79a81e60 tx=0x7f217c032630 comp rx=0 tx=0).stop 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 shutdown_connections 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf79b0f170 0x55cf79b11590 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f216dffb640 0 client.17232 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf79a95350 0x55cf79a98790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf79a8aa90 0x55cf79a98250 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.045 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 --2- 172.21.15.73:0/1201431268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf79a8a170 0x55cf79a9d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 >> 172.21.15.73:0/1201431268 conn(0x55cf79a86680 msgr2=0x55cf79a90a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 shutdown_connections 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 1 -- 172.21.15.73:0/1201431268 wait complete. 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.044+0000 7f218fa66740 20 client.17232 trim_cache size 0 max 16384 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:18:21.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 msgr2=0x7f21700a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 0x7f21700a53c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f21800107d0 tx=0x7f2180004a40 comp rx=0 tx=0).stop 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 msgr2=0x7f218819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f217800e9f0 tx=0x7f217800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 shutdown_connections 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f21700a2f00 0x7f21700a53c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f21881a3000 0x7f21881a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f218810d2f0 0x7f218819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 --2- 172.21.15.73:0/1877955370 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2188108220 0x7f218819e790 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 >> 172.21.15.73:0/1877955370 conn(0x7f2188075f20 msgr2=0x7f21880772d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 shutdown_connections 2024-06-06T07:18:21.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:21.045+0000 7f218cd5a640 1 -- 172.21.15.73:0/1877955370 wait complete. 2024-06-06T07:18:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:21 smithi073 ceph-mon[32303]: pgmap v539: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:21 smithi136 ceph-mon[35901]: pgmap v539: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:21 smithi170 ceph-mon[36108]: pgmap v539: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:21.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 34986 9.15 MB/sec execute 86 sec latency 84.282 ms 2024-06-06T07:18:22.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:18:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: ignoring --setuser ceph since I am not root 2024-06-06T07:18:22.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:18:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: ignoring --setgroup ceph since I am not root 2024-06-06T07:18:22.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:18:21 smithi136 ceph-mds[73797]: -- 172.21.15.136:0/340622202 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6e2cf1a40 con 0x55c6e2cc4800 2024-06-06T07:18:22.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:18:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: starting mds.e at 2024-06-06T07:18:22.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:22 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:18:22.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:22 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:18:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:22 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:18:22.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35154 9.12 MB/sec execute 87 sec latency 53.027 ms 2024-06-06T07:18:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: pgmap v540: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T07:18:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/285483202,v1:172.21.15.136:6835/285483202] up:boot 2024-06-06T07:18:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:18:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:18:23.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:23 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: pgmap v540: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/285483202,v1:172.21.15.136:6835/285483202] up:boot 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:18:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:23 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: pgmap v540: 129 pgs: 129 active+clean; 664 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 329 op/s 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: osdmap e129: 12 total, 12 up, 12 in 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/285483202,v1:172.21.15.136:6835/285483202] up:boot 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:18:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:23 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:23.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35298 9.06 MB/sec execute 88 sec latency 116.652 ms 2024-06-06T07:18:24.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35452 9.05 MB/sec execute 89 sec latency 100.039 ms 2024-06-06T07:18:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:25 smithi073 ceph-mon[32303]: pgmap v542: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:25 smithi136 ceph-mon[35901]: pgmap v542: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:25 smithi170 ceph-mon[36108]: pgmap v542: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:25.805 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35617 9.03 MB/sec execute 90 sec latency 95.875 ms 2024-06-06T07:18:26.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35785 9.08 MB/sec execute 91 sec latency 62.067 ms 2024-06-06T07:18:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:27 smithi073 ceph-mon[32303]: pgmap v543: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:27 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:27 smithi136 ceph-mon[35901]: pgmap v543: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:27 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:27 smithi170 ceph-mon[36108]: pgmap v543: 129 pgs: 129 active+clean; 673 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 346 op/s 2024-06-06T07:18:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:27 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:27.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 35945 9.06 MB/sec execute 92 sec latency 54.576 ms 2024-06-06T07:18:28.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36094 9.03 MB/sec execute 93 sec latency 144.599 ms 2024-06-06T07:18:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:29 smithi073 ceph-mon[32303]: pgmap v544: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:29 smithi136 ceph-mon[35901]: pgmap v544: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:29 smithi170 ceph-mon[36108]: pgmap v544: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:29.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36280 9.04 MB/sec execute 94 sec latency 63.088 ms 2024-06-06T07:18:30.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36464 8.99 MB/sec execute 95 sec latency 125.162 ms 2024-06-06T07:18:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:31 smithi073 ceph-mon[32303]: pgmap v545: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:31 smithi136 ceph-mon[35901]: pgmap v545: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:31 smithi170 ceph-mon[36108]: pgmap v545: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:31.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36637 9.02 MB/sec execute 96 sec latency 81.794 ms 2024-06-06T07:18:32.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36769 8.96 MB/sec execute 97 sec latency 74.134 ms 2024-06-06T07:18:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:33 smithi073 ceph-mon[32303]: pgmap v546: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:33 smithi136 ceph-mon[35901]: pgmap v546: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:33 smithi170 ceph-mon[36108]: pgmap v546: 129 pgs: 129 active+clean; 675 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 361 op/s 2024-06-06T07:18:33.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 36980 9.09 MB/sec execute 98 sec latency 53.686 ms 2024-06-06T07:18:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 -- 172.21.15.73:0/426823213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c100900 msgr2=0x7ff07c100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.175 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 --2- 172.21.15.73:0/426823213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c100900 0x7ff07c100d80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009930 tx=0x7ff06c02a420 comp rx=0 tx=0).stop 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 -- 172.21.15.73:0/426823213 shutdown_connections 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 --2- 172.21.15.73:0/426823213 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c10d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 --2- 172.21.15.73:0/426823213 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c100900 0x7ff07c100d80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 --2- 172.21.15.73:0/426823213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c1068c0 0x7ff07c106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 -- 172.21.15.73:0/426823213 >> 172.21.15.73:0/426823213 conn(0x7ff07c0779f0 msgr2=0x7ff07c077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 -- 172.21.15.73:0/426823213 shutdown_connections 2024-06-06T07:18:34.176 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.174+0000 7ff084676640 1 -- 172.21.15.73:0/426823213 wait complete. 2024-06-06T07:18:34.177 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.177+0000 7ff084676640 1 Processor -- start 2024-06-06T07:18:34.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.178+0000 7ff084676640 1 -- start start 2024-06-06T07:18:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff084676640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c100900 0x7ff07c196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff084676640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff084676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c1068c0 0x7ff07c197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff084676640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c110b80 con 0x7ff07c100900 2024-06-06T07:18:34.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff081c11640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff081c11640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:45560/0 (socket says 172.21.15.73:45560) 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff081c11640 1 -- 172.21.15.73:0/1277010923 learned_addr learned my addr 172.21.15.73:0/1277010923 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff082c13640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c1068c0 0x7ff07c197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.179+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c110a00 con 0x7ff07c1012c0 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c110d00 con 0x7ff07c1068c0 2024-06-06T07:18:34.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff082412640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c100900 0x7ff07c196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff081c11640 1 -- 172.21.15.73:0/1277010923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c1068c0 msgr2=0x7ff07c197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff081c11640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c1068c0 0x7ff07c197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff081c11640 1 -- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c100900 msgr2=0x7ff07c196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff081c11640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c100900 0x7ff07c196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.180+0000 7ff081c11640 1 -- 172.21.15.73:0/1277010923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff07c1a6600 con 0x7ff07c1012c0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.181+0000 7ff081c11640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009230 tx=0x7ff06c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.182+0000 7ff0737fe640 1 -- 172.21.15.73:0/1277010923 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c03f030 con 0x7ff07c1012c0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.182+0000 7ff0737fe640 1 -- 172.21.15.73:0/1277010923 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff06c004440 con 0x7ff07c1012c0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.182+0000 7ff0737fe640 1 -- 172.21.15.73:0/1277010923 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06c035070 con 0x7ff07c1012c0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.182+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff07c1a6890 con 0x7ff07c1012c0 2024-06-06T07:18:34.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.183+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff07c1a6c70 con 0x7ff07c1012c0 2024-06-06T07:18:34.190 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.186+0000 7ff087384740 1 Processor -- start 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 -- start start 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0124e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0127750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aaa006d4f0 con 0x55aaa0109050 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aaa000b870 con 0x55aaa0109970 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.187+0000 7ff087384740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aaa00694b0 con 0x55aaa0125360 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39854/0 (socket says 172.21.15.73:39854) 2024-06-06T07:18:34.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 -- 172.21.15.73:0/2950282009 learned_addr learned my addr 172.21.15.73:0/2950282009 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082412640 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0124e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 -- 172.21.15.73:0/2950282009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 msgr2=0x55aaa0127750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0127750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 -- 172.21.15.73:0/2950282009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 msgr2=0x55aaa0124e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0124e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.189+0000 7ff082c13640 1 -- 172.21.15.73:0/2950282009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa00279b0 con 0x55aaa0109050 2024-06-06T07:18:34.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.190+0000 7ff082c13640 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff074033450 tx=0x7ff074033910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbe89f5adad2553 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.190+0000 7ff0737fe640 1 -- 172.21.15.73:0/1277010923 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff06c042030 con 0x7ff07c1012c0 2024-06-06T07:18:34.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.190+0000 7ff070ff9640 1 -- 172.21.15.73:0/2950282009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff074004d20 con 0x55aaa0109050 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.190+0000 7ff070ff9640 1 -- 172.21.15.73:0/2950282009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff074004050 con 0x55aaa0109050 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.191+0000 7ff087384740 1 -- 172.21.15.73:0/2950282009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 msgr2=0x55aaa0109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.191+0000 7ff087384740 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff074033450 tx=0x7ff074033910 comp rx=0 tx=0).stop 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.191+0000 7ff070ff9640 1 -- 172.21.15.73:0/2950282009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 0 0 0) 0x7ff074031770 con 0x55aaa0109050 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.191+0000 7ff0737fe640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 0x7ff0680a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.192+0000 7ff082c13640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 0x7ff0680a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff0737fe640 1 -- 172.21.15.73:0/1277010923 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7ff06c0efa80 con 0x7ff07c1012c0 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff082c13640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 0x7ff0680a5440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff074004990 tx=0x7ff074004200 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff087384740 1 -- 172.21.15.73:0/2950282009 shutdown_connections 2024-06-06T07:18:34.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff087384740 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0127750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff087384740 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0124e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff087384740 1 --2- 172.21.15.73:0/2950282009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa0109430 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.193+0000 7ff087384740 1 -- 172.21.15.73:0/2950282009 >> 172.21.15.73:0/2950282009 conn(0x55aaa0105680 msgr2=0x55aaa0105a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.194+0000 7ff087384740 1 -- 172.21.15.73:0/2950282009 shutdown_connections 2024-06-06T07:18:34.196 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.195+0000 7ff087384740 1 -- 172.21.15.73:0/2950282009 wait complete. 2024-06-06T07:18:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 -- start start 2024-06-06T07:18:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 Processor -- start 2024-06-06T07:18:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0116c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0111a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55aaa010c040 con 0x55aaa0109050 2024-06-06T07:18:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55aa9fff6ae0 con 0x55aaa0109970 2024-06-06T07:18:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55aaa010c1c0 con 0x55aaa0125360 2024-06-06T07:18:34.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff082412640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff082412640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39856/0 (socket says 172.21.15.73:39856) 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff082412640 1 -- 172.21.15.73:0/2001051772 learned_addr learned my addr 172.21.15.73:0/2001051772 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.196+0000 7ff087384740 10 client.0 _ll_register_callbacks cb 0x55aaa002f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082c13640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0111a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082412640 1 -- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 msgr2=0x55aaa0111a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082412640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0111a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082412640 1 -- 172.21.15.73:0/2001051772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 msgr2=0x55aaa0116c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082412640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0116c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.197+0000 7ff082412640 1 -- 172.21.15.73:0/2001051772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9fd81210 con 0x55aaa0109050 2024-06-06T07:18:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.198+0000 7ff082412640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff078012700 tx=0x7ff078009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a693e20ed36f7bbf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.198+0000 7ff065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:18:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff078011bb0 con 0x55aaa0109050 2024-06-06T07:18:34.200 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff074004050 con 0x55aaa0109050 2024-06-06T07:18:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7ff078011d50 con 0x55aaa0109050 2024-06-06T07:18:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7ff07800f460 con 0x55aaa0109050 2024-06-06T07:18:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff087384740 10 client.17250 fetch_fsmap learned FSMap version 72 2024-06-06T07:18:34.201 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.199+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55aa9fe4d620 con 0x55aaa0109050 2024-06-06T07:18:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.202+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff074004050 con 0x55aaa0109050 2024-06-06T07:18:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.202+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 72) ==== 8370+0+0 (secure 0 0 0) 0x7ff07801c7d0 con 0x55aaa0109050 2024-06-06T07:18:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.202+0000 7ff087384740 10 client.17250 fetch_fsmap finished waiting for FSMap version 72 2024-06-06T07:18:34.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff087384740 10 client.17250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff087384740 20 client.17250 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa0121f30 0x55aaa01243d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff087384740 4 client.17250 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:18:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aaa0080f10 con 0x55aaa0121f30 2024-06-06T07:18:34.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.203+0000 7ff081c11640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa0121f30 0x55aaa01243d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.205+0000 7ff081c11640 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa0121f30 0x55aaa01243d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c119cc32051c6a5 server_cookie=8c97b017c2f7490c in_seq=0 out_seq=0 2024-06-06T07:18:34.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.205+0000 7ff065ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:34.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.206+0000 7ff065ffb640 1 -- 172.21.15.73:0/2001051772 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aaa0080f10 con 0x55aaa0121f30 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.206+0000 7ff065ffb640 10 client.17250 handle_command_reply: tid=0 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.206+0000 7ff087384740 1 client.17250 shutdown 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 msgr2=0x55aaa01166f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff078012700 tx=0x7ff078009f90 comp rx=0 tx=0).stop 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 shutdown_connections 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa0121f30 0x55aaa01243d0 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0125360 0x55aaa0111a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109970 0x55aaa0116c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.207+0000 7ff087384740 1 --2- 172.21.15.73:0/2001051772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109050 0x55aaa01166f0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.208+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 >> 172.21.15.73:0/2001051772 conn(0x55aaa0105680 msgr2=0x55aaa010b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.208+0000 7ff065ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.208+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 shutdown_connections 2024-06-06T07:18:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.208+0000 7ff087384740 1 -- 172.21.15.73:0/2001051772 wait complete. 2024-06-06T07:18:34.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.208+0000 7ff087384740 20 client.17250 trim_cache size 0 max 16384 2024-06-06T07:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.231+0000 7ff087384740 1 Processor -- start 2024-06-06T07:18:34.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.231+0000 7ff087384740 1 -- start start 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109970 0x55aaa012cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0125360 0x55aaa012d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1a87b0 con 0x55aaa0125360 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1aa350 con 0x55aaa0109050 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff087384740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1a8ee0 con 0x55aaa0109970 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082412640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082c13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0125360 0x55aaa012d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082412640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:45588/0 (socket says 172.21.15.73:45588) 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082412640 1 -- 172.21.15.73:0/810396424 learned_addr learned my addr 172.21.15.73:0/810396424 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff081c11640 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109970 0x55aaa012cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082412640 1 -- 172.21.15.73:0/810396424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109970 msgr2=0x55aaa012cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.232+0000 7ff082412640 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109970 0x55aaa012cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff082412640 1 -- 172.21.15.73:0/810396424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0125360 msgr2=0x55aaa012d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff082412640 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0125360 0x55aaa012d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff082412640 1 -- 172.21.15.73:0/810396424 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aaa01054b0 con 0x55aaa0109050 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff082412640 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0780126d0 tx=0x7ff0780096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48f4a48cc6ecc78b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff067fff640 1 -- 172.21.15.73:0/810396424 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07802e070 con 0x55aaa0109050 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff067fff640 1 -- 172.21.15.73:0/810396424 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff078011dc0 con 0x55aaa0109050 2024-06-06T07:18:34.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.233+0000 7ff067fff640 1 -- 172.21.15.73:0/810396424 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07801ce60 con 0x55aaa0109050 2024-06-06T07:18:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- 172.21.15.73:0/810396424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 msgr2=0x55aaa01119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff0780126d0 tx=0x7ff0780096c0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- 172.21.15.73:0/810396424 shutdown_connections 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0125360 0x55aaa012d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109970 0x55aaa012cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 --2- 172.21.15.73:0/810396424 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0109050 0x55aaa01119f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- 172.21.15.73:0/810396424 >> 172.21.15.73:0/810396424 conn(0x55aaa0105680 msgr2=0x55aaa01112e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- 172.21.15.73:0/810396424 shutdown_connections 2024-06-06T07:18:34.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- 172.21.15.73:0/810396424 wait complete. 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 -- start start 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.234+0000 7ff087384740 1 Processor -- start 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 0x55aaa011dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 0x55aaa0118250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff081c11640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1a9c80 con 0x55aaa0109970 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff081c11640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39866/0 (socket says 172.21.15.73:39866) 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c1a95b0 con 0x55aaa0125360 2024-06-06T07:18:34.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff081c11640 1 -- 172.21.15.73:0/1038841457 learned_addr learned my addr 172.21.15.73:0/1038841457 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff07c073490 con 0x55aaa0109050 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 0x55aaa011dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff087384740 10 client.0 _ll_register_callbacks cb 0x7ff0780097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 -- 172.21.15.73:0/1038841457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 msgr2=0x55aaa0118250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082c13640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 0x55aaa0118250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 0x55aaa0118250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 -- 172.21.15.73:0/1038841457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 msgr2=0x55aaa011e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:34.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 -- 172.21.15.73:0/1038841457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aaa0118a50 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff081c11640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082c13640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 0x55aaa0118250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff082412640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 0x55aaa011dcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0780126d0 tx=0x7ff078005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4925476d388b9ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.235+0000 7ff065ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff07801ce60 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff078011dc0 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7ff078005450 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) ==== 24+0+0 (secure 0 0 0) 0x7ff07802e040 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 10 client.26671 fetch_fsmap learned FSMap version 72 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) -- 0x55aaa0197150 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff078011dc0 con 0x55aaa0109050 2024-06-06T07:18:34.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 72) ==== 8370+0+0 (secure 0 0 0) 0x7ff07801b620 con 0x55aaa0109050 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 10 client.26671 fetch_fsmap finished waiting for FSMap version 72 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 10 client.26671 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 20 client.26671 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa01975e0 0x55aaa0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 4 client.26671 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.236+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aaa007ac20 con 0x55aaa01975e0 2024-06-06T07:18:34.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.237+0000 7ff081c11640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa01975e0 0x55aaa0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.237+0000 7ff065ffb640 10 client.26671 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:34.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.237+0000 7ff081c11640 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa01975e0 0x55aaa0199a00 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32a2cc11717ca8cc server_cookie=c46eeda98bb1c97a in_seq=0 out_seq=0 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.348+0000 7ff065ffb640 1 -- 172.21.15.73:0/1038841457 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55aaa007ac20 con 0x55aaa01975e0 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.348+0000 7ff065ffb640 10 client.26671 handle_command_reply: tid=0 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.348+0000 7ff087384740 1 client.26671 shutdown 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 msgr2=0x55aaa011dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 0x55aaa011dcf0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff0780126d0 tx=0x7ff078005760 comp rx=0 tx=0).stop 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 shutdown_connections 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55aaa01975e0 0x55aaa0199a00 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55aaa0125360 0x55aaa0118250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff065ffb640 0 client.26671 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55aaa0109970 0x55aaa011e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 --2- 172.21.15.73:0/1038841457 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55aaa0109050 0x55aaa011dcf0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 >> 172.21.15.73:0/1038841457 conn(0x55aaa0105680 msgr2=0x55aaa0103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 shutdown_connections 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 1 -- 172.21.15.73:0/1038841457 wait complete. 2024-06-06T07:18:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.349+0000 7ff087384740 20 client.26671 trim_cache size 0 max 16384 2024-06-06T07:18:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:18:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:18:34.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 msgr2=0x7ff0680a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 0x7ff0680a5440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff074004990 tx=0x7ff074004200 comp rx=0 tx=0).stop 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 msgr2=0x7ff07c197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ff06c009230 tx=0x7ff06c004040 comp rx=0 tx=0).stop 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 shutdown_connections 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff0680a2f80 0x7ff0680a5440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff07c1068c0 0x7ff07c197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff07c1012c0 0x7ff07c197330 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 --2- 172.21.15.73:0/1277010923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff07c100900 0x7ff07c196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 >> 172.21.15.73:0/1277010923 conn(0x7ff07c0779f0 msgr2=0x7ff07c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 shutdown_connections 2024-06-06T07:18:34.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:34.352+0000 7ff084676640 1 -- 172.21.15.73:0/1277010923 wait complete. 2024-06-06T07:18:34.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 37177 9.12 MB/sec execute 99 sec latency 110.198 ms 2024-06-06T07:18:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:35 smithi073 ceph-mon[32303]: pgmap v547: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 446 op/s 2024-06-06T07:18:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:35 smithi136 ceph-mon[35901]: pgmap v547: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 446 op/s 2024-06-06T07:18:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:35 smithi170 ceph-mon[36108]: pgmap v547: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 446 op/s 2024-06-06T07:18:35.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 37367 9.15 MB/sec execute 100 sec latency 48.540 ms 2024-06-06T07:18:36.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 37517 9.12 MB/sec execute 101 sec latency 54.292 ms 2024-06-06T07:18:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:37 smithi073 ceph-mon[32303]: pgmap v548: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-06-06T07:18:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:37 smithi136 ceph-mon[35901]: pgmap v548: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-06-06T07:18:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:37 smithi170 ceph-mon[36108]: pgmap v548: 129 pgs: 129 active+clean; 681 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.2 MiB/s wr, 288 op/s 2024-06-06T07:18:37.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 37689 9.10 MB/sec execute 102 sec latency 65.642 ms 2024-06-06T07:18:38.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 37844 9.10 MB/sec execute 103 sec latency 101.789 ms 2024-06-06T07:18:39.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:39 smithi073 ceph-mon[32303]: pgmap v549: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 432 op/s 2024-06-06T07:18:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:39 smithi136 ceph-mon[35901]: pgmap v549: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 432 op/s 2024-06-06T07:18:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:39 smithi170 ceph-mon[36108]: pgmap v549: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.6 MiB/s wr, 432 op/s 2024-06-06T07:18:39.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38007 9.07 MB/sec execute 104 sec latency 63.847 ms 2024-06-06T07:18:40.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38189 9.10 MB/sec execute 105 sec latency 59.480 ms 2024-06-06T07:18:41.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:41 smithi073 ceph-mon[32303]: pgmap v550: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:41 smithi136 ceph-mon[35901]: pgmap v550: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:41 smithi170 ceph-mon[36108]: pgmap v550: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:41.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38362 9.09 MB/sec execute 106 sec latency 64.542 ms 2024-06-06T07:18:42.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:42 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:42.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:42 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:42.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:42 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:42.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38522 9.09 MB/sec execute 107 sec latency 63.920 ms 2024-06-06T07:18:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:43 smithi073 ceph-mon[32303]: pgmap v551: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:43 smithi136 ceph-mon[35901]: pgmap v551: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:43 smithi170 ceph-mon[36108]: pgmap v551: 129 pgs: 129 active+clean; 699 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.4 MiB/s wr, 282 op/s 2024-06-06T07:18:43.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38705 9.09 MB/sec execute 108 sec latency 88.846 ms 2024-06-06T07:18:44.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 38880 9.11 MB/sec execute 109 sec latency 75.326 ms 2024-06-06T07:18:45.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:45 smithi073 ceph-mon[32303]: pgmap v552: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 430 op/s 2024-06-06T07:18:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:45 smithi136 ceph-mon[35901]: pgmap v552: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 430 op/s 2024-06-06T07:18:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:45 smithi170 ceph-mon[36108]: pgmap v552: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.9 MiB/s wr, 430 op/s 2024-06-06T07:18:45.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39046 9.09 MB/sec execute 110 sec latency 67.991 ms 2024-06-06T07:18:46.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39194 9.06 MB/sec execute 111 sec latency 94.629 ms 2024-06-06T07:18:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:47 smithi073 ceph-mon[32303]: pgmap v553: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 292 op/s 2024-06-06T07:18:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:47 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:47 smithi136 ceph-mon[35901]: pgmap v553: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 292 op/s 2024-06-06T07:18:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:47 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 -- 172.21.15.73:0/1786442662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0408102ab0 msgr2=0x7f0408102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 --2- 172.21.15.73:0/1786442662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0408102ab0 0x7f0408102e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f03f8009900 tx=0x7f03f802a490 comp rx=0 tx=0).stop 2024-06-06T07:18:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 -- 172.21.15.73:0/1786442662 shutdown_connections 2024-06-06T07:18:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 --2- 172.21.15.73:0/1786442662 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f040810bfd0 0x7f040810e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 --2- 172.21.15.73:0/1786442662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04081033d0 0x7f040810ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 --2- 172.21.15.73:0/1786442662 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f0408102ab0 0x7f0408102e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.594+0000 7f040e66d640 1 -- 172.21.15.73:0/1786442662 >> 172.21.15.73:0/1786442662 conn(0x7f0408077a20 msgr2=0x7f0408077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.595+0000 7f040e66d640 1 -- 172.21.15.73:0/1786442662 shutdown_connections 2024-06-06T07:18:47.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.595+0000 7f040e66d640 1 -- 172.21.15.73:0/1786442662 wait complete. 2024-06-06T07:18:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:47 smithi170 ceph-mon[36108]: pgmap v553: 129 pgs: 129 active+clean; 709 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.0 MiB/s wr, 292 op/s 2024-06-06T07:18:47.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:47 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:18:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 Processor -- start 2024-06-06T07:18:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 -- start start 2024-06-06T07:18:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 0x7f040819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 0x7f040819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0408111c40 con 0x7f040810bfd0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f0408111ac0 con 0x7f04081033d0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040e66d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f0408111dc0 con 0x7f0408102ab0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f0407fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 0x7f040819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f04077fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f0407fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 0x7f040819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:49090/0 (socket says 172.21.15.73:49090) 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f04077fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57418/0 (socket says 172.21.15.73:57418) 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f0407fff640 1 -- 172.21.15.73:0/1639490166 learned_addr learned my addr 172.21.15.73:0/1639490166 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.596+0000 7f040cc0a640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 0x7f040819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 -- 172.21.15.73:0/1639490166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 msgr2=0x7f040819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 0x7f040819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 -- 172.21.15.73:0/1639490166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 msgr2=0x7f040819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 -- 172.21.15.73:0/1639490166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f040819f3a0 con 0x7f040810bfd0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f04077fe640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040cc0a640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 0x7f040819ec70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0400009fd0 tx=0x7f040000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f04057fa640 1 -- 172.21.15.73:0/1639490166 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040000ce20 con 0x7f040810bfd0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f04057fa640 1 -- 172.21.15.73:0/1639490166 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0400005020 con 0x7f040810bfd0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f040819f690 con 0x7f040810bfd0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f04057fa640 1 -- 172.21.15.73:0/1639490166 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400010690 con 0x7f040810bfd0 2024-06-06T07:18:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.597+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04081a6f20 con 0x7f040810bfd0 2024-06-06T07:18:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.600+0000 7f04057fa640 1 -- 172.21.15.73:0/1639490166 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f0400004a40 con 0x7f040810bfd0 2024-06-06T07:18:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.602+0000 7f04057fa640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 0x7f03f00a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.602+0000 7f04057fa640 1 -- 172.21.15.73:0/1639490166 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f04000cfcb0 con 0x7f040810bfd0 2024-06-06T07:18:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.602+0000 7f0407fff640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 0x7f03f00a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 Processor -- start 2024-06-06T07:18:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0407fff640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 0x7f03f00a5330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03f80096c0 tx=0x7f03f80046c0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 -- start start 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 0x556ef36ea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556ef364e4f0 con 0x556ef36f5350 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ef35ec870 con 0x556ef36eaa90 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0411377740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556ef364a4b0 con 0x556ef36ea170 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0407fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 0x556ef36ea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f04077fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.606 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f040cc0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0407fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 0x556ef36ea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:49094/0 (socket says 172.21.15.73:49094) 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.603+0000 7f0407fff640 1 -- 172.21.15.73:0/2852053812 learned_addr learned my addr 172.21.15.73:0/2852053812 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 -- 172.21.15.73:0/2852053812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 msgr2=0x556ef36ea550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 0x556ef36ea550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 -- 172.21.15.73:0/2852053812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 msgr2=0x556ef36f4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 -- 172.21.15.73:0/2852053812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef36089b0 con 0x556ef36f5350 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36f7740 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f04000cd050 tx=0x7f040009c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1ac4cb90725d10 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f03ee7fc640 1 -- 172.21.15.73:0/2852053812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400004d30 con 0x556ef36f5350 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f03ee7fc640 1 -- 172.21.15.73:0/2852053812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04000cd2d0 con 0x556ef36f5350 2024-06-06T07:18:47.607 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.604+0000 7f03ee7fc640 1 -- 172.21.15.73:0/2852053812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0400097b70 con 0x556ef36f5350 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 -- 172.21.15.73:0/2852053812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 msgr2=0x556ef36f7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36f7740 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f04000cd050 tx=0x7f040009c660 comp rx=0 tx=0).stop 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 -- 172.21.15.73:0/2852053812 shutdown_connections 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36f7740 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 --2- 172.21.15.73:0/2852053812 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36ea170 0x556ef36ea550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 -- 172.21.15.73:0/2852053812 >> 172.21.15.73:0/2852053812 conn(0x556ef36e6680 msgr2=0x556ef36e6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 -- 172.21.15.73:0/2852053812 shutdown_connections 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.605+0000 7f0411377740 1 -- 172.21.15.73:0/2852053812 wait complete. 2024-06-06T07:18:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 -- start start 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 Processor -- start 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556ef36f7fd0 con 0x556ef36ea170 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556ef35d7ae0 con 0x556ef36eaa90 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0407fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.606+0000 7f0411377740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556ef36f8150 con 0x556ef36f5350 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40600/0 (socket says 172.21.15.73:40600) 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 -- 172.21.15.73:0/2196064193 learned_addr learned my addr 172.21.15.73:0/2196064193 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0411377740 10 client.0 _ll_register_callbacks cb 0x556ef36109b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f04077fe640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 -- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 msgr2=0x556ef36f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36f2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 -- 172.21.15.73:0/2196064193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 msgr2=0x556ef36f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 -- 172.21.15.73:0/2196064193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef3362210 con 0x556ef36ea170 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0407fff640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005ab0 tx=0x7f03f8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4886b57de5d1416a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f03e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f8035040 con 0x556ef36ea170 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f03f803c640 con 0x556ef36ea170 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.607+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04000cd2d0 con 0x556ef36ea170 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.608+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f03f803c840 con 0x556ef36ea170 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.608+0000 7f0411377740 10 client.17265 fetch_fsmap learned FSMap version 74 2024-06-06T07:18:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.608+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x556ef342e620 con 0x556ef36ea170 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f04000cd2d0 con 0x556ef36ea170 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f03f804ab90 con 0x556ef36ea170 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 10 client.17265 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 10 client.17265 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 20 client.17265 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef3710250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 4 client.17265 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ef3661f10 con 0x556ef370de00 2024-06-06T07:18:47.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.609+0000 7f04077fe640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef3710250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.615 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.610+0000 7f03e77fe640 10 client.17265 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.610+0000 7f04077fe640 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef3710250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d6df80ec7d010a server_cookie=8d4a2a72cb2b7ab7 in_seq=0 out_seq=0 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f03e77fe640 1 -- 172.21.15.73:0/2196064193 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ef3661f10 con 0x556ef370de00 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f03e77fe640 10 client.17265 handle_command_reply: tid=0 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f0411377740 1 client.17265 shutdown 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 msgr2=0x556ef36f22d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f03f8005ab0 tx=0x7f03f8009260 comp rx=0 tx=0).stop 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 shutdown_connections 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.611+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef3710250 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f03e77fe640 0 client.17265 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36eaa90 0x556ef36f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 --2- 172.21.15.73:0/2196064193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36ea170 0x556ef36f22d0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 >> 172.21.15.73:0/2196064193 conn(0x556ef36e6680 msgr2=0x556ef36ec3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 shutdown_connections 2024-06-06T07:18:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 1 -- 172.21.15.73:0/2196064193 wait complete. 2024-06-06T07:18:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.612+0000 7f0411377740 20 client.17265 trim_cache size 0 max 16384 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.637+0000 7f0411377740 1 Processor -- start 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.637+0000 7f0411377740 1 -- start start 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3715b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40614/0 (socket says 172.21.15.73:40614) 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0407fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3715b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0407fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3715b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57428/0 (socket says 172.21.15.73:57428) 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36e4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03f400ff60 con 0x556ef36eaa90 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4011aa0 con 0x556ef36ea170 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f0411377740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4010630 con 0x556ef36f5350 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 -- 172.21.15.73:0/3439259463 learned_addr learned my addr 172.21.15.73:0/3439259463 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f040cc0a640 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36e4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 -- 172.21.15.73:0/3439259463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 msgr2=0x556ef36e4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36e4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 -- 172.21.15.73:0/3439259463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 msgr2=0x556ef3715b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3715b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.638+0000 7f04077fe640 1 -- 172.21.15.73:0/3439259463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ef36eb540 con 0x556ef36eaa90 2024-06-06T07:18:47.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.639+0000 7f04077fe640 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f040810dd10 tx=0x7f03fc00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0397e8ebc6538d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.639+0000 7f03edffb640 1 -- 172.21.15.73:0/3439259463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc004220 con 0x556ef36eaa90 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.639+0000 7f03edffb640 1 -- 172.21.15.73:0/3439259463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03fc0043c0 con 0x556ef36eaa90 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.639+0000 7f03edffb640 1 -- 172.21.15.73:0/3439259463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03fc0047a0 con 0x556ef36eaa90 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 -- 172.21.15.73:0/3439259463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 msgr2=0x556ef37160a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f040810dd10 tx=0x7f03fc00a2d0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 -- 172.21.15.73:0/3439259463 shutdown_connections 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36f5350 0x556ef36e4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36eaa90 0x556ef37160a0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 --2- 172.21.15.73:0/3439259463 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3715b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 -- 172.21.15.73:0/3439259463 >> 172.21.15.73:0/3439259463 conn(0x556ef36e6680 msgr2=0x556ef36ed630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 -- 172.21.15.73:0/3439259463 shutdown_connections 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.640+0000 7f0411377740 1 -- 172.21.15.73:0/3439259463 wait complete. 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 -- start start 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 Processor -- start 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3705e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36eaa90 0x556ef3706390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f03f40113d0 con 0x556ef36f5350 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f03f4010d00 con 0x556ef36ea170 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f03f400df40 con 0x556ef36eaa90 2024-06-06T07:18:47.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0411377740 10 client.0 _ll_register_callbacks cb 0x7f03f8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f040cc0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f0407fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3705e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.641+0000 7f04077fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36eaa90 0x556ef3706390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40626/0 (socket says 172.21.15.73:40626) 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 -- 172.21.15.73:0/2975587698 learned_addr learned my addr 172.21.15.73:0/2975587698 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 -- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36eaa90 msgr2=0x556ef3706390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36eaa90 0x556ef3706390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 -- 172.21.15.73:0/2975587698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 msgr2=0x556ef3705e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:18:47.646 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3705e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 -- 172.21.15.73:0/2975587698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ef36ef390 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f0407fff640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3705e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f040cc0a640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f040009bb20 tx=0x7f040009b830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380185aef7d9ee55 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f03e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04000cd2d0 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f040009b270 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.642+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03fc0043c0 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f04000850d0 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 10 client.17271 fetch_fsmap learned FSMap version 74 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x556ef3789590 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03fc0043c0 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f04000a4ce0 con 0x556ef36f5350 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 10 client.17271 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 10 client.17271 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:18:47.647 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 20 client.17271 populate_metadata read hostname 'smithi073' 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef378b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 4 client.17271 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ef365bc20 con 0x556ef370de00 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.643+0000 7f0407fff640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef378b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.644+0000 7f0407fff640 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef378b830 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76df83c7bd0d31fe server_cookie=ddd7b1f5807057b in_seq=0 out_seq=0 2024-06-06T07:18:47.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.644+0000 7f03e77fe640 10 client.17271 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.710+0000 7f03e77fe640 1 -- 172.21.15.73:0/2975587698 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556ef365bc20 con 0x556ef370de00 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.710+0000 7f03e77fe640 10 client.17271 handle_command_reply: tid=0 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.710+0000 7f0411377740 1 client.17271 shutdown 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 msgr2=0x556ef36eeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f040009bb20 tx=0x7f040009b830 comp rx=0 tx=0).stop 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 shutdown_connections 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556ef370de00 0x556ef378b830 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556ef36f5350 0x556ef36eeb60 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556ef36eaa90 0x556ef3706390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f03e77fe640 0 client.17271 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 --2- 172.21.15.73:0/2975587698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556ef36ea170 0x556ef3705e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 >> 172.21.15.73:0/2975587698 conn(0x556ef36e6680 msgr2=0x556ef36f3b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 shutdown_connections 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 1 -- 172.21.15.73:0/2975587698 wait complete. 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.711+0000 7f0411377740 20 client.17271 trim_cache size 0 max 16384 2024-06-06T07:18:47.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:18:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:18:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:18:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 msgr2=0x7f03f00a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 0x7f03f00a5330 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f03f80096c0 tx=0x7f03f80046c0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 msgr2=0x7f040819ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 0x7f040819ec70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f0400009fd0 tx=0x7f040000ef10 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 shutdown_connections 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f040810bfd0 0x7f040819ec70 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f04081033d0 0x7f040819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f03f00a2e70 0x7f03f00a5330 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 --2- 172.21.15.73:0/1639490166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f0408102ab0 0x7f040819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 >> 172.21.15.73:0/1639490166 conn(0x7f0408077a20 msgr2=0x7f0408107f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 shutdown_connections 2024-06-06T07:18:47.716 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:18:47.712+0000 7f040e66d640 1 -- 172.21.15.73:0/1639490166 wait complete. 2024-06-06T07:18:47.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39365 9.05 MB/sec execute 112 sec latency 79.005 ms 2024-06-06T07:18:48.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39546 9.07 MB/sec execute 113 sec latency 89.383 ms 2024-06-06T07:18:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:49 smithi073 ceph-mon[32303]: pgmap v554: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-06-06T07:18:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:49 smithi136 ceph-mon[35901]: pgmap v554: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-06-06T07:18:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:49 smithi170 ceph-mon[36108]: pgmap v554: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 7.1 MiB/s wr, 440 op/s 2024-06-06T07:18:49.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39739 9.09 MB/sec execute 114 sec latency 189.089 ms 2024-06-06T07:18:50.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 39928 9.08 MB/sec execute 115 sec latency 114.317 ms 2024-06-06T07:18:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:51 smithi073 ceph-mon[32303]: pgmap v555: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:51 smithi136 ceph-mon[35901]: pgmap v555: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:51 smithi170 ceph-mon[36108]: pgmap v555: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:51.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40089 9.08 MB/sec execute 116 sec latency 64.318 ms 2024-06-06T07:18:52.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40273 9.12 MB/sec execute 117 sec latency 60.650 ms 2024-06-06T07:18:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:53 smithi073 ceph-mon[32303]: pgmap v556: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:53 smithi136 ceph-mon[35901]: pgmap v556: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:53.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:53 smithi170 ceph-mon[36108]: pgmap v556: 129 pgs: 129 active+clean; 706 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 296 op/s 2024-06-06T07:18:53.806 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40449 9.11 MB/sec execute 118 sec latency 162.531 ms 2024-06-06T07:18:54.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40630 9.12 MB/sec execute 119 sec latency 73.465 ms 2024-06-06T07:18:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:55 smithi073 ceph-mon[32303]: pgmap v557: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 435 op/s 2024-06-06T07:18:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:55 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:18:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:55 smithi136 ceph-mon[35901]: pgmap v557: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 435 op/s 2024-06-06T07:18:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:55 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:18:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:55 smithi170 ceph-mon[36108]: pgmap v557: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.9 MiB/s wr, 435 op/s 2024-06-06T07:18:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:55 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:18:55.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40777 9.12 MB/sec execute 120 sec latency 65.775 ms 2024-06-06T07:18:56.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 40928 9.12 MB/sec execute 121 sec latency 98.464 ms 2024-06-06T07:18:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:57 smithi073 ceph-mon[32303]: pgmap v558: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-06-06T07:18:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:57 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:18:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:57 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:18:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:57 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:18:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:57 smithi136 ceph-mon[35901]: pgmap v558: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-06-06T07:18:57.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:57 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:18:57.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:57 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:18:57.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:57 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:18:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:57 smithi170 ceph-mon[36108]: pgmap v558: 129 pgs: 129 active+clean; 720 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.3 MiB/s wr, 287 op/s 2024-06-06T07:18:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:57 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:18:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:57 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:18:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:57 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:18:57.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41128 9.16 MB/sec execute 122 sec latency 85.588 ms 2024-06-06T07:18:58.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41276 9.14 MB/sec execute 123 sec latency 89.363 ms 2024-06-06T07:18:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:18:59 smithi073 ceph-mon[32303]: pgmap v559: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 415 op/s 2024-06-06T07:18:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:18:59 smithi136 ceph-mon[35901]: pgmap v559: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 415 op/s 2024-06-06T07:18:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:18:59 smithi170 ceph-mon[36108]: pgmap v559: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.0 MiB/s wr, 415 op/s 2024-06-06T07:18:59.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41440 9.16 MB/sec execute 124 sec latency 69.782 ms 2024-06-06T07:19:00.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41580 9.13 MB/sec execute 125 sec latency 117.630 ms 2024-06-06T07:19:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 -- 172.21.15.73:0/2861130667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc100b80 msgr2=0x7f9fcc101000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/2861130667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc100b80 0x7f9fcc101000 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009860 tx=0x7f9fc002a340 comp rx=0 tx=0).stop 2024-06-06T07:19:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 -- 172.21.15.73:0/2861130667 shutdown_connections 2024-06-06T07:19:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/2861130667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc101540 0x7f9fcc10d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/2861130667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc100b80 0x7f9fcc101000 secure :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9fc0009860 tx=0x7f9fc002a340 comp rx=0 tx=0).stop 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/2861130667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc105870 0x7f9fcc105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.847+0000 7f9fd18fc640 1 -- 172.21.15.73:0/2861130667 >> 172.21.15.73:0/2861130667 conn(0x7f9fcc077680 msgr2=0x7f9fcc077a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.848+0000 7f9fd18fc640 1 -- 172.21.15.73:0/2861130667 shutdown_connections 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.848+0000 7f9fd18fc640 1 -- 172.21.15.73:0/2861130667 wait complete. 2024-06-06T07:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.848+0000 7f9fd18fc640 1 Processor -- start 2024-06-06T07:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 -- start start 2024-06-06T07:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 0x7f9fcc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc19cac0 0x7f9fcc19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc111c30 con 0x7f9fcc101540 2024-06-06T07:19:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc111ab0 con 0x7f9fcc105870 2024-06-06T07:19:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fd18fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc111db0 con 0x7f9fcc19cac0 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc19cac0 0x7f9fcc19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 0x7f9fcc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:32890/0 (socket says 172.21.15.73:32890) 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fca7fc640 1 -- 172.21.15.73:0/314704050 learned_addr learned my addr 172.21.15.73:0/314704050 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.849+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 0x7f9fcc198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41998/0 (socket says 172.21.15.73:41998) 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 -- 172.21.15.73:0/314704050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc19cac0 msgr2=0x7f9fcc19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc19cac0 0x7f9fcc19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 -- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 msgr2=0x7f9fcc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 0x7f9fcc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 -- 172.21.15.73:0/314704050 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fcc19d5b0 con 0x7f9fcc105870 2024-06-06T07:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00057c0 tx=0x7f9fc0032e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fd08fa640 1 -- 172.21.15.73:0/314704050 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0037050 con 0x7f9fcc105870 2024-06-06T07:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fcc19d840 con 0x7f9fcc105870 2024-06-06T07:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fd08fa640 1 -- 172.21.15.73:0/314704050 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc0038040 con 0x7f9fcc105870 2024-06-06T07:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fd08fa640 1 -- 172.21.15.73:0/314704050 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc004f860 con 0x7f9fcc105870 2024-06-06T07:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.850+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fcc10e3a0 con 0x7f9fcc105870 2024-06-06T07:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.854+0000 7f9fd08fa640 1 -- 172.21.15.73:0/314704050 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9fc0056020 con 0x7f9fcc105870 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.855+0000 7f9fd460a740 1 Processor -- start 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.855+0000 7f9fd460a740 1 -- start start 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.855+0000 7f9fd08fa640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 0x7f9fb40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 0x7f9fb40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd08fa640 1 -- 172.21.15.73:0/314704050 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f9fc01002c0 con 0x7f9fcc105870 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad76e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad77b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556cad6d24f0 con 0x556cad779350 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556cad670870 con 0x556cad76ea90 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fd460a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556cad6ce4b0 con 0x556cad76e170 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:32892/0 (socket says 172.21.15.73:32892) 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 -- 172.21.15.73:0/3965115534 learned_addr learned my addr 172.21.15.73:0/3965115534 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad76e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad77b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 -- 172.21.15.73:0/3965115534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 msgr2=0x556cad76e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad76e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 -- 172.21.15.73:0/3965115534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 msgr2=0x556cad77b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad77b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fca7fc640 1 -- 172.21.15.73:0/3965115534 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cad68c9b0 con 0x556cad76ea90 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad77b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad76e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.856+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 0x7f9fb40a52e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009c20 tx=0x7f9fc4011810 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.857+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00d4c20 tx=0x7f9fc00cc170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63905e9da827a066 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.857+0000 7f9fb1ffb640 1 -- 172.21.15.73:0/3965115534 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00cc570 con 0x556cad76ea90 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.857+0000 7f9fb1ffb640 1 -- 172.21.15.73:0/3965115534 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc00d0920 con 0x556cad76ea90 2024-06-06T07:19:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.857+0000 7f9fb1ffb640 1 -- 172.21.15.73:0/3965115534 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc0003c10 con 0x556cad76ea90 2024-06-06T07:19:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- 172.21.15.73:0/3965115534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 msgr2=0x556cad778e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00d4c20 tx=0x7f9fc00cc170 comp rx=0 tx=0).stop 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- 172.21.15.73:0/3965115534 shutdown_connections 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad77b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad778e10 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3965115534 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad76e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- 172.21.15.73:0/3965115534 >> 172.21.15.73:0/3965115534 conn(0x556cad76a680 msgr2=0x556cad76aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- 172.21.15.73:0/3965115534 shutdown_connections 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- 172.21.15.73:0/3965115534 wait complete. 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.858+0000 7f9fd460a740 1 -- start start 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 Processor -- start 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76ea90 0x556cad7768a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 0x556cad776de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556cad77bd40 con 0x556cad76e170 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556cad65bae0 con 0x556cad779350 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556cad77bec0 con 0x556cad76ea90 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42016/0 (socket says 172.21.15.73:42016) 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fcaffd640 1 -- 172.21.15.73:0/637679560 learned_addr learned my addr 172.21.15.73:0/637679560 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.859+0000 7f9fd460a740 10 client.0 _ll_register_callbacks cb 0x556cad6949b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76ea90 0x556cad7768a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 0x556cad776de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 -- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76ea90 msgr2=0x556cad7768a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76ea90 0x556cad7768a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 -- 172.21.15.73:0/637679560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 msgr2=0x556cad776de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 0x556cad776de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 -- 172.21.15.73:0/637679560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cad3e6210 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 0x556cad776de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00eb80 tx=0x7f9fbc00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe6c770f0c52e19 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fbc003b70 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f9fbc003d10 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.860+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc00d0920 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f9fbc025050 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 10 client.17283 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x556cad4b2620 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc00d0920 con 0x556cad76e170 2024-06-06T07:19:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f9fbc00b040 con 0x556cad76e170 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 10 client.17283 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 20 client.17283 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad78b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 4 client.17283 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.861+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cad6e5f10 con 0x556cad789130 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.862+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad78b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.862+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad78b580 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=548edb700b971bb5 server_cookie=ddd0dfd080ebd7c9 in_seq=0 out_seq=0 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.862+0000 7f9faaffd640 10 client.17283 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.863+0000 7f9faaffd640 1 -- 172.21.15.73:0/637679560 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556cad6e5f10 con 0x556cad789130 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.863+0000 7f9faaffd640 10 client.17283 handle_command_reply: tid=0 2024-06-06T07:19:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.863+0000 7f9fd460a740 1 client.17283 shutdown 2024-06-06T07:19:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 msgr2=0x556cad776360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f9fbc00eb80 tx=0x7f9fbc00d7b0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 shutdown_connections 2024-06-06T07:19:00.868 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad78b580 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9faaffd640 0 client.17283 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad779350 0x556cad776de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76ea90 0x556cad7768a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 --2- 172.21.15.73:0/637679560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76e170 0x556cad776360 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 >> 172.21.15.73:0/637679560 conn(0x556cad76a680 msgr2=0x556cad7703b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 shutdown_connections 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.864+0000 7f9fd460a740 1 -- 172.21.15.73:0/637679560 wait complete. 2024-06-06T07:19:00.869 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.865+0000 7f9fd460a740 20 client.17283 trim_cache size 0 max 16384 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 Processor -- start 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 -- start start 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad790e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad7913d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad776320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc06f280 con 0x556cad779350 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc070dc0 con 0x556cad76ea90 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fd460a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc06f950 con 0x556cad76e170 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad790e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad776320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fcaffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad790e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36248/0 (socket says 172.21.15.73:36248) 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fcaffd640 1 -- 172.21.15.73:0/4019169027 learned_addr learned my addr 172.21.15.73:0/4019169027 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.887+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad7913d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 -- 172.21.15.73:0/4019169027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 msgr2=0x556cad790e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad790e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 -- 172.21.15.73:0/4019169027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 msgr2=0x556cad776320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad776320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 -- 172.21.15.73:0/4019169027 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cad77bab0 con 0x556cad76ea90 2024-06-06T07:19:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad776320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fca7fc640 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad7913d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00ca820 tx=0x7f9fc0100170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25aa4c49884453ae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fb17fa640 1 -- 172.21.15.73:0/4019169027 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00d0920 con 0x556cad76ea90 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fb17fa640 1 -- 172.21.15.73:0/4019169027 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc00c8be0 con 0x556cad76ea90 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.888+0000 7f9fb17fa640 1 -- 172.21.15.73:0/4019169027 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc00fda50 con 0x556cad76ea90 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- 172.21.15.73:0/4019169027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 msgr2=0x556cad7913d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad7913d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00ca820 tx=0x7f9fc0100170 comp rx=0 tx=0).stop 2024-06-06T07:19:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- 172.21.15.73:0/4019169027 shutdown_connections 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad779350 0x556cad776320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76ea90 0x556cad7913d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 --2- 172.21.15.73:0/4019169027 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad76e170 0x556cad790e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- 172.21.15.73:0/4019169027 >> 172.21.15.73:0/4019169027 conn(0x556cad76a680 msgr2=0x556cad76aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- 172.21.15.73:0/4019169027 shutdown_connections 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- 172.21.15.73:0/4019169027 wait complete. 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.889+0000 7f9fd460a740 1 -- start start 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 Processor -- start 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 0x556cad7ec1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 0x556cad7ec710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 0x556cad77c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc0706f0 con 0x556cad76ea90 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc070020 con 0x556cad76e170 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9fcc06d260 con 0x556cad779350 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fd460a740 10 client.0 _ll_register_callbacks cb 0x7f9fbc009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 0x556cad7ec710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 0x556cad77c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fca7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 0x556cad7ec710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42038/0 (socket says 172.21.15.73:42038) 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fca7fc640 1 -- 172.21.15.73:0/3345329899 learned_addr learned my addr 172.21.15.73:0/3345329899 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 0x556cad7ec1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 -- 172.21.15.73:0/3345329899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 msgr2=0x556cad7ec1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 0x556cad7ec1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 -- 172.21.15.73:0/3345329899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 msgr2=0x556cad7ec710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 0x556cad7ec710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcb7fe640 1 -- 172.21.15.73:0/3345329899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cad77cd10 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.890+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 0x556cad7ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9fcb7fe640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 0x556cad77c620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x556cad76a110 tx=0x7f9fc4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd2299780093d91 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9faaffd640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fc4017040 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(129..129 src has 1..129) ==== 9466+0+0 (secure 0 0 0) 0x7f9fc400fca0 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fc00c8be0 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f9fc4038050 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9fd460a740 10 client.26707 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x556cad7f3990 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.891+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fc00c8be0 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f9fc40283c0 con 0x556cad779350 2024-06-06T07:19:00.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 10 client.26707 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 10 client.26707 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 20 client.26707 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad7f6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 4 client.26707 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556cad6dfc20 con 0x556cad789130 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.892+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad7f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.893+0000 7f9faaffd640 10 client.26707 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.893+0000 7f9fcaffd640 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad7f6170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c73081cd668f748 server_cookie=e7d5c8c2e98b914f in_seq=0 out_seq=0 2024-06-06T07:19:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9faaffd640 1 -- 172.21.15.73:0/3345329899 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556cad6dfc20 con 0x556cad789130 2024-06-06T07:19:00.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9faaffd640 10 client.26707 handle_command_reply: tid=0 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 client.26707 shutdown 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 msgr2=0x556cad77c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 0x556cad77c620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x556cad76a110 tx=0x7f9fc4004730 comp rx=0 tx=0).stop 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 shutdown_connections 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556cad789130 0x556cad7f6170 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556cad779350 0x556cad77c620 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556cad76ea90 0x556cad7ec710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9faaffd640 0 client.26707 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 --2- 172.21.15.73:0/3345329899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556cad76e170 0x556cad7ec1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.954+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 >> 172.21.15.73:0/3345329899 conn(0x556cad76a680 msgr2=0x556cad768f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.955+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 shutdown_connections 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.955+0000 7f9fd460a740 1 -- 172.21.15.73:0/3345329899 wait complete. 2024-06-06T07:19:00.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.955+0000 7f9fd460a740 20 client.26707 trim_cache size 0 max 16384 2024-06-06T07:19:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:19:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:19:00.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 msgr2=0x7f9fb40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 0x7f9fb40a52e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9fc4009c20 tx=0x7f9fc4011810 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 msgr2=0x7f9fcc198790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9fc00057c0 tx=0x7f9fc0032e80 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 shutdown_connections 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9fb40a2e20 0x7f9fb40a52e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9fcc19cac0 0x7f9fcc19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9fcc105870 0x7f9fcc198790 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 --2- 172.21.15.73:0/314704050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9fcc101540 0x7f9fcc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:00.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 >> 172.21.15.73:0/314704050 conn(0x7f9fcc077680 msgr2=0x7f9fcc0fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 shutdown_connections 2024-06-06T07:19:00.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:00.956+0000 7f9fd18fc640 1 -- 172.21.15.73:0/314704050 wait complete. 2024-06-06T07:19:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:01 smithi073 ceph-mon[32303]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:01 smithi136 ceph-mon[35901]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:01 smithi170 ceph-mon[36108]: pgmap v560: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:01.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41722 9.11 MB/sec execute 126 sec latency 81.713 ms 2024-06-06T07:19:02.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 41883 9.13 MB/sec execute 127 sec latency 108.967 ms 2024-06-06T07:19:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:03 smithi073 ceph-mon[32303]: pgmap v561: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:03 smithi136 ceph-mon[35901]: pgmap v561: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:03 smithi170 ceph-mon[36108]: pgmap v561: 129 pgs: 129 active+clean; 733 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 267 op/s 2024-06-06T07:19:03.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42014 9.09 MB/sec execute 128 sec latency 112.158 ms 2024-06-06T07:19:04.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42142 9.03 MB/sec execute 129 sec latency 133.622 ms 2024-06-06T07:19:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:05 smithi073 ceph-mon[32303]: pgmap v562: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-06-06T07:19:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:05 smithi073 ceph-mon[32303]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T07:19:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:05 smithi136 ceph-mon[35901]: pgmap v562: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-06-06T07:19:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:05 smithi136 ceph-mon[35901]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T07:19:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:05 smithi170 ceph-mon[36108]: pgmap v562: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 438 op/s 2024-06-06T07:19:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:05 smithi170 ceph-mon[36108]: osdmap e130: 12 total, 12 up, 12 in 2024-06-06T07:19:05.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42303 9.01 MB/sec execute 130 sec latency 71.145 ms 2024-06-06T07:19:06.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42479 8.99 MB/sec execute 131 sec latency 84.067 ms 2024-06-06T07:19:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:07 smithi073 ceph-mon[32303]: pgmap v564: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 358 op/s 2024-06-06T07:19:07.554 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:07 smithi136 ceph-mon[35901]: pgmap v564: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 358 op/s 2024-06-06T07:19:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:07 smithi170 ceph-mon[36108]: pgmap v564: 129 pgs: 129 active+clean; 738 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 358 op/s 2024-06-06T07:19:07.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42657 8.97 MB/sec execute 132 sec latency 95.630 ms 2024-06-06T07:19:08.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 42821 8.98 MB/sec execute 133 sec latency 110.088 ms 2024-06-06T07:19:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:09 smithi073 ceph-mon[32303]: pgmap v565: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:09 smithi136 ceph-mon[35901]: pgmap v565: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:09 smithi170 ceph-mon[36108]: pgmap v565: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:09.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43013 9.02 MB/sec execute 134 sec latency 70.016 ms 2024-06-06T07:19:10.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43211 9.09 MB/sec execute 135 sec latency 114.061 ms 2024-06-06T07:19:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:11 smithi073 ceph-mon[32303]: pgmap v566: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:11 smithi136 ceph-mon[35901]: pgmap v566: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:11 smithi170 ceph-mon[36108]: pgmap v566: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:11.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43381 9.09 MB/sec execute 136 sec latency 99.132 ms 2024-06-06T07:19:12.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43551 9.08 MB/sec execute 137 sec latency 57.392 ms 2024-06-06T07:19:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:13 smithi073 ceph-mon[32303]: pgmap v567: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:13 smithi073 ceph-mon[32303]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T07:19:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:13 smithi136 ceph-mon[35901]: pgmap v567: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:13 smithi136 ceph-mon[35901]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T07:19:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:13 smithi170 ceph-mon[36108]: pgmap v567: 129 pgs: 129 active+clean; 756 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.8 MiB/s wr, 365 op/s 2024-06-06T07:19:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:13 smithi170 ceph-mon[36108]: osdmap e131: 12 total, 12 up, 12 in 2024-06-06T07:19:13.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43757 9.10 MB/sec execute 138 sec latency 60.716 ms 2024-06-06T07:19:14.085 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 -- 172.21.15.73:0/390389969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 msgr2=0x7f6b1810e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/390389969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b1810e400 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009ca0 tx=0x7f6b1002a950 comp rx=0 tx=0).stop 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 -- 172.21.15.73:0/390389969 shutdown_connections 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/390389969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b1810e400 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/390389969 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b1810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/390389969 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b18103050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 -- 172.21.15.73:0/390389969 >> 172.21.15.73:0/390389969 conn(0x7f6b18077a20 msgr2=0x7f6b18077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 -- 172.21.15.73:0/390389969 shutdown_connections 2024-06-06T07:19:14.086 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.085+0000 7f6b1f79e640 1 -- 172.21.15.73:0/390389969 wait complete. 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 Processor -- start 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 -- start start 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b181008c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b18101340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18111c60 con 0x7f6b1810bfb0 2024-06-06T07:19:14.088 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18111ae0 con 0x7f6b18103590 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1f79e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b18111de0 con 0x7f6b18102c70 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b181008c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54526/0 (socket says 172.21.15.73:54526) 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b181008c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:52988/0 (socket says 172.21.15.73:52988) 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3956069884 learned_addr learned my addr 172.21.15.73:0/3956069884 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b18101340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3956069884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 msgr2=0x7f6b181008c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b181008c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 msgr2=0x7f6b18101340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b18101340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.089 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.086+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3956069884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b181a6560 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ea60 tx=0x7f6b0800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b0e7fc640 1 -- 172.21.15.73:0/3956069884 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800cde0 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b0e7fc640 1 -- 172.21.15.73:0/3956069884 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b080035f0 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b0e7fc640 1 -- 172.21.15.73:0/3956069884 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08010690 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b181a6850 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.087+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b181a6df0 con 0x7f6b18103590 2024-06-06T07:19:14.090 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.090+0000 7f6b0e7fc640 1 -- 172.21.15.73:0/3956069884 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6b080037a0 con 0x7f6b18103590 2024-06-06T07:19:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 Processor -- start 2024-06-06T07:19:14.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 -- start start 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808170 0x561157808550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157815740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56115776c4f0 con 0x561157813350 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56115770a870 con 0x561157808170 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b22586740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5611577684b0 con 0x561157808a90 2024-06-06T07:19:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b0e7fc640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 0x7f6b040a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:52996/0 (socket says 172.21.15.73:52996) 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b1d53a640 1 -- 172.21.15.73:0/692288455 learned_addr learned my addr 172.21.15.73:0/692288455 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157815740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.091+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 0x7f6b040a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 -- 172.21.15.73:0/692288455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808170 msgr2=0x561157808550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808170 0x561157808550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 -- 172.21.15.73:0/692288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 msgr2=0x561157815740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157815740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 -- 172.21.15.73:0/692288455 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611577269b0 con 0x561157808a90 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b0e7fc640 1 -- 172.21.15.73:0/3956069884 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f6b080cf9c0 con 0x7f6b18103590 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157815740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009d20 tx=0x7f6b14011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f994dfead3a1cd13 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 0x7f6b040a5310 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10009820 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b037fe640 1 -- 172.21.15.73:0/692288455 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1401e070 con 0x561157808a90 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b037fe640 1 -- 172.21.15.73:0/692288455 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b14019ae0 con 0x561157808a90 2024-06-06T07:19:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.092+0000 7f6b037fe640 1 -- 172.21.15.73:0/692288455 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1400ea60 con 0x561157808a90 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 -- 172.21.15.73:0/692288455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 msgr2=0x561157812e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6b14009d20 tx=0x7f6b14011810 comp rx=0 tx=0).stop 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 -- 172.21.15.73:0/692288455 shutdown_connections 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157815740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x561157812e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 --2- 172.21.15.73:0/692288455 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808170 0x561157808550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 -- 172.21.15.73:0/692288455 >> 172.21.15.73:0/692288455 conn(0x561157804680 msgr2=0x561157804a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 -- 172.21.15.73:0/692288455 shutdown_connections 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.093+0000 7f6b22586740 1 -- 172.21.15.73:0/692288455 wait complete. 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 -- start start 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 Processor -- start 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157810d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561157815f80 con 0x561157813350 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5611576f5ae0 con 0x561157808a90 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x561157816100 con 0x561157808170 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b22586740 10 client.0 _ll_register_callbacks cb 0x56115772e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1cd39640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1cd39640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54548/0 (socket says 172.21.15.73:54548) 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:53006/0 (socket says 172.21.15.73:53006) 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3823633762 learned_addr learned my addr 172.21.15.73:0/3823633762 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.094+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157810d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 -- 172.21.15.73:0/3823633762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 msgr2=0x561157810850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 -- 172.21.15.73:0/3823633762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 msgr2=0x561157810d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157810d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 -- 172.21.15.73:0/3823633762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561157480210 con 0x561157808170 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b1401c040 tx=0x7f6b14004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c74aaf6cd35e97c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b1401e050 con 0x561157808170 2024-06-06T07:19:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f6b14019630 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.095+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b14019ae0 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6b14033050 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 10 client.26719 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x56115754c620 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b14019ae0 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f6b140035f0 con 0x561157808170 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 10 client.26719 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 10 client.26719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 20 client.26719 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578257a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 4 client.26719 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56115777ff10 con 0x5611578233b0 2024-06-06T07:19:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.096+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578257a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.097+0000 7f6b00ff9640 10 client.26719 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.097+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578257a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1068dcd0677a708 server_cookie=cd9e6b3b00ad9375 in_seq=0 out_seq=0 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3823633762 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56115777ff10 con 0x5611578233b0 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b00ff9640 10 client.26719 handle_command_reply: tid=0 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 client.26719 shutdown 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 msgr2=0x561157810310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6b1401c040 tx=0x7f6b14004270 comp rx=0 tx=0).stop 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 shutdown_connections 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578257a0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157810d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b00ff9640 0 client.26719 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:14.106 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x561157810850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 --2- 172.21.15.73:0/3823633762 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x561157810310 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 >> 172.21.15.73:0/3823633762 conn(0x561157804680 msgr2=0x56115780a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 shutdown_connections 2024-06-06T07:19:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 1 -- 172.21.15.73:0/3823633762 wait complete. 2024-06-06T07:19:14.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.098+0000 7f6b22586740 20 client.26719 trim_cache size 0 max 16384 2024-06-06T07:19:14.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.123+0000 7f6b22586740 1 Processor -- start 2024-06-06T07:19:14.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 -- start start 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x56115782aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x56115782b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157809b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b10c0 con 0x561157813350 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b2c00 con 0x561157808a90 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b22586740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b1790 con 0x561157808170 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b1dd3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157809b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b1dd3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157809b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55324/0 (socket says 172.21.15.73:55324) 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b1dd3b640 1 -- 172.21.15.73:0/864533420 learned_addr learned my addr 172.21.15.73:0/864533420 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.125+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x56115782b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 -- 172.21.15.73:0/864533420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 msgr2=0x56115782aee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:19:14.127 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x56115782aee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 -- 172.21.15.73:0/864533420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 msgr2=0x561157809b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157809b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 -- 172.21.15.73:0/864533420 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56115781f1e0 con 0x561157808a90 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.126+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x56115782b420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ea30 tx=0x7f6b080ccbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78baf2b09066ce4 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.127+0000 7f6b02ffd640 1 -- 172.21.15.73:0/864533420 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0809c1d0 con 0x561157808a90 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.127+0000 7f6b02ffd640 1 -- 172.21.15.73:0/864533420 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b0809bad0 con 0x561157808a90 2024-06-06T07:19:14.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.127+0000 7f6b02ffd640 1 -- 172.21.15.73:0/864533420 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08098140 con 0x561157808a90 2024-06-06T07:19:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 -- 172.21.15.73:0/864533420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 msgr2=0x56115782b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x56115782b420 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ea30 tx=0x7f6b080ccbb0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 -- 172.21.15.73:0/864533420 shutdown_connections 2024-06-06T07:19:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157813350 0x561157809b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157808a90 0x56115782b420 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 --2- 172.21.15.73:0/864533420 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808170 0x56115782aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.129+0000 7f6b22586740 1 -- 172.21.15.73:0/864533420 >> 172.21.15.73:0/864533420 conn(0x561157804680 msgr2=0x56115780ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.130+0000 7f6b22586740 1 -- 172.21.15.73:0/864533420 shutdown_connections 2024-06-06T07:19:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.130+0000 7f6b22586740 1 -- 172.21.15.73:0/864533420 wait complete. 2024-06-06T07:19:14.131 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.130+0000 7f6b22586740 1 -- start start 2024-06-06T07:19:14.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 Processor -- start 2024-06-06T07:19:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 0x56115781b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.135 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x56115781bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 0x561157816c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 0x56115781b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b1d53a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 0x56115781b710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55330/0 (socket says 172.21.15.73:55330) 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b1d53a640 1 -- 172.21.15.73:0/3889996466 learned_addr learned my addr 172.21.15.73:0/3889996466 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b2530 con 0x561157808170 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181b1e60 con 0x561157813350 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.131+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6b181af0c0 con 0x561157808a90 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b22586740 10 client.0 _ll_register_callbacks cb 0x7f6b1401c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x56115781bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 0x561157816c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.136 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3889996466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 msgr2=0x561157816c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 0x561157816c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3889996466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 msgr2=0x56115781b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 0x56115781b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561157817450 con 0x561157808a90 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1dd3b640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 0x561157816c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b1cd39640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x56115781bc50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6b08097f30 tx=0x7f6b080cd230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c165bf8bf87117b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.132+0000 7f6b00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08099b70 con 0x561157808a90 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f6b08004000 con 0x561157808a90 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b0809bad0 con 0x561157808a90 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f6b0809c1d0 con 0x561157808a90 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b22586740 10 client.26722 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:14.137 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.133+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x56115788db20 con 0x561157808a90 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b0809bad0 con 0x561157808a90 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f6b08097820 con 0x561157808a90 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 10 client.26722 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 10 client.26722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 20 client.26722 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578900a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 4 client.26722 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561157779c20 con 0x5611578233b0 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.134+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578900a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.135+0000 7f6b1d53a640 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578900a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db1589f959f476d server_cookie=9de93ce47f96f512 in_seq=0 out_seq=0 2024-06-06T07:19:14.138 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.135+0000 7f6b00ff9640 10 client.26722 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:14.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.202+0000 7f6b00ff9640 1 -- 172.21.15.73:0/3889996466 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561157779c20 con 0x5611578233b0 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.202+0000 7f6b00ff9640 10 client.26722 handle_command_reply: tid=0 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 client.26722 shutdown 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 msgr2=0x56115781bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x56115781bc50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6b08097f30 tx=0x7f6b080cd230 comp rx=0 tx=0).stop 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 shutdown_connections 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5611578233b0 0x5611578900a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x561157813350 0x561157816c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x561157808a90 0x56115781bc50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 --2- 172.21.15.73:0/3889996466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561157808170 0x56115781b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b00ff9640 0 client.26722 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 >> 172.21.15.73:0/3889996466 conn(0x561157804680 msgr2=0x561157809b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 shutdown_connections 2024-06-06T07:19:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 1 -- 172.21.15.73:0/3889996466 wait complete. 2024-06-06T07:19:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.203+0000 7f6b22586740 20 client.26722 trim_cache size 0 max 16384 2024-06-06T07:19:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:19:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:19:14.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 msgr2=0x7f6b040a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 0x7f6b040a5310 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6b10006010 tx=0x7f6b10009820 comp rx=0 tx=0).stop 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 msgr2=0x7f6b18100e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800ea60 tx=0x7f6b0800ef20 comp rx=0 tx=0).stop 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 shutdown_connections 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b1810bfb0 0x7f6b18101340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6b18103590 0x7f6b18100e00 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6b040a2e50 0x7f6b040a5310 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 --2- 172.21.15.73:0/3956069884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6b18102c70 0x7f6b181008c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 >> 172.21.15.73:0/3956069884 conn(0x7f6b18077a20 msgr2=0x7f6b180fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 shutdown_connections 2024-06-06T07:19:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:14.204+0000 7f6b1f79e640 1 -- 172.21.15.73:0/3956069884 wait complete. 2024-06-06T07:19:14.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 43909 9.08 MB/sec execute 139 sec latency 91.854 ms 2024-06-06T07:19:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:15 smithi073 ceph-mon[32303]: pgmap v569: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 325 op/s 2024-06-06T07:19:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:15 smithi136 ceph-mon[35901]: pgmap v569: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 325 op/s 2024-06-06T07:19:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:15 smithi170 ceph-mon[36108]: pgmap v569: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.3 MiB/s wr, 325 op/s 2024-06-06T07:19:15.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44050 9.07 MB/sec execute 140 sec latency 60.546 ms 2024-06-06T07:19:16.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44222 9.08 MB/sec execute 141 sec latency 56.418 ms 2024-06-06T07:19:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:17 smithi073 ceph-mon[32303]: pgmap v570: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-06-06T07:19:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:17 smithi136 ceph-mon[35901]: pgmap v570: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-06-06T07:19:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:17 smithi170 ceph-mon[36108]: pgmap v570: 129 pgs: 129 active+clean; 773 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 315 op/s 2024-06-06T07:19:17.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44415 9.09 MB/sec execute 142 sec latency 55.723 ms 2024-06-06T07:19:18.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44593 9.10 MB/sec execute 143 sec latency 130.884 ms 2024-06-06T07:19:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:19 smithi073 ceph-mon[32303]: pgmap v571: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:19 smithi136 ceph-mon[35901]: pgmap v571: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:19 smithi170 ceph-mon[36108]: pgmap v571: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:19.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44751 9.10 MB/sec execute 144 sec latency 89.898 ms 2024-06-06T07:19:20.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 44914 9.08 MB/sec execute 145 sec latency 78.048 ms 2024-06-06T07:19:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:21 smithi136 ceph-mon[35901]: pgmap v572: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:21 smithi170 ceph-mon[36108]: pgmap v572: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:21 smithi073 ceph-mon[32303]: pgmap v572: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:21.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45080 9.12 MB/sec execute 146 sec latency 71.389 ms 2024-06-06T07:19:22.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45230 9.10 MB/sec execute 147 sec latency 78.527 ms 2024-06-06T07:19:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:23 smithi136 ceph-mon[35901]: pgmap v573: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:23.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:23 smithi170 ceph-mon[36108]: pgmap v573: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:23 smithi073 ceph-mon[32303]: pgmap v573: 129 pgs: 129 active+clean; 794 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.2 MiB/s wr, 328 op/s 2024-06-06T07:19:23.807 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45404 9.11 MB/sec execute 148 sec latency 71.260 ms 2024-06-06T07:19:24.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45571 9.10 MB/sec execute 149 sec latency 77.498 ms 2024-06-06T07:19:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:25 smithi136 ceph-mon[35901]: pgmap v574: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.0 MiB/s wr, 450 op/s 2024-06-06T07:19:25.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:25 smithi170 ceph-mon[36108]: pgmap v574: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.0 MiB/s wr, 450 op/s 2024-06-06T07:19:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:25 smithi073 ceph-mon[32303]: pgmap v574: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 8.0 MiB/s wr, 450 op/s 2024-06-06T07:19:25.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45724 9.06 MB/sec execute 150 sec latency 66.565 ms 2024-06-06T07:19:26.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 45875 9.02 MB/sec execute 151 sec latency 89.743 ms 2024-06-06T07:19:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 -- 172.21.15.73:0/332180038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983011b180 msgr2=0x7f983011d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 --2- 172.21.15.73:0/332180038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983011b180 0x7f983011d570 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9820009900 tx=0x7f982002a490 comp rx=0 tx=0).stop 2024-06-06T07:19:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 -- 172.21.15.73:0/332180038 shutdown_connections 2024-06-06T07:19:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 --2- 172.21.15.73:0/332180038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f983011b180 0x7f983011d570 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 --2- 172.21.15.73:0/332180038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110570 0x7f9830108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 --2- 172.21.15.73:0/332180038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9830110190 0x7f98301089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 -- 172.21.15.73:0/332180038 >> 172.21.15.73:0/332180038 conn(0x7f983006ca70 msgr2=0x7f983006ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 -- 172.21.15.73:0/332180038 shutdown_connections 2024-06-06T07:19:27.343 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.341+0000 7f9836015640 1 -- 172.21.15.73:0/332180038 wait complete. 2024-06-06T07:19:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.342+0000 7f9836015640 1 Processor -- start 2024-06-06T07:19:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.342+0000 7f9836015640 1 -- start start 2024-06-06T07:19:27.344 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 0x7f98301a7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9830120ab0 con 0x7f983011b180 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54764/0 (socket says 172.21.15.73:54764) 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43230/0 (socket says 172.21.15.73:43230) 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982effd640 1 -- 172.21.15.73:0/131584644 learned_addr learned my addr 172.21.15.73:0/131584644 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f982f7fe640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 0x7f98301a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9830120930 con 0x7f9830110190 2024-06-06T07:19:27.345 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.343+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9830120c30 con 0x7f98301ab8f0 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 -- 172.21.15.73:0/131584644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 msgr2=0x7f98301abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 -- 172.21.15.73:0/131584644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 msgr2=0x7f98301a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 0x7f98301a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 -- 172.21.15.73:0/131584644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98301ac3e0 con 0x7f983011b180 2024-06-06T07:19:27.346 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982ffff640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982effd640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f982800ea80 tx=0x7f982800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.344+0000 7f982f7fe640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 0x7f98301a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.345+0000 7f982cff9640 1 -- 172.21.15.73:0/131584644 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982800ce40 con 0x7f983011b180 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.345+0000 7f982cff9640 1 -- 172.21.15.73:0/131584644 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9828011920 con 0x7f983011b180 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.345+0000 7f982cff9640 1 -- 172.21.15.73:0/131584644 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828010a00 con 0x7f983011b180 2024-06-06T07:19:27.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.347+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98301ac6d0 con 0x7f983011b180 2024-06-06T07:19:27.348 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.347+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98301b3de0 con 0x7f983011b180 2024-06-06T07:19:27.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.350+0000 7f9838dff740 1 Processor -- start 2024-06-06T07:19:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.352+0000 7f9838dff740 1 -- start start 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 0x55d3d07de740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3d07354f0 con 0x55d3d07d1170 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d3d06d3870 con 0x55d3d07dc350 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f9838dff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d3d07314b0 con 0x55d3d07d1a90 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f982cff9640 1 -- 172.21.15.73:0/131584644 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9828011470 con 0x7f983011b180 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.353+0000 7f982effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 0x55d3d07de740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.354+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.354+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:43232/0 (socket says 172.21.15.73:43232) 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.354+0000 7f982f7fe640 1 -- 172.21.15.73:0/797021957 learned_addr learned my addr 172.21.15.73:0/797021957 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.354+0000 7f982cff9640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 0x7f98180a5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.356+0000 7f982ffff640 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982effd640 1 -- 172.21.15.73:0/797021957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 msgr2=0x55d3d07dbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982effd640 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982effd640 1 -- 172.21.15.73:0/797021957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 msgr2=0x55d3d07d1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982effd640 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982effd640 1 -- 172.21.15.73:0/797021957 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d06ef9b0 con 0x55d3d07dc350 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.357+0000 7f982ffff640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 0x7f98180a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.358+0000 7f982f7fe640 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.358+0000 7f982effd640 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 0x55d3d07de740 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98280cca00 tx=0x7f98280d7730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44dde4fef453fbf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.358+0000 7f982cff9640 1 -- 172.21.15.73:0/131584644 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f98280cf720 con 0x7f983011b180 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.359+0000 7f9815ffb640 1 -- 172.21.15.73:0/797021957 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9828097af0 con 0x55d3d07dc350 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.359+0000 7f9815ffb640 1 -- 172.21.15.73:0/797021957 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9828011760 con 0x55d3d07dc350 2024-06-06T07:19:27.362 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.359+0000 7f9815ffb640 1 -- 172.21.15.73:0/797021957 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982809b1b0 con 0x55d3d07dc350 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.360+0000 7f9838dff740 1 -- 172.21.15.73:0/797021957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 msgr2=0x55d3d07de740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.360+0000 7f9838dff740 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 0x55d3d07de740 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98280cca00 tx=0x7f98280d7730 comp rx=0 tx=0).stop 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.360+0000 7f982ffff640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 0x7f98180a5490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9820009230 tx=0x7f982002a490 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.361+0000 7f9838dff740 1 -- 172.21.15.73:0/797021957 shutdown_connections 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.361+0000 7f9838dff740 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07dc350 0x55d3d07de740 secure :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f98280cca00 tx=0x7f98280d7730 comp rx=0 tx=0).stop 2024-06-06T07:19:27.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.361+0000 7f9838dff740 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1a90 0x55d3d07dbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.361+0000 7f9838dff740 1 --2- 172.21.15.73:0/797021957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.361+0000 7f9838dff740 1 -- 172.21.15.73:0/797021957 >> 172.21.15.73:0/797021957 conn(0x55d3d07cd680 msgr2=0x55d3d07cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.364+0000 7f9838dff740 1 -- 172.21.15.73:0/797021957 shutdown_connections 2024-06-06T07:19:27.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.365+0000 7f9838dff740 1 -- 172.21.15.73:0/797021957 wait complete. 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.365+0000 7f9838dff740 1 -- start start 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 Processor -- start 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1170 0x55d3d07d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d3d07eff90 con 0x55d3d07da040 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d3d06beae0 con 0x55d3d07d1a90 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d3d07f0110 con 0x55d3d07d1170 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f9838dff740 10 client.0 _ll_register_callbacks cb 0x55d3d06f79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.366+0000 7f982ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54782/0 (socket says 172.21.15.73:54782) 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 -- 172.21.15.73:0/652496536 learned_addr learned my addr 172.21.15.73:0/652496536 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982effd640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 -- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1170 msgr2=0x55d3d07d9570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:19:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1170 0x55d3d07d9570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 -- 172.21.15.73:0/652496536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 msgr2=0x55d3d07d9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07d9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982ffff640 1 -- 172.21.15.73:0/652496536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d0449210 con 0x55d3d07da040 2024-06-06T07:19:27.370 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.367+0000 7f982effd640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07d9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.368+0000 7f982ffff640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9820033000 tx=0x7f9820009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c67196e77d24ad2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.368+0000 7f980effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:19:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.369+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982004f060 con 0x55d3d07da040 2024-06-06T07:19:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.369+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f98200037f0 con 0x55d3d07da040 2024-06-06T07:19:27.372 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.369+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d3d0748f10 con 0x55d3d07da040 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.370+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f9820056050 con 0x55d3d07da040 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.370+0000 7f9838dff740 10 client.17316 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.370+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55d3d0515620 con 0x55d3d07da040 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55d3d0748f10 con 0x55d3d07da040 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f982003aad0 con 0x55d3d07da040 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 10 client.17316 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 20 client.17316 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d07fdf50 0x55d3d08003d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 4 client.17316 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:19:27.373 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.371+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3d0742c20 con 0x55d3d07fdf50 2024-06-06T07:19:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.372+0000 7f982f7fe640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d07fdf50 0x55d3d08003d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.373+0000 7f982f7fe640 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d07fdf50 0x55d3d08003d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=914166907465f7cf server_cookie=c489ddea5c6c3dba in_seq=0 out_seq=0 2024-06-06T07:19:27.374 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.373+0000 7f980effd640 10 client.17316 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.375+0000 7f980effd640 1 -- 172.21.15.73:0/652496536 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d3d0742c20 con 0x55d3d07fdf50 2024-06-06T07:19:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.375+0000 7f980effd640 10 client.17316 handle_command_reply: tid=0 2024-06-06T07:19:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.375+0000 7f9838dff740 1 client.17316 shutdown 2024-06-06T07:19:27.377 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 msgr2=0x55d3d07f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f9820033000 tx=0x7f9820009260 comp rx=0 tx=0).stop 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 shutdown_connections 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d07fdf50 0x55d3d08003d0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07da040 0x55d3d07f93c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07d9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 --2- 172.21.15.73:0/652496536 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07d1170 0x55d3d07d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 >> 172.21.15.73:0/652496536 conn(0x55d3d07cd680 msgr2=0x55d3d07d3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f980effd640 0 client.17316 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 shutdown_connections 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 1 -- 172.21.15.73:0/652496536 wait complete. 2024-06-06T07:19:27.378 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.376+0000 7f9838dff740 20 client.17316 trim_cache size 0 max 16384 2024-06-06T07:19:27.410 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.409+0000 7f9838dff740 1 Processor -- start 2024-06-06T07:19:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f9838dff740 1 -- start start 2024-06-06T07:19:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f9838dff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.411 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f9838dff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f9838dff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.410+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54798/0 (socket says 172.21.15.73:54798) 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 -- 172.21.15.73:0/1678883742 learned_addr learned my addr 172.21.15.73:0/1678883742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982ffff640 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982effd640 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98301b5560 con 0x55d3d07d1170 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98301b70a0 con 0x55d3d07d1a90 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98301b5c30 con 0x55d3d07da040 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 -- 172.21.15.73:0/1678883742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 msgr2=0x55d3d07d7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.413 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07d7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 -- 172.21.15.73:0/1678883742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 msgr2=0x55d3d07f91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f91f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.411+0000 7f982f7fe640 1 -- 172.21.15.73:0/1678883742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3d07d7ac0 con 0x55d3d07d1170 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.412+0000 7f982f7fe640 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9824000c00 tx=0x7f982400ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b841669cbb0311 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.412+0000 7f98157fa640 1 -- 172.21.15.73:0/1678883742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982400ae40 con 0x55d3d07d1170 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.412+0000 7f98157fa640 1 -- 172.21.15.73:0/1678883742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9824022070 con 0x55d3d07d1170 2024-06-06T07:19:27.414 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.412+0000 7f98157fa640 1 -- 172.21.15.73:0/1678883742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982401db90 con 0x55d3d07d1170 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 msgr2=0x55d3d07d3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9824000c00 tx=0x7f982400ab90 comp rx=0 tx=0).stop 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 shutdown_connections 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07d7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 --2- 172.21.15.73:0/1678883742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07d3c00 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 >> 172.21.15.73:0/1678883742 conn(0x55d3d07cd680 msgr2=0x55d3d07d4730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 shutdown_connections 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.413+0000 7f9838dff740 1 -- 172.21.15.73:0/1678883742 wait complete. 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.414+0000 7f9838dff740 1 -- start start 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.414+0000 7f9838dff740 1 Processor -- start 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.414+0000 7f9838dff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07f0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f98301b69d0 con 0x55d3d07d1170 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f98301b6300 con 0x55d3d07d1a90 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f98301a8380 con 0x55d3d07da040 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f9838dff740 10 client.0 _ll_register_callbacks cb 0x7f9820041040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54804/0 (socket says 172.21.15.73:54804) 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982ffff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07f0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982f7fe640 1 -- 172.21.15.73:0/624278647 learned_addr learned my addr 172.21.15.73:0/624278647 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 -- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 msgr2=0x55d3d07f0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07f0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 -- 172.21.15.73:0/624278647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 msgr2=0x55d3d07f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 -- 172.21.15.73:0/624278647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3d07f0da0 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982f7fe640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f982effd640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f0320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f982800efd0 tx=0x7f98280d3000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d37aa7a6e5663b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.415+0000 7f980effd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98280b9060 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f98280bb770 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9824022070 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f98280a9050 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 10 client.26555 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55d3d08679c0 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9824022070 con 0x55d3d07d1a90 2024-06-06T07:19:27.421 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f982800f040 con 0x55d3d07d1a90 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 10 client.26555 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 10 client.26555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 20 client.26555 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d0867cb0 0x55d3d086a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 4 client.26555 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.416+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3d073cf70 con 0x55d3d0867cb0 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.417+0000 7f982f7fe640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d0867cb0 0x55d3d086a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.417+0000 7f980effd640 10 client.26555 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:27.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.417+0000 7f982f7fe640 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d0867cb0 0x55d3d086a0a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1e473d3d352c18a server_cookie=5454208a6cf9a4a2 in_seq=0 out_seq=0 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.535+0000 7f980effd640 1 -- 172.21.15.73:0/624278647 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d3d073cf70 con 0x55d3d0867cb0 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.535+0000 7f980effd640 10 client.26555 handle_command_reply: tid=0 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.535+0000 7f9838dff740 1 client.26555 shutdown 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 msgr2=0x55d3d07f0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f0320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f982800efd0 tx=0x7f98280d3000 comp rx=0 tx=0).stop 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 shutdown_connections 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d3d0867cb0 0x55d3d086a0a0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f980effd640 0 client.26555 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d3d07da040 0x55d3d07f0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d3d07d1a90 0x55d3d07f0320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 --2- 172.21.15.73:0/624278647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d3d07d1170 0x55d3d07f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 >> 172.21.15.73:0/624278647 conn(0x55d3d07cd680 msgr2=0x55d3d07d7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 shutdown_connections 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 1 -- 172.21.15.73:0/624278647 wait complete. 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.536+0000 7f9838dff740 20 client.26555 trim_cache size 0 max 16384 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:19:27.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 msgr2=0x7f98180a5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 0x7f98180a5490 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f9820009230 tx=0x7f982002a490 comp rx=0 tx=0).stop 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 msgr2=0x7f98301a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f982800ea80 tx=0x7f982800ef40 comp rx=0 tx=0).stop 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 shutdown_connections 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f98180a2fd0 0x7f98180a5490 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.539 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f98301ab8f0 0x7f98301abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f983011b180 0x7f98301a7570 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 --2- 172.21.15.73:0/131584644 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9830110190 0x7f98301a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 >> 172.21.15.73:0/131584644 conn(0x7f983006ca70 msgr2=0x7f983010de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 shutdown_connections 2024-06-06T07:19:27.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:27.537+0000 7f9836015640 1 -- 172.21.15.73:0/131584644 wait complete. 2024-06-06T07:19:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:27 smithi136 ceph-mon[35901]: pgmap v575: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 296 op/s 2024-06-06T07:19:27.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:27 smithi170 ceph-mon[36108]: pgmap v575: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 296 op/s 2024-06-06T07:19:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:27 smithi073 ceph-mon[32303]: pgmap v575: 129 pgs: 129 active+clean; 794 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.2 MiB/s wr, 296 op/s 2024-06-06T07:19:27.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46027 9.00 MB/sec execute 152 sec latency 141.622 ms 2024-06-06T07:19:28.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46211 9.01 MB/sec execute 153 sec latency 85.578 ms 2024-06-06T07:19:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:29 smithi136 ceph-mon[35901]: pgmap v576: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 444 op/s 2024-06-06T07:19:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:29 smithi170 ceph-mon[36108]: pgmap v576: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 444 op/s 2024-06-06T07:19:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:29 smithi073 ceph-mon[32303]: pgmap v576: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 444 op/s 2024-06-06T07:19:29.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46406 9.02 MB/sec execute 154 sec latency 73.935 ms 2024-06-06T07:19:30.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46590 9.04 MB/sec execute 155 sec latency 86.496 ms 2024-06-06T07:19:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:31 smithi136 ceph-mon[35901]: pgmap v577: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:31 smithi170 ceph-mon[36108]: pgmap v577: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:31 smithi073 ceph-mon[32303]: pgmap v577: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:31.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46755 9.04 MB/sec execute 156 sec latency 94.553 ms 2024-06-06T07:19:32.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 46974 9.11 MB/sec execute 157 sec latency 69.224 ms 2024-06-06T07:19:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:33 smithi136 ceph-mon[35901]: pgmap v578: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:33 smithi170 ceph-mon[36108]: pgmap v578: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:33 smithi073 ceph-mon[32303]: pgmap v578: 129 pgs: 129 active+clean; 801 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.3 MiB/s wr, 300 op/s 2024-06-06T07:19:33.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47150 9.10 MB/sec execute 158 sec latency 78.160 ms 2024-06-06T07:19:34.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47325 9.12 MB/sec execute 159 sec latency 61.289 ms 2024-06-06T07:19:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:35 smithi136 ceph-mon[35901]: pgmap v579: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 438 op/s 2024-06-06T07:19:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:35 smithi170 ceph-mon[36108]: pgmap v579: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 438 op/s 2024-06-06T07:19:35.777 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:35 smithi073 ceph-mon[32303]: pgmap v579: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 438 op/s 2024-06-06T07:19:35.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47494 9.11 MB/sec execute 160 sec latency 83.843 ms 2024-06-06T07:19:36.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47658 9.12 MB/sec execute 161 sec latency 56.832 ms 2024-06-06T07:19:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:37 smithi136 ceph-mon[35901]: pgmap v580: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 285 op/s 2024-06-06T07:19:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:37 smithi170 ceph-mon[36108]: pgmap v580: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 285 op/s 2024-06-06T07:19:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:37 smithi073 ceph-mon[32303]: pgmap v580: 129 pgs: 129 active+clean; 811 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 285 op/s 2024-06-06T07:19:37.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47790 9.09 MB/sec execute 162 sec latency 138.514 ms 2024-06-06T07:19:38.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 47962 9.10 MB/sec execute 163 sec latency 63.517 ms 2024-06-06T07:19:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:39 smithi136 ceph-mon[35901]: pgmap v581: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-06-06T07:19:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:39 smithi170 ceph-mon[36108]: pgmap v581: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-06-06T07:19:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:39 smithi073 ceph-mon[32303]: pgmap v581: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.8 MiB/s wr, 412 op/s 2024-06-06T07:19:39.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48144 9.12 MB/sec execute 164 sec latency 66.968 ms 2024-06-06T07:19:40.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 -- 172.21.15.73:0/3330905294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c100b10 msgr2=0x7fea8c100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 --2- 172.21.15.73:0/3330905294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c100b10 0x7fea8c100f70 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fea7c009900 tx=0x7fea7c02a410 comp rx=0 tx=0).stop 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 -- 172.21.15.73:0/3330905294 shutdown_connections 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 --2- 172.21.15.73:0/3330905294 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c1014b0 0x7fea8c10d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 --2- 172.21.15.73:0/3330905294 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c100b10 0x7fea8c100f70 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 --2- 172.21.15.73:0/3330905294 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c106940 0x7fea8c106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 -- 172.21.15.73:0/3330905294 >> 172.21.15.73:0/3330905294 conn(0x7fea8c077780 msgr2=0x7fea8c077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 -- 172.21.15.73:0/3330905294 shutdown_connections 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.662+0000 7fea932f5640 1 -- 172.21.15.73:0/3330905294 wait complete. 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 Processor -- start 2024-06-06T07:19:40.664 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 -- start start 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 0x7fea8c198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 0x7fea8c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 0x7fea8c19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c111c70 con 0x7fea8c106940 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c111af0 con 0x7fea8c1014b0 2024-06-06T07:19:40.666 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.663+0000 7fea932f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c111df0 con 0x7fea8c100b10 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 0x7fea8c19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea91091640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 0x7fea8c198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 0x7fea8c19ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47656/0 (socket says 172.21.15.73:47656) 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea91892640 1 -- 172.21.15.73:0/3928111170 learned_addr learned my addr 172.21.15.73:0/3928111170 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 0x7fea8c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 -- 172.21.15.73:0/3928111170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 msgr2=0x7fea8c198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 0x7fea8c198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 -- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 msgr2=0x7fea8c19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 0x7fea8c19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 -- 172.21.15.73:0/3928111170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8c19d140 con 0x7fea8c1014b0 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea91091640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 0x7fea8c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.664+0000 7fea90890640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 0x7fea8c198690 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02a930 tx=0x7fea7c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.667 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.665+0000 7fea827fc640 1 -- 172.21.15.73:0/3928111170 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c03f030 con 0x7fea8c1014b0 2024-06-06T07:19:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.665+0000 7fea827fc640 1 -- 172.21.15.73:0/3928111170 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea7c003aa0 con 0x7fea8c1014b0 2024-06-06T07:19:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.665+0000 7fea827fc640 1 -- 172.21.15.73:0/3928111170 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c035070 con 0x7fea8c1014b0 2024-06-06T07:19:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.665+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8c19d3d0 con 0x7fea8c1014b0 2024-06-06T07:19:40.668 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.665+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8c1a4cc0 con 0x7fea8c1014b0 2024-06-06T07:19:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.668+0000 7fea95fff740 1 Processor -- start 2024-06-06T07:19:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 -- start start 2024-06-06T07:19:40.669 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea827fc640 1 -- 172.21.15.73:0/3928111170 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fea7c042030 con 0x7fea8c1014b0 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38a90 0x556756a42e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a45740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55675699c4f0 con 0x556756a38170 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55675693a870 con 0x556756a38a90 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea95fff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5567569984b0 con 0x556756a43350 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.670 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea90890640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a45740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47662/0 (socket says 172.21.15.73:47662) 2024-06-06T07:19:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.669+0000 7fea91892640 1 -- 172.21.15.73:0/2974212675 learned_addr learned my addr 172.21.15.73:0/2974212675 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:40.671 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91091640 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38a90 0x556756a42e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 -- 172.21.15.73:0/2974212675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 msgr2=0x556756a45740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a45740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 -- 172.21.15.73:0/2974212675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38a90 msgr2=0x556756a42e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38a90 0x556756a42e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 -- 172.21.15.73:0/2974212675 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567569569b0 con 0x556756a38170 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea90890640 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a45740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea91892640 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fea84009ce0 tx=0x7fea840117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223e3d4d3f5cc5ae server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.670+0000 7fea827fc640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 0x7fea780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.671+0000 7fea777fe640 1 -- 172.21.15.73:0/2974212675 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84012450 con 0x556756a38170 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.671+0000 7fea91091640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 0x7fea780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.673 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.671+0000 7fea777fe640 1 -- 172.21.15.73:0/2974212675 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea84012aa0 con 0x556756a38170 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.671+0000 7fea777fe640 1 -- 172.21.15.73:0/2974212675 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8400fb50 con 0x556756a38170 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.671+0000 7fea827fc640 1 -- 172.21.15.73:0/3928111170 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fea7c0efc30 con 0x7fea8c1014b0 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- 172.21.15.73:0/2974212675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 msgr2=0x556756a38550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea91091640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 0x7fea780a5260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556756a39710 tx=0x7fea88009680 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fea84009ce0 tx=0x7fea840117b0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- 172.21.15.73:0/2974212675 shutdown_connections 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a45740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38a90 0x556756a42e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 --2- 172.21.15.73:0/2974212675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38170 0x556756a38550 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- 172.21.15.73:0/2974212675 >> 172.21.15.73:0/2974212675 conn(0x556756a34680 msgr2=0x556756a34a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- 172.21.15.73:0/2974212675 shutdown_connections 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- 172.21.15.73:0/2974212675 wait complete. 2024-06-06T07:19:40.674 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 -- start start 2024-06-06T07:19:40.675 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.672+0000 7fea95fff740 1 Processor -- start 2024-06-06T07:19:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.677 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a40d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556756a4a390 con 0x556756a38a90 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556756925ae0 con 0x556756a38170 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556756a4a510 con 0x556756a43350 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea95fff740 10 client.0 _ll_register_callbacks cb 0x55675695e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea91091640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea91091640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:60470/0 (socket says 172.21.15.73:60470) 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea91091640 1 -- 172.21.15.73:0/2750259530 learned_addr learned my addr 172.21.15.73:0/2750259530 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47676/0 (socket says 172.21.15.73:47676) 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea91892640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.678 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 -- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 msgr2=0x556756a40d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a40d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 -- 172.21.15.73:0/2750259530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 msgr2=0x556756a40280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea90890640 1 -- 172.21.15.73:0/2750259530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567566b0210 con 0x556756a38a90 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.673+0000 7fea91091640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea90890640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ecf10 tx=0x7fea7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f20479e440658a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c049070 con 0x556756a38a90 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea84012aa0 con 0x556756a38a90 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fea7c0ed190 con 0x556756a38a90 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fea7c0bb640 con 0x556756a38a90 2024-06-06T07:19:40.679 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea95fff740 10 client.17334 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55675677c620 con 0x556756a38a90 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea84012aa0 con 0x556756a38a90 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fea7c0bbf40 con 0x556756a38a90 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea95fff740 10 client.17334 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.674+0000 7fea95fff740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea95fff740 20 client.17334 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756a57990 0x556756a59e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea95fff740 4 client.17334 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567569aff10 con 0x556756a57990 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea91091640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756a57990 0x556756a59e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea74ff9640 10 client.17334 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.675+0000 7fea91091640 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756a57990 0x556756a59e10 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddeb43a309a1abe3 server_cookie=2fcee93d332a56f8 in_seq=0 out_seq=0 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea74ff9640 1 -- 172.21.15.73:0/2750259530 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5567569aff10 con 0x556756a57990 2024-06-06T07:19:40.680 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea74ff9640 10 client.17334 handle_command_reply: tid=0 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 client.17334 shutdown 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 msgr2=0x556756a407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fea7c0ecf10 tx=0x7fea7c005fd0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 shutdown_connections 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756a57990 0x556756a59e10 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a43350 0x556756a40d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a38a90 0x556756a407c0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 --2- 172.21.15.73:0/2750259530 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a38170 0x556756a40280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea74ff9640 0 client.17334 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.676+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 >> 172.21.15.73:0/2750259530 conn(0x556756a34680 msgr2=0x556756a3a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.677+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 shutdown_connections 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.677+0000 7fea95fff740 1 -- 172.21.15.73:0/2750259530 wait complete. 2024-06-06T07:19:40.681 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.677+0000 7fea95fff740 20 client.17334 trim_cache size 0 max 16384 2024-06-06T07:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.699+0000 7fea95fff740 1 Processor -- start 2024-06-06T07:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 -- start start 2024-06-06T07:19:40.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a3ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a40950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1acca0 con 0x556756a40e90 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ae7e0 con 0x556756a43350 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea95fff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ad370 con 0x556756a38170 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47686/0 (socket says 172.21.15.73:47686) 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea90890640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a40950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 -- 172.21.15.73:0/3762345659 learned_addr learned my addr 172.21.15.73:0/3762345659 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91091640 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a3ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 -- 172.21.15.73:0/3762345659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 msgr2=0x556756a3ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a3ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 -- 172.21.15.73:0/3762345659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 msgr2=0x556756a40950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a40950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.700+0000 7fea91892640 1 -- 172.21.15.73:0/3762345659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556756a5a490 con 0x556756a40e90 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.701+0000 7fea91892640 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fea84009c80 tx=0x7fea84009a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbeda8940ff9d873 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.701+0000 7fea76ffd640 1 -- 172.21.15.73:0/3762345659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84028030 con 0x556756a40e90 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.701+0000 7fea76ffd640 1 -- 172.21.15.73:0/3762345659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea84009dd0 con 0x556756a40e90 2024-06-06T07:19:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.701+0000 7fea76ffd640 1 -- 172.21.15.73:0/3762345659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea84004dc0 con 0x556756a40e90 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- 172.21.15.73:0/3762345659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 msgr2=0x556756a49b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fea84009c80 tx=0x7fea84009a10 comp rx=0 tx=0).stop 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- 172.21.15.73:0/3762345659 shutdown_connections 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a49b40 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a40950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 --2- 172.21.15.73:0/3762345659 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a3ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- 172.21.15.73:0/3762345659 >> 172.21.15.73:0/3762345659 conn(0x556756a34680 msgr2=0x556756a40240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- 172.21.15.73:0/3762345659 shutdown_connections 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- 172.21.15.73:0/3762345659 wait complete. 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 -- start start 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.702+0000 7fea95fff740 1 Processor -- start 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a468a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a4e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ae110 con 0x556756a40e90 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1ada40 con 0x556756a43350 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fea8c1aaca0 con 0x556756a38170 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea95fff740 10 client.0 _ll_register_callbacks cb 0x7fea7c005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea91091640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea91091640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:58034/0 (socket says 172.21.15.73:58034) 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea91091640 1 -- 172.21.15.73:0/3363219397 learned_addr learned my addr 172.21.15.73:0/3363219397 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea91892640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a4e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 -- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 msgr2=0x556756a46360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 -- 172.21.15.73:0/3363219397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 msgr2=0x556756a4e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a4e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea90890640 1 -- 172.21.15.73:0/3363219397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556756a471a0 con 0x556756a40e90 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.703+0000 7fea91091640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea90890640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a468a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x556756a41310 tx=0x7fea7c0b8580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510d4065b5c9e793 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea74ff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea7c049040 con 0x556756a40e90 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7fea7c0c46c0 con 0x556756a40e90 2024-06-06T07:19:40.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea84009dd0 con 0x556756a40e90 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fea7c0ed3a0 con 0x556756a40e90 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea95fff740 10 client.17340 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x556756a4fdd0 con 0x556756a40e90 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.704+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea84009dd0 con 0x556756a40e90 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fea7c0b7a30 con 0x556756a40e90 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 10 client.17340 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 20 client.17340 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756ac6ee0 0x556756ac92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 4 client.17340 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5567569a9c20 con 0x556756ac6ee0 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea91091640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756ac6ee0 0x556756ac92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:40.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea74ff9640 10 client.17340 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:40.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.705+0000 7fea91091640 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756ac6ee0 0x556756ac92d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0afd49b8ca2d078 server_cookie=2a8b26dc011c7e88 in_seq=0 out_seq=0 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea74ff9640 1 -- 172.21.15.73:0/3363219397 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5567569a9c20 con 0x556756ac6ee0 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea74ff9640 10 client.17340 handle_command_reply: tid=0 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea95fff740 1 client.17340 shutdown 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 msgr2=0x556756a468a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a468a0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x556756a41310 tx=0x7fea7c0b8580 comp rx=0 tx=0).stop 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.771+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 shutdown_connections 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x556756ac6ee0 0x556756ac92d0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea74ff9640 0 client.17340 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:40.773 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556756a43350 0x556756a4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556756a40e90 0x556756a468a0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 --2- 172.21.15.73:0/3363219397 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556756a38170 0x556756a46360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 >> 172.21.15.73:0/3363219397 conn(0x556756a34680 msgr2=0x556756a525e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 shutdown_connections 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 1 -- 172.21.15.73:0/3363219397 wait complete. 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.772+0000 7fea95fff740 20 client.17340 trim_cache size 0 max 16384 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:19:40.774 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:19:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:19:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.773+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 msgr2=0x7fea780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.773+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 0x7fea780a5260 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x556756a39710 tx=0x7fea88009680 comp rx=0 tx=0).stop 2024-06-06T07:19:40.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.773+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 msgr2=0x7fea8c198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.773+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 0x7fea8c198690 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fea7c02a930 tx=0x7fea7c004040 comp rx=0 tx=0).stop 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 shutdown_connections 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea8c106940 0x7fea8c19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fea8c1014b0 0x7fea8c198690 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fea780a2da0 0x7fea780a5260 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 --2- 172.21.15.73:0/3928111170 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fea8c100b10 0x7fea8c198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 >> 172.21.15.73:0/3928111170 conn(0x7fea8c077780 msgr2=0x7fea8c0ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 shutdown_connections 2024-06-06T07:19:40.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:40.774+0000 7fea932f5640 1 -- 172.21.15.73:0/3928111170 wait complete. 2024-06-06T07:19:40.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48312 9.10 MB/sec execute 165 sec latency 66.062 ms 2024-06-06T07:19:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:41 smithi136 ceph-mon[35901]: pgmap v582: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:41 smithi170 ceph-mon[36108]: pgmap v582: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:41 smithi073 ceph-mon[32303]: pgmap v582: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:41.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48438 9.08 MB/sec execute 166 sec latency 64.837 ms 2024-06-06T07:19:42.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48585 9.06 MB/sec execute 167 sec latency 84.698 ms 2024-06-06T07:19:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:43 smithi136 ceph-mon[35901]: pgmap v583: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:43 smithi170 ceph-mon[36108]: pgmap v583: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:43 smithi073 ceph-mon[32303]: pgmap v583: 129 pgs: 129 active+clean; 825 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.9 MiB/s wr, 264 op/s 2024-06-06T07:19:43.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48767 9.08 MB/sec execute 168 sec latency 89.713 ms 2024-06-06T07:19:44.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 48913 9.07 MB/sec execute 169 sec latency 117.572 ms 2024-06-06T07:19:45.575 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:45 smithi073 ceph-mon[32303]: pgmap v584: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:19:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:45 smithi170 ceph-mon[36108]: pgmap v584: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:19:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:45 smithi136 ceph-mon[35901]: pgmap v584: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:19:45.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49079 9.07 MB/sec execute 170 sec latency 89.967 ms 2024-06-06T07:19:46.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49226 9.06 MB/sec execute 171 sec latency 75.916 ms 2024-06-06T07:19:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:47 smithi170 ceph-mon[36108]: pgmap v585: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-06T07:19:47.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:47 smithi073 ceph-mon[32303]: pgmap v585: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-06T07:19:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:47 smithi136 ceph-mon[35901]: pgmap v585: 129 pgs: 129 active+clean; 831 MiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-06T07:19:47.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49383 9.05 MB/sec execute 172 sec latency 81.653 ms 2024-06-06T07:19:48.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49550 9.04 MB/sec execute 173 sec latency 95.160 ms 2024-06-06T07:19:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:49 smithi170 ceph-mon[36108]: pgmap v586: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2024-06-06T07:19:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:49 smithi073 ceph-mon[32303]: pgmap v586: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2024-06-06T07:19:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:49 smithi136 ceph-mon[35901]: pgmap v586: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 420 op/s 2024-06-06T07:19:49.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49706 9.03 MB/sec execute 174 sec latency 69.888 ms 2024-06-06T07:19:50.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 49880 9.02 MB/sec execute 175 sec latency 83.902 ms 2024-06-06T07:19:51.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:51 smithi170 ceph-mon[36108]: pgmap v587: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:51.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:51 smithi073 ceph-mon[32303]: pgmap v587: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:51 smithi136 ceph-mon[35901]: pgmap v587: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:51.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50051 9.03 MB/sec execute 176 sec latency 79.433 ms 2024-06-06T07:19:52.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50202 9.02 MB/sec execute 177 sec latency 64.675 ms 2024-06-06T07:19:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:53 smithi170 ceph-mon[36108]: pgmap v588: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:53 smithi073 ceph-mon[32303]: pgmap v588: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:53 smithi136 ceph-mon[35901]: pgmap v588: 129 pgs: 129 active+clean; 836 MiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 293 op/s 2024-06-06T07:19:53.808 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50394 9.05 MB/sec execute 178 sec latency 65.004 ms 2024-06-06T07:19:53.909 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 -- 172.21.15.73:0/1332077990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1874100260 msgr2=0x7f1874100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 --2- 172.21.15.73:0/1332077990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1874100260 0x7f1874100640 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f1864009900 tx=0x7f186402a410 comp rx=0 tx=0).stop 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 -- 172.21.15.73:0/1332077990 shutdown_connections 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 --2- 172.21.15.73:0/1332077990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874105330 0x7f187410cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 --2- 172.21.15.73:0/1332077990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874100c10 0x7f1874104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 --2- 172.21.15.73:0/1332077990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1874100260 0x7f1874100640 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 -- 172.21.15.73:0/1332077990 >> 172.21.15.73:0/1332077990 conn(0x7f1874076e30 msgr2=0x7f18740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.908+0000 7f187a0f5640 1 -- 172.21.15.73:0/1332077990 shutdown_connections 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.909+0000 7f187a0f5640 1 -- 172.21.15.73:0/1332077990 wait complete. 2024-06-06T07:19:53.910 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.909+0000 7f187a0f5640 1 Processor -- start 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.909+0000 7f187a0f5640 1 -- start start 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f187410fa60 con 0x7f187419ec00 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f187410f8e0 con 0x7f1874105330 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f187a0f5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f187410fbe0 con 0x7f1874100260 2024-06-06T07:19:53.912 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f18737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f18737fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:37836/0 (socket says 172.21.15.73:37836) 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57986/0 (socket says 172.21.15.73:57986) 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f18737fe640 1 -- 172.21.15.73:0/2720451673 learned_addr learned my addr 172.21.15.73:0/2720451673 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1872ffd640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:46340/0 (socket says 172.21.15.73:46340) 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 -- 172.21.15.73:0/2720451673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 msgr2=0x7f187419a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 -- 172.21.15.73:0/2720451673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 msgr2=0x7f187419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1873fff640 1 -- 172.21.15.73:0/2720451673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f187419f6f0 con 0x7f187419ec00 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f18737fe640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:53.913 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.910+0000 7f1872ffd640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f1873fff640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f186c009fd0 tx=0x7f186c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f1870ff9640 1 -- 172.21.15.73:0/2720451673 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c00ce20 con 0x7f187419ec00 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f187419f9e0 con 0x7f187419ec00 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f1870ff9640 1 -- 172.21.15.73:0/2720451673 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f186c005020 con 0x7f187419ec00 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f1870ff9640 1 -- 172.21.15.73:0/2720451673 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c010690 con 0x7f187419ec00 2024-06-06T07:19:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.911+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18741a71d0 con 0x7f187419ec00 2024-06-06T07:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.913+0000 7f1870ff9640 1 -- 172.21.15.73:0/2720451673 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f186c010830 con 0x7f187419ec00 2024-06-06T07:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.914+0000 7f187cdfe740 1 Processor -- start 2024-06-06T07:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.914+0000 7f187cdfe740 1 -- start start 2024-06-06T07:19:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.914+0000 7f1870ff9640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 0x7f185c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f1872ffd640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 0x7f185c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01df2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee01d4c4f0 con 0x55ee01de8a90 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee01cea870 con 0x55ee01de8170 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f187cdfe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ee01d484b0 con 0x55ee01df3350 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:46356/0 (socket says 172.21.15.73:46356) 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f1873fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01df2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 -- 172.21.15.73:0/209243827 learned_addr learned my addr 172.21.15.73:0/209243827 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 -- 172.21.15.73:0/209243827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 msgr2=0x55ee01df5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 -- 172.21.15.73:0/209243827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 msgr2=0x55ee01df2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01df2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 -- 172.21.15.73:0/209243827 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee01d069b0 con 0x55ee01de8170 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f1872ffd640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 0x7f185c0a5260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1874105070 tx=0x7f186800eca0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f1870ff9640 1 -- 172.21.15.73:0/2720451673 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f186c0cfe80 con 0x7f187419ec00 2024-06-06T07:19:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.915+0000 7f18737fe640 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1864006010 tx=0x7f18640044b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=528de0a176f6ee76 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.916+0000 7f1859ffb640 1 -- 172.21.15.73:0/209243827 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864040070 con 0x55ee01de8170 2024-06-06T07:19:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.916+0000 7f1859ffb640 1 -- 172.21.15.73:0/209243827 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18640047a0 con 0x55ee01de8170 2024-06-06T07:19:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.916+0000 7f1859ffb640 1 -- 172.21.15.73:0/209243827 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864035040 con 0x55ee01de8170 2024-06-06T07:19:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- 172.21.15.73:0/209243827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 msgr2=0x55ee01de8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f1864006010 tx=0x7f18640044b0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- 172.21.15.73:0/209243827 shutdown_connections 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01df2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 --2- 172.21.15.73:0/209243827 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01de8550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- 172.21.15.73:0/209243827 >> 172.21.15.73:0/209243827 conn(0x55ee01de4680 msgr2=0x55ee01de4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- 172.21.15.73:0/209243827 shutdown_connections 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- 172.21.15.73:0/209243827 wait complete. 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.917+0000 7f187cdfe740 1 -- start start 2024-06-06T07:19:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 Processor -- start 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01de8a90 0x55ee01df0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 0x55ee01df0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ee01df5f20 con 0x55ee01de8170 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ee01cd5ae0 con 0x55ee01df3350 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ee01df60a0 con 0x55ee01de8a90 2024-06-06T07:19:53.925 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f187cdfe740 10 client.0 _ll_register_callbacks cb 0x55ee01d0e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f18737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 0x55ee01df0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f18737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58008/0 (socket says 172.21.15.73:58008) 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f18737fe640 1 -- 172.21.15.73:0/3879313567 learned_addr learned my addr 172.21.15.73:0/3879313567 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1872ffd640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01de8a90 0x55ee01df0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 -- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01de8a90 msgr2=0x55ee01df0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01de8a90 0x55ee01df0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 -- 172.21.15.73:0/3879313567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 msgr2=0x55ee01df0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f1873fff640 1 -- 172.21.15.73:0/3879313567 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee01a60210 con 0x55ee01df3350 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.918+0000 7f18737fe640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:53.926 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f1873fff640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 0x55ee01df0d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f186c0cd190 tx=0x7f186c09c7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59537c638b02edd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f1852ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c09bf90 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f186c0051d0 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18640047a0 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f186c0a0340 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f187cdfe740 10 client.26585 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.919+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55ee01b2c620 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18640047a0 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f186c0770c0 con 0x55ee01df3350 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 10 client.26585 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 10 client.26585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 20 client.26585 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:53.927 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e03500 0x55ee01e058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 4 client.26585 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee01d5ff10 con 0x55ee01e03500 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f18737fe640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e03500 0x55ee01e058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.921+0000 7f1852ffd640 10 client.26585 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.920+0000 7f18737fe640 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e03500 0x55ee01e058f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c80b2f9a8c9b2fd8 server_cookie=9630d6dae81291e8 in_seq=0 out_seq=0 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.921+0000 7f1852ffd640 1 -- 172.21.15.73:0/3879313567 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee01d5ff10 con 0x55ee01e03500 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.921+0000 7f1852ffd640 10 client.26585 handle_command_reply: tid=0 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.921+0000 7f187cdfe740 1 client.26585 shutdown 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 msgr2=0x55ee01df0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 0x55ee01df0d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f186c0cd190 tx=0x7f186c09c7d0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 shutdown_connections 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e03500 0x55ee01e058f0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.928 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01df3350 0x55ee01df0d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01de8a90 0x55ee01df0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3879313567 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01df0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f1852ffd640 0 client.26585 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 >> 172.21.15.73:0/3879313567 conn(0x55ee01de4680 msgr2=0x55ee01dea400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 shutdown_connections 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 1 -- 172.21.15.73:0/3879313567 wait complete. 2024-06-06T07:19:53.929 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.922+0000 7f187cdfe740 20 client.26585 trim_cache size 0 max 16384 2024-06-06T07:19:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.947+0000 7f187cdfe740 1 Processor -- start 2024-06-06T07:19:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 -- start start 2024-06-06T07:19:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01deab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01de3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18741b1020 con 0x55ee01de8a90 2024-06-06T07:19:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18741b2b60 con 0x55ee01de8170 2024-06-06T07:19:53.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f187cdfe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18741b16f0 con 0x55ee01df3350 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1872ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01de3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:37858/0 (socket says 172.21.15.73:37858) 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 -- 172.21.15.73:0/3578088575 learned_addr learned my addr 172.21.15.73:0/3578088575 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f18737fe640 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01deab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 -- 172.21.15.73:0/3578088575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 msgr2=0x55ee01deab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01deab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 -- 172.21.15.73:0/3578088575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 msgr2=0x55ee01de3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01de3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.948+0000 7f1873fff640 1 -- 172.21.15.73:0/3578088575 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee01de88c0 con 0x55ee01df3350 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.949+0000 7f1872ffd640 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01de3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.949+0000 7f1873fff640 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186c09bed0 tx=0x7f186c09ba20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aae813c3a9357d82 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.949+0000 7f18597fa640 1 -- 172.21.15.73:0/3578088575 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c0b4070 con 0x55ee01df3350 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.949+0000 7f18597fa640 1 -- 172.21.15.73:0/3578088575 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f186c0a4c00 con 0x55ee01df3350 2024-06-06T07:19:53.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.949+0000 7f18597fa640 1 -- 172.21.15.73:0/3578088575 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f186c077630 con 0x55ee01df3350 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 -- 172.21.15.73:0/3578088575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 msgr2=0x55ee01e0b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f186c09bed0 tx=0x7f186c09ba20 comp rx=0 tx=0).stop 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 -- 172.21.15.73:0/3578088575 shutdown_connections 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01e0b200 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8a90 0x55ee01de3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 --2- 172.21.15.73:0/3578088575 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8170 0x55ee01deab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 -- 172.21.15.73:0/3578088575 >> 172.21.15.73:0/3578088575 conn(0x55ee01de4680 msgr2=0x55ee01de2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:53.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 -- 172.21.15.73:0/3578088575 shutdown_connections 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.950+0000 7f187cdfe740 1 -- 172.21.15.73:0/3578088575 wait complete. 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 -- start start 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 Processor -- start 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8a90 0x55ee01e6f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f18741b2490 con 0x55ee01de8170 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f18741b1dc0 con 0x55ee01de8a90 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f18741af020 con 0x55ee01df3350 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f187cdfe740 10 client.0 _ll_register_callbacks cb 0x7f186c09c690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:19:53.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f18737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f18737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58032/0 (socket says 172.21.15.73:58032) 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f18737fe640 1 -- 172.21.15.73:0/571241518 learned_addr learned my addr 172.21.15.73:0/571241518 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.951+0000 7f1873fff640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f1872ffd640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8a90 0x55ee01e6f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 -- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 msgr2=0x55ee01df6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 -- 172.21.15.73:0/571241518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8a90 msgr2=0x55ee01e6f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8a90 0x55ee01e6f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 -- 172.21.15.73:0/571241518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee01df6f60 con 0x55ee01de8170 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f18737fe640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1864005c40 tx=0x7f1864009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada37ac2d91604ab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:19:53.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f1852ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:19:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.952+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1864055070 con 0x55ee01de8170 2024-06-06T07:19:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(131..131 src has 1..131) ==== 9554+0+0 (secure 0 0 0) 0x7f1864003bd0 con 0x55ee01de8170 2024-06-06T07:19:53.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f186c0a4c00 con 0x55ee01de8170 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f186405e050 con 0x55ee01de8170 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 10 client.17358 fetch_fsmap learned FSMap version 74 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55ee01e76490 con 0x55ee01de8170 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f186c0a4c00 con 0x55ee01de8170 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f1864058020 con 0x55ee01de8170 2024-06-06T07:19:53.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 10 client.17358 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 10 client.17358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 20 client.17358 populate_metadata read hostname 'smithi073' 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e767d0 0x55ee01e78bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 4 client.17358 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.953+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee01d59c20 con 0x55ee01e767d0 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.954+0000 7f1872ffd640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e767d0 0x55ee01e78bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.954+0000 7f1872ffd640 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e767d0 0x55ee01e78bc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=836f0fbed32b5aed server_cookie=5a4d4662a55c64d8 in_seq=0 out_seq=0 2024-06-06T07:19:53.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:53.954+0000 7f1852ffd640 10 client.17358 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:54.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.047+0000 7f1852ffd640 1 -- 172.21.15.73:0/571241518 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ee01d59c20 con 0x55ee01e767d0 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.047+0000 7f1852ffd640 10 client.17358 handle_command_reply: tid=0 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.047+0000 7f187cdfe740 1 client.17358 shutdown 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.047+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 msgr2=0x55ee01e6ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.047+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f1864005c40 tx=0x7f1864009260 comp rx=0 tx=0).stop 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 shutdown_connections 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ee01e767d0 0x55ee01e78bc0 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f1852ffd640 0 client.17358 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ee01df3350 0x55ee01df6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ee01de8a90 0x55ee01e6f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 --2- 172.21.15.73:0/571241518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ee01de8170 0x55ee01e6ece0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 >> 172.21.15.73:0/571241518 conn(0x55ee01de4680 msgr2=0x55ee01de37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 shutdown_connections 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 1 -- 172.21.15.73:0/571241518 wait complete. 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187cdfe740 20 client.17358 trim_cache size 0 max 16384 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:19:54.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 msgr2=0x7f185c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 0x7f185c0a5260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1874105070 tx=0x7f186800eca0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 msgr2=0x7f187419f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.048+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f186c009fd0 tx=0x7f186c00ef10 comp rx=0 tx=0).stop 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 shutdown_connections 2024-06-06T07:19:54.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f187419ec00 0x7f187419f0b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f1874105330 0x7f187419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f185c0a2da0 0x7f185c0a5260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 --2- 172.21.15.73:0/2720451673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f1874100260 0x7f187419a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 >> 172.21.15.73:0/2720451673 conn(0x7f1874076e30 msgr2=0x7f1874103a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 shutdown_connections 2024-06-06T07:19:54.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:19:54.049+0000 7f187a0f5640 1 -- 172.21.15.73:0/2720451673 wait complete. 2024-06-06T07:19:54.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50564 9.06 MB/sec execute 179 sec latency 140.202 ms 2024-06-06T07:19:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:55 smithi170 ceph-mon[36108]: pgmap v589: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-06T07:19:55.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:55 smithi073 ceph-mon[32303]: pgmap v589: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-06T07:19:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:55 smithi136 ceph-mon[35901]: pgmap v589: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 421 op/s 2024-06-06T07:19:55.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50749 9.06 MB/sec execute 180 sec latency 274.561 ms 2024-06-06T07:19:56.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 50919 9.06 MB/sec execute 181 sec latency 108.652 ms 2024-06-06T07:19:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:57 smithi170 ceph-mon[36108]: pgmap v590: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-06-06T07:19:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:57 smithi073 ceph-mon[32303]: pgmap v590: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-06-06T07:19:57.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:57 smithi136 ceph-mon[35901]: pgmap v590: 129 pgs: 129 active+clean; 846 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.3 MiB/s wr, 266 op/s 2024-06-06T07:19:57.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51063 9.05 MB/sec execute 182 sec latency 78.287 ms 2024-06-06T07:19:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:58 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:19:58.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:58 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:19:58.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:58 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:19:58.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51217 9.05 MB/sec execute 183 sec latency 85.230 ms 2024-06-06T07:19:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:59 smithi170 ceph-mon[36108]: pgmap v591: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 410 op/s 2024-06-06T07:19:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:59 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:19:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:59 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:19:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:19:59 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:19:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:59 smithi073 ceph-mon[32303]: pgmap v591: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 410 op/s 2024-06-06T07:19:59.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:59 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:19:59.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:59 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:19:59.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:19:59 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:19:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:59 smithi136 ceph-mon[35901]: pgmap v591: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.7 MiB/s wr, 410 op/s 2024-06-06T07:19:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:59 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:19:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:59 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:19:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:19:59 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:19:59.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51375 9.07 MB/sec execute 184 sec latency 73.704 ms 2024-06-06T07:20:00.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:19:59.999+0000 7f12df37d700 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:20:00.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:00 smithi170 ceph-mon[36108]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:20:00.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:00 smithi073 ceph-mon[32303]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:20:00.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:00 smithi136 ceph-mon[35901]: overall HEALTH_ERR 1 MDSs report damaged metadata 2024-06-06T07:20:00.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51519 9.05 MB/sec execute 185 sec latency 72.287 ms 2024-06-06T07:20:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:01 smithi170 ceph-mon[36108]: pgmap v592: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:01.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:01 smithi073 ceph-mon[32303]: pgmap v592: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:01 smithi136 ceph-mon[35901]: pgmap v592: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:01.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51712 9.08 MB/sec execute 186 sec latency 101.944 ms 2024-06-06T07:20:02.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 51904 9.07 MB/sec execute 187 sec latency 78.426 ms 2024-06-06T07:20:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:03 smithi170 ceph-mon[36108]: pgmap v593: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:03.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:03 smithi073 ceph-mon[32303]: pgmap v593: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:03 smithi136 ceph-mon[35901]: pgmap v593: 129 pgs: 129 active+clean; 855 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:03.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52041 9.06 MB/sec execute 188 sec latency 90.101 ms 2024-06-06T07:20:04.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52167 9.02 MB/sec execute 189 sec latency 152.082 ms 2024-06-06T07:20:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:05 smithi170 ceph-mon[36108]: pgmap v594: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-06-06T07:20:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:05 smithi170 ceph-mon[36108]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T07:20:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:05 smithi073 ceph-mon[32303]: pgmap v594: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-06-06T07:20:05.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:05 smithi073 ceph-mon[32303]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T07:20:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:05 smithi136 ceph-mon[35901]: pgmap v594: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.2 MiB/s wr, 424 op/s 2024-06-06T07:20:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:05 smithi136 ceph-mon[35901]: osdmap e132: 12 total, 12 up, 12 in 2024-06-06T07:20:05.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52340 9.04 MB/sec execute 190 sec latency 195.468 ms 2024-06-06T07:20:06.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52502 9.03 MB/sec execute 191 sec latency 79.460 ms 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.189+0000 7fd78d748640 1 -- 172.21.15.73:0/1461774669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881010d0 msgr2=0x7fd78810de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 --2- 172.21.15.73:0/1461774669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881010d0 0x7fd78810de20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009860 tx=0x7fd77c02a340 comp rx=0 tx=0).stop 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 -- 172.21.15.73:0/1461774669 shutdown_connections 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 --2- 172.21.15.73:0/1461774669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd788110750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 --2- 172.21.15.73:0/1461774669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881010d0 0x7fd78810de20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 --2- 172.21.15.73:0/1461774669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881007b0 0x7fd788100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 -- 172.21.15.73:0/1461774669 >> 172.21.15.73:0/1461774669 conn(0x7fd788077a20 msgr2=0x7fd788077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 -- 172.21.15.73:0/1461774669 shutdown_connections 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 -- 172.21.15.73:0/1461774669 wait complete. 2024-06-06T07:20:07.191 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 Processor -- start 2024-06-06T07:20:07.192 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.190+0000 7fd78d748640 1 -- start start 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 0x7fd78819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd78819ea00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd788113e40 con 0x7fd7881010d0 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd788113cc0 con 0x7fd7881007b0 2024-06-06T07:20:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd78d748640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd788113fc0 con 0x7fd78810e360 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 0x7fd78819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:42082/0 (socket says 172.21.15.73:42082) 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd786ffd640 1 -- 172.21.15.73:0/2604148445 learned_addr learned my addr 172.21.15.73:0/2604148445 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7877fe640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd78819ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 -- 172.21.15.73:0/2604148445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 msgr2=0x7fd78819ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd78819ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 -- 172.21.15.73:0/2604148445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 msgr2=0x7fd78819a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7867fc640 1 -- 172.21.15.73:0/2604148445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd78819f130 con 0x7fd7881010d0 2024-06-06T07:20:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd786ffd640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.191+0000 7fd7877fe640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd78819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd7867fc640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 0x7fd78819a680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd77c02a860 tx=0x7fd77c030dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd76ffff640 1 -- 172.21.15.73:0/2604148445 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c037050 con 0x7fd7881010d0 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd76ffff640 1 -- 172.21.15.73:0/2604148445 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77c043070 con 0x7fd7881010d0 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd78819f3c0 con 0x7fd7881010d0 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd76ffff640 1 -- 172.21.15.73:0/2604148445 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c003ca0 con 0x7fd7881010d0 2024-06-06T07:20:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.192+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7881a6cb0 con 0x7fd7881010d0 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.195+0000 7fd79044c740 1 Processor -- start 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.195+0000 7fd76ffff640 1 -- 172.21.15.73:0/2604148445 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd77c004040 con 0x7fd7881010d0 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.195+0000 7fd79044c740 1 -- start start 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e808550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e815740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.197 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a0e76c4f0 con 0x562a0e808a90 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a0e70a870 con 0x562a0e808170 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd79044c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a0e7684b0 con 0x562a0e813350 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59606/0 (socket says 172.21.15.73:59606) 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd7877fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e808550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 -- 172.21.15.73:0/4191168881 learned_addr learned my addr 172.21.15.73:0/4191168881 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd7867fc640 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e815740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 -- 172.21.15.73:0/4191168881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 msgr2=0x562a0e815740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e815740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 -- 172.21.15.73:0/4191168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 msgr2=0x562a0e808550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e808550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd786ffd640 1 -- 172.21.15.73:0/4191168881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0e7269b0 con 0x562a0e808a90 2024-06-06T07:20:07.198 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.196+0000 7fd7877fe640 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e808550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:07.199 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd76ffff640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 0x7fd7700a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd7877fe640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 0x7fd7700a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd76ffff640 1 -- 172.21.15.73:0/2604148445 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd77c0eff50 con 0x7fd7881010d0 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd786ffd640 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd7780115c0 tx=0x7fd778011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ebe9006e6ef08f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd76d7fa640 1 -- 172.21.15.73:0/4191168881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77800f6c0 con 0x562a0e808a90 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd76d7fa640 1 -- 172.21.15.73:0/4191168881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77800fd10 con 0x562a0e808a90 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.197+0000 7fd76d7fa640 1 -- 172.21.15.73:0/4191168881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778019df0 con 0x562a0e808a90 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.198+0000 7fd7877fe640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 0x7fd7700a52e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd780009fd0 tx=0x7fd780009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.198+0000 7fd79044c740 1 -- 172.21.15.73:0/4191168881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 msgr2=0x562a0e812e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.198+0000 7fd79044c740 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd7780115c0 tx=0x7fd778011a80 comp rx=0 tx=0).stop 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 -- 172.21.15.73:0/4191168881 shutdown_connections 2024-06-06T07:20:07.203 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e815740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e812e10 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 --2- 172.21.15.73:0/4191168881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e808550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 -- 172.21.15.73:0/4191168881 >> 172.21.15.73:0/4191168881 conn(0x562a0e804680 msgr2=0x562a0e804a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 -- 172.21.15.73:0/4191168881 shutdown_connections 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 -- 172.21.15.73:0/4191168881 wait complete. 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.199+0000 7fd79044c740 1 -- start start 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 Processor -- start 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e810250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562a0e815f80 con 0x562a0e808a90 2024-06-06T07:20:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562a0e6f5ae0 con 0x562a0e808170 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562a0e816100 con 0x562a0e813350 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd79044c740 10 client.0 _ll_register_callbacks cb 0x562a0e72e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59618/0 (socket says 172.21.15.73:59618) 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 -- 172.21.15.73:0/3558293002 learned_addr learned my addr 172.21.15.73:0/3558293002 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7877fe640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd786ffd640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e810250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 -- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 msgr2=0x562a0e810cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 -- 172.21.15.73:0/3558293002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 msgr2=0x562a0e810250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e810250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.200+0000 7fd7867fc640 1 -- 172.21.15.73:0/3558293002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0e480210 con 0x562a0e808a90 2024-06-06T07:20:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd7867fc640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0b8af0 tx=0x7fd77c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54ac5eb0b03630 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd76aff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0bbc60 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd77c0b8790 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77800fd10 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fd77c0bba80 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd79044c740 10 client.17370 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x562a0e54c620 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.201+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77800fd10 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fd77c0f20e0 con 0x562a0e808a90 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 10 client.17370 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 10 client.17370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 20 client.17370 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:07.206 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e825860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 4 client.17370 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:20:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a0e77ff10 con 0x562a0e8233d0 2024-06-06T07:20:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd786ffd640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e825860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd76aff5640 10 client.17370 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:07.207 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.202+0000 7fd786ffd640 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e825860 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48263a0c6ac27db8 server_cookie=160112ba68e531a2 in_seq=0 out_seq=0 2024-06-06T07:20:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd76aff5640 1 -- 172.21.15.73:0/3558293002 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562a0e77ff10 con 0x562a0e8233d0 2024-06-06T07:20:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd76aff5640 10 client.17370 handle_command_reply: tid=0 2024-06-06T07:20:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 client.17370 shutdown 2024-06-06T07:20:07.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 msgr2=0x562a0e810790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0b8af0 tx=0x7fd77c005bd0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 shutdown_connections 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e825860 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd76aff5640 0 client.17370 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e810790 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 --2- 172.21.15.73:0/3558293002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e810250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.203+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 >> 172.21.15.73:0/3558293002 conn(0x562a0e804680 msgr2=0x562a0e80a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.204+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 shutdown_connections 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.204+0000 7fd79044c740 1 -- 172.21.15.73:0/3558293002 wait complete. 2024-06-06T07:20:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.204+0000 7fd79044c740 20 client.17370 trim_cache size 0 max 16384 2024-06-06T07:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 Processor -- start 2024-06-06T07:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 -- start start 2024-06-06T07:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e82b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.227 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881b0fe0 con 0x562a0e808a90 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881b2b20 con 0x562a0e808170 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd79044c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881b16b0 con 0x562a0e813350 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59634/0 (socket says 172.21.15.73:59634) 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd7867fc640 1 -- 172.21.15.73:0/420261340 learned_addr learned my addr 172.21.15.73:0/420261340 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd7877fe640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.226+0000 7fd786ffd640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e82b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 -- 172.21.15.73:0/420261340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 msgr2=0x562a0e810210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 -- 172.21.15.73:0/420261340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 msgr2=0x562a0e82b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e82b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 -- 172.21.15.73:0/420261340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a0e803f60 con 0x562a0e808a90 2024-06-06T07:20:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd786ffd640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e82b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:07.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd7867fc640 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009830 tx=0x7fd77c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c03c5ef47618697 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd76cff9640 1 -- 172.21.15.73:0/420261340 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0bbc60 con 0x562a0e808a90 2024-06-06T07:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd76cff9640 1 -- 172.21.15.73:0/420261340 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77c038040 con 0x562a0e808a90 2024-06-06T07:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.227+0000 7fd76cff9640 1 -- 172.21.15.73:0/420261340 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0bc120 con 0x562a0e808a90 2024-06-06T07:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- 172.21.15.73:0/420261340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 msgr2=0x562a0e82b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fd77c009830 tx=0x7fd77c0058d0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- 172.21.15.73:0/420261340 shutdown_connections 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e813350 0x562a0e810210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e82b620 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 --2- 172.21.15.73:0/420261340 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e808170 0x562a0e82b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- 172.21.15.73:0/420261340 >> 172.21.15.73:0/420261340 conn(0x562a0e804680 msgr2=0x562a0e80b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- 172.21.15.73:0/420261340 shutdown_connections 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- 172.21.15.73:0/420261340 wait complete. 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.228+0000 7fd79044c740 1 -- start start 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 Processor -- start 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 0x562a0e817270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881b2450 con 0x562a0e808a90 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881b1d80 con 0x562a0e813350 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd7881aefe0 con 0x562a0e808170 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd79044c740 10 client.0 _ll_register_callbacks cb 0x7fd77c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:54846/0 (socket says 172.21.15.73:54846) 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 -- 172.21.15.73:0/1467127487 learned_addr learned my addr 172.21.15.73:0/1467127487 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd7867fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59640/0 (socket says 172.21.15.73:59640) 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd7877fe640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 0x562a0e817270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 -- 172.21.15.73:0/1467127487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 msgr2=0x562a0e817270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 0x562a0e817270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 -- 172.21.15.73:0/1467127487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 msgr2=0x562a0e816d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd786ffd640 1 -- 172.21.15.73:0/1467127487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a0e817910 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd7867fc640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.229+0000 7fd7877fe640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 0x562a0e817270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd786ffd640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd7780115c0 tx=0x7fd778009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2701d7d3382e57a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd76aff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd778009e10 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 9642+0+0 (secure 0 0 0) 0x7fd778016040 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd77c038040 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fd778019410 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd79044c740 10 client.26788 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x562a0e88df70 con 0x562a0e808170 2024-06-06T07:20:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.230+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd77c038040 con 0x562a0e808170 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fd77800f6c0 con 0x562a0e808170 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 10 client.26788 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 10 client.26788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 20 client.26788 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e890590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 4 client.26788 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a0e779c20 con 0x562a0e8233d0 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd7867fc640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e890590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd76aff5640 10 client.26788 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:07.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.231+0000 7fd7867fc640 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e890590 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58487aef9bdd78e2 server_cookie=73cbe5cb4f894e59 in_seq=0 out_seq=0 2024-06-06T07:20:07.305 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.304+0000 7fd76aff5640 1 -- 172.21.15.73:0/1467127487 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562a0e779c20 con 0x562a0e8233d0 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.304+0000 7fd76aff5640 10 client.26788 handle_command_reply: tid=0 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.304+0000 7fd79044c740 1 client.26788 shutdown 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 msgr2=0x562a0e81b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd7780115c0 tx=0x7fd778009260 comp rx=0 tx=0).stop 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 shutdown_connections 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x562a0e8233d0 0x562a0e890590 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562a0e813350 0x562a0e817270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562a0e808a90 0x562a0e816d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 --2- 172.21.15.73:0/1467127487 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562a0e808170 0x562a0e81b970 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 >> 172.21.15.73:0/1467127487 conn(0x562a0e804680 msgr2=0x562a0e811a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.306 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd76aff5640 0 client.26788 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 shutdown_connections 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 1 -- 172.21.15.73:0/1467127487 wait complete. 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.305+0000 7fd79044c740 20 client.26788 trim_cache size 0 max 16384 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:20:07.307 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 msgr2=0x7fd7700a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 0x7fd7700a52e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd780009fd0 tx=0x7fd780009500 comp rx=0 tx=0).stop 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 msgr2=0x7fd78819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 0x7fd78819a680 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fd77c02a860 tx=0x7fd77c030dc0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 shutdown_connections 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd7700a2e20 0x7fd7700a52e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.308 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd78810e360 0x7fd78819ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7881010d0 0x7fd78819a680 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 --2- 172.21.15.73:0/2604148445 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd7881007b0 0x7fd78819a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.306+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 >> 172.21.15.73:0/2604148445 conn(0x7fd788077a20 msgr2=0x7fd78810f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.307+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 shutdown_connections 2024-06-06T07:20:07.309 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:07.307+0000 7fd78d748640 1 -- 172.21.15.73:0/2604148445 wait complete. 2024-06-06T07:20:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:07 smithi170 ceph-mon[36108]: pgmap v596: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-06-06T07:20:07.602 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:07 smithi136 ceph-mon[35901]: pgmap v596: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-06-06T07:20:07.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:07 smithi073 ceph-mon[32303]: pgmap v596: 129 pgs: 129 active+clean; 870 MiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 356 op/s 2024-06-06T07:20:07.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52665 9.01 MB/sec execute 192 sec latency 88.455 ms 2024-06-06T07:20:08.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 52828 9.02 MB/sec execute 193 sec latency 59.463 ms 2024-06-06T07:20:09.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:09 smithi170 ceph-mon[36108]: pgmap v597: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:09.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:09 smithi073 ceph-mon[32303]: pgmap v597: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:09 smithi136 ceph-mon[35901]: pgmap v597: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:09.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53008 9.03 MB/sec execute 194 sec latency 111.379 ms 2024-06-06T07:20:10.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53211 9.05 MB/sec execute 195 sec latency 57.372 ms 2024-06-06T07:20:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:11 smithi170 ceph-mon[36108]: pgmap v598: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:11.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:11 smithi073 ceph-mon[32303]: pgmap v598: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:11 smithi136 ceph-mon[35901]: pgmap v598: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:11.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53369 9.05 MB/sec execute 196 sec latency 67.558 ms 2024-06-06T07:20:12.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53526 9.04 MB/sec execute 197 sec latency 148.208 ms 2024-06-06T07:20:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:13 smithi073 ceph-mon[32303]: pgmap v599: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:13.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:13 smithi073 ceph-mon[32303]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T07:20:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:13 smithi136 ceph-mon[35901]: pgmap v599: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:13 smithi136 ceph-mon[35901]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T07:20:13.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53701 9.05 MB/sec execute 198 sec latency 139.854 ms 2024-06-06T07:20:13.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:13 smithi170 ceph-mon[36108]: pgmap v599: 129 pgs: 129 active+clean; 866 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 363 op/s 2024-06-06T07:20:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:13 smithi170 ceph-mon[36108]: osdmap e133: 12 total, 12 up, 12 in 2024-06-06T07:20:14.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 53864 9.05 MB/sec execute 199 sec latency 74.289 ms 2024-06-06T07:20:15.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:15 smithi073 ceph-mon[32303]: pgmap v601: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 354 op/s 2024-06-06T07:20:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:15 smithi136 ceph-mon[35901]: pgmap v601: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 354 op/s 2024-06-06T07:20:15.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54031 9.06 MB/sec execute 200 sec latency 91.579 ms 2024-06-06T07:20:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:15 smithi170 ceph-mon[36108]: pgmap v601: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 354 op/s 2024-06-06T07:20:16.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54180 9.05 MB/sec execute 201 sec latency 67.205 ms 2024-06-06T07:20:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:17 smithi073 ceph-mon[32303]: pgmap v602: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 342 op/s 2024-06-06T07:20:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:17 smithi136 ceph-mon[35901]: pgmap v602: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 342 op/s 2024-06-06T07:20:17.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54317 9.04 MB/sec execute 202 sec latency 200.456 ms 2024-06-06T07:20:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:17 smithi170 ceph-mon[36108]: pgmap v602: 129 pgs: 129 active+clean; 885 MiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 342 op/s 2024-06-06T07:20:18.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54484 9.02 MB/sec execute 203 sec latency 66.484 ms 2024-06-06T07:20:19.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:19 smithi073 ceph-mon[32303]: pgmap v603: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:19.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:19 smithi136 ceph-mon[35901]: pgmap v603: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:19.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54622 9.00 MB/sec execute 204 sec latency 124.450 ms 2024-06-06T07:20:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:19 smithi170 ceph-mon[36108]: pgmap v603: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.446+0000 7ff37cabd640 1 -- 172.21.15.73:0/316931316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 msgr2=0x7ff3781089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.446+0000 7ff37cabd640 1 --2- 172.21.15.73:0/316931316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781089a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff370009cd0 tx=0x7ff37002a9a0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.447+0000 7ff37cabd640 1 -- 172.21.15.73:0/316931316 shutdown_connections 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.447+0000 7ff37cabd640 1 --2- 172.21.15.73:0/316931316 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff37811b180 0x7ff37811d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.447+0000 7ff37cabd640 1 --2- 172.21.15.73:0/316931316 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff378110570 0x7ff378108ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.447+0000 7ff37cabd640 1 --2- 172.21.15.73:0/316931316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781089a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.447+0000 7ff37cabd640 1 -- 172.21.15.73:0/316931316 >> 172.21.15.73:0/316931316 conn(0x7ff37806ca70 msgr2=0x7ff37806ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.448+0000 7ff37cabd640 1 -- 172.21.15.73:0/316931316 shutdown_connections 2024-06-06T07:20:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.449+0000 7ff37cabd640 1 -- 172.21.15.73:0/316931316 wait complete. 2024-06-06T07:20:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.449+0000 7ff37cabd640 1 Processor -- start 2024-06-06T07:20:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 -- start start 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37811b180 0x7ff3781a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3781ab840 0x7ff3781abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff378120ba0 con 0x7ff378110190 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff378120a20 con 0x7ff3781ab840 2024-06-06T07:20:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.451+0000 7ff37cabd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff378120d20 con 0x7ff37811b180 2024-06-06T07:20:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54406/0 (socket says 172.21.15.73:54406) 2024-06-06T07:20:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 -- 172.21.15.73:0/1730433864 learned_addr learned my addr 172.21.15.73:0/1730433864 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff375d9b640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37811b180 0x7ff3781a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 -- 172.21.15.73:0/1730433864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37811b180 msgr2=0x7ff3781a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37811b180 0x7ff3781a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 -- 172.21.15.73:0/1730433864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3781ab840 msgr2=0x7ff3781abcf0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3781ab840 0x7ff3781abcf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.452+0000 7ff37659c640 1 -- 172.21.15.73:0/1730433864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3781ac330 con 0x7ff378110190 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff37659c640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff370009430 tx=0x7ff37002fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff36f7fe640 1 -- 172.21.15.73:0/1730433864 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370034030 con 0x7ff378110190 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff36f7fe640 1 -- 172.21.15.73:0/1730433864 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff37002feb0 con 0x7ff378110190 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3781ac5c0 con 0x7ff378110190 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff36f7fe640 1 -- 172.21.15.73:0/1730433864 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370039680 con 0x7ff378110190 2024-06-06T07:20:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.453+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3781b3d80 con 0x7ff378110190 2024-06-06T07:20:20.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.457+0000 7ff36f7fe640 1 -- 172.21.15.73:0/1730433864 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff370037020 con 0x7ff378110190 2024-06-06T07:20:20.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.457+0000 7ff37f7cf740 1 Processor -- start 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 -- start start 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff36f7fe640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 0x7ff3640a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7bce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bf740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f7164f0 con 0x55dd8f7b2a90 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f6b4870 con 0x55dd8f7bd350 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f7124b0 con 0x55dd8f7b2170 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35332/0 (socket says 172.21.15.73:35332) 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff37659c640 1 -- 172.21.15.73:0/217907364 learned_addr learned my addr 172.21.15.73:0/217907364 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff375d9b640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7bce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff376d9d640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bf740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.458+0000 7ff376d9d640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 0x7ff3640a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 -- 172.21.15.73:0/217907364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 msgr2=0x55dd8f7b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 -- 172.21.15.73:0/217907364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 msgr2=0x55dd8f7bf740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bf740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 -- 172.21.15.73:0/217907364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8f6d09b0 con 0x55dd8f7b2a90 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff37659c640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff375d9b640 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7bce10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff37810f460 tx=0x7ff36000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bc783b0601129bb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff36f7fe640 1 -- 172.21.15.73:0/1730433864 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7ff3700f07c0 con 0x7ff378110190 2024-06-06T07:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.459+0000 7ff376d9d640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 0x7ff3640a53c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55dd8f7afaa0 tx=0x7ff3680068f0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.460+0000 7ff36cff9640 1 -- 172.21.15.73:0/217907364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36000be30 con 0x55dd8f7b2a90 2024-06-06T07:20:20.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.460+0000 7ff36cff9640 1 -- 172.21.15.73:0/217907364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff360019070 con 0x55dd8f7b2a90 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.460+0000 7ff36cff9640 1 -- 172.21.15.73:0/217907364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff360014db0 con 0x55dd8f7b2a90 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 -- 172.21.15.73:0/217907364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 msgr2=0x55dd8f7bce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7bce10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff37810f460 tx=0x7ff36000bbe0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 -- 172.21.15.73:0/217907364 shutdown_connections 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bf740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7bce10 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/217907364 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 -- 172.21.15.73:0/217907364 >> 172.21.15.73:0/217907364 conn(0x55dd8f7ae680 msgr2=0x55dd8f7aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 -- 172.21.15.73:0/217907364 shutdown_connections 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.461+0000 7ff37f7cf740 1 -- 172.21.15.73:0/217907364 wait complete. 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.462+0000 7ff37f7cf740 1 -- start start 2024-06-06T07:20:20.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.462+0000 7ff37f7cf740 1 Processor -- start 2024-06-06T07:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7ba250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7ba790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7ba250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7ba250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35344/0 (socket says 172.21.15.73:35344) 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff375d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7ba790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37659c640 1 -- 172.21.15.73:0/966922667 learned_addr learned my addr 172.21.15.73:0/966922667 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff376d9d640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f7bff80 con 0x55dd8f7b2a90 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f69fae0 con 0x55dd8f7bd350 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.463+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55dd8f7c0100 con 0x55dd8f7b2170 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff37f7cf740 10 client.0 _ll_register_callbacks cb 0x55dd8f6d89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 -- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 msgr2=0x55dd8f7ba250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7ba250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 -- 172.21.15.73:0/966922667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 msgr2=0x55dd8f7ba790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7ba790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8f42a210 con 0x55dd8f7bd350 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff375d9b640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:20.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff376d9d640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bacd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff368008fd0 tx=0x7ff368006880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5b21ff22bd91b2a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff360019070 con 0x55dd8f7bd350 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff35dffb640 10 client.26615 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.464+0000 7ff35dffb640 1 -- 172.21.15.73:0/966922667 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff368027060 con 0x55dd8f7bd350 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.465+0000 7ff35dffb640 1 -- 172.21.15.73:0/966922667 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7ff368022470 con 0x55dd8f7bd350 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.465+0000 7ff35dffb640 1 -- 172.21.15.73:0/966922667 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ff36802e050 con 0x55dd8f7bd350 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.466+0000 7ff37f7cf740 10 client.26615 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:20.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.466+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=74}) -- 0x55dd8f4f6620 con 0x55dd8f7bd350 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff35dffb640 1 -- 172.21.15.73:0/966922667 <== mon.2 v2:172.21.15.136:3300/0 4 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7ff368022670 con 0x55dd8f7bd350 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff37f7cf740 10 client.26615 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff37f7cf740 10 client.26615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff37f7cf740 20 client.26615 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f7cf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.468+0000 7ff37f7cf740 4 client.26615 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:20:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.469+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff360019070 con 0x55dd8f7cd140 2024-06-06T07:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.469+0000 7ff37659c640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f7cf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.470+0000 7ff37659c640 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f7cf530 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e977fbdce294a680 server_cookie=c0f040bcb6968262 in_seq=0 out_seq=0 2024-06-06T07:20:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.470+0000 7ff35dffb640 10 client.26615 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff35dffb640 1 -- 172.21.15.73:0/966922667 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff360019070 con 0x55dd8f7cd140 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff35dffb640 10 client.26615 handle_command_reply: tid=0 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 client.26615 shutdown 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 msgr2=0x55dd8f7bacd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bacd0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff368008fd0 tx=0x7ff368006880 comp rx=0 tx=0).stop 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 shutdown_connections 2024-06-06T07:20:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f7cf530 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff35dffb640 0 client.26615 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7bd350 0x55dd8f7bacd0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7ba790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/966922667 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7ba250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.473+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 >> 172.21.15.73:0/966922667 conn(0x55dd8f7ae680 msgr2=0x55dd8f7b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.474+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 shutdown_connections 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.474+0000 7ff37f7cf740 1 -- 172.21.15.73:0/966922667 wait complete. 2024-06-06T07:20:20.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.474+0000 7ff37f7cf740 20 client.26615 trim_cache size 0 max 16384 2024-06-06T07:20:20.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 Processor -- start 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 -- start start 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7baaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 0x55dd8f7d4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781bdc10 con 0x55dd8f7bd350 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.502+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781bf750 con 0x55dd8f7b2a90 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:35348/0 (socket says 172.21.15.73:35348) 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781be2e0 con 0x55dd8f7b2170 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 -- 172.21.15.73:0/3871019431 learned_addr learned my addr 172.21.15.73:0/3871019431 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff375d9b640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7baaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff376d9d640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 0x55dd8f7d4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 -- 172.21.15.73:0/3871019431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 msgr2=0x55dd8f7baaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7baaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 -- 172.21.15.73:0/3871019431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 msgr2=0x55dd8f7d4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 0x55dd8f7d4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 -- 172.21.15.73:0/3871019431 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd8f7b28c0 con 0x55dd8f7b2170 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff376d9d640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 0x55dd8f7d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.503+0000 7ff37659c640 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3700bc5d0 tx=0x7ff3700044e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f2a969b35613336f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.504+0000 7ff35ffff640 1 -- 172.21.15.73:0/3871019431 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3700bcfd0 con 0x55dd8f7b2170 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.504+0000 7ff35ffff640 1 -- 172.21.15.73:0/3871019431 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3700bc8d0 con 0x55dd8f7b2170 2024-06-06T07:20:20.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.504+0000 7ff35ffff640 1 -- 172.21.15.73:0/3871019431 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3700b8f40 con 0x55dd8f7b2170 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.504+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3871019431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 msgr2=0x55dd8f7b4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.504+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff3700bc5d0 tx=0x7ff3700044e0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3871019431 shutdown_connections 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7bd350 0x55dd8f7d4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f7baaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3871019431 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7b2170 0x55dd8f7b4b30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3871019431 >> 172.21.15.73:0/3871019431 conn(0x55dd8f7ae680 msgr2=0x55dd8f7bc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3871019431 shutdown_connections 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3871019431 wait complete. 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 -- start start 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.505+0000 7ff37f7cf740 1 Processor -- start 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f8304e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7bd350 0x55dd8f7c5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781bf080 con 0x55dd8f7b2170 2024-06-06T07:20:20.512 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781be9b0 con 0x55dd8f7b2a90 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3781bbc10 con 0x55dd8f7bd350 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37f7cf740 10 client.0 _ll_register_callbacks cb 0x7ff36800bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54436/0 (socket says 172.21.15.73:54436) 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff37659c640 1 -- 172.21.15.73:0/3206894345 learned_addr learned my addr 172.21.15.73:0/3206894345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff375d9b640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f8304e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.506+0000 7ff376d9d640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7bd350 0x55dd8f7c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 -- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7bd350 msgr2=0x55dd8f7c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7bd350 0x55dd8f7c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 -- 172.21.15.73:0/3206894345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 msgr2=0x55dd8f8304e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f8304e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 -- 172.21.15.73:0/3206894345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd8f7c5790 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37659c640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff3700bc860 tx=0x7ff370004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6076f992c453ae7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff35dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3700d4030 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7ff3700babd0 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.507+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3700bc8d0 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7ff3700bcfd0 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 10 client.17394 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x55dd8f837880 con 0x55dd8f7b2170 2024-06-06T07:20:20.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3700bc8d0 con 0x55dd8f7b2170 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7ff370041070 con 0x55dd8f7b2170 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 10 client.17394 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 10 client.17394 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 20 client.17394 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f839ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 4 client.17394 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd8f729f10 con 0x55dd8f7cd140 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.508+0000 7ff375d9b640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f839ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.509+0000 7ff35dffb640 10 client.17394 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:20.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.509+0000 7ff375d9b640 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f839ea0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5397e619e4a92ddc server_cookie=a344729938701fe3 in_seq=0 out_seq=0 2024-06-06T07:20:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff35dffb640 1 -- 172.21.15.73:0/3206894345 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dd8f729f10 con 0x55dd8f7cd140 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff35dffb640 10 client.17394 handle_command_reply: tid=0 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 client.17394 shutdown 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 msgr2=0x55dd8f82ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff3700bc860 tx=0x7ff370004980 comp rx=0 tx=0).stop 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 shutdown_connections 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55dd8f7cd140 0x55dd8f839ea0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55dd8f7bd350 0x55dd8f7c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff35dffb640 0 client.17394 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55dd8f7b2a90 0x55dd8f8304e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 --2- 172.21.15.73:0/3206894345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dd8f7b2170 0x55dd8f82ffa0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.597+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 >> 172.21.15.73:0/3206894345 conn(0x55dd8f7ae680 msgr2=0x55dd8f7c8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 shutdown_connections 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37f7cf740 1 -- 172.21.15.73:0/3206894345 wait complete. 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37f7cf740 20 client.17394 trim_cache size 0 max 16384 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:20:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:20:20.602 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 msgr2=0x7ff3640a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 0x7ff3640a53c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55dd8f7afaa0 tx=0x7ff3680068f0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 msgr2=0x7ff3781a6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.598+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff370009430 tx=0x7ff37002fcc0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 shutdown_connections 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3781ab840 0x7ff3781abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3640a2f00 0x7ff3640a53c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff37811b180 0x7ff3781a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 --2- 172.21.15.73:0/1730433864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff378110190 0x7ff3781a6f80 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:20.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 >> 172.21.15.73:0/1730433864 conn(0x7ff37806ca70 msgr2=0x7ff37810de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 shutdown_connections 2024-06-06T07:20:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:20.599+0000 7ff37cabd640 1 -- 172.21.15.73:0/1730433864 wait complete. 2024-06-06T07:20:20.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54790 8.99 MB/sec execute 205 sec latency 91.654 ms 2024-06-06T07:20:21.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:21 smithi073 ceph-mon[32303]: pgmap v604: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:21.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:21 smithi136 ceph-mon[35901]: pgmap v604: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:21.809 INFO:tasks.workunit.client.0.smithi073.stdout: 10 54945 8.99 MB/sec execute 206 sec latency 83.897 ms 2024-06-06T07:20:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:21 smithi170 ceph-mon[36108]: pgmap v604: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:22.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55114 8.99 MB/sec execute 207 sec latency 147.745 ms 2024-06-06T07:20:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:23 smithi073 ceph-mon[32303]: pgmap v605: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:23 smithi136 ceph-mon[35901]: pgmap v605: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:23.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55338 9.04 MB/sec execute 208 sec latency 626.631 ms 2024-06-06T07:20:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:23 smithi170 ceph-mon[36108]: pgmap v605: 129 pgs: 129 active+clean; 896 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.8 MiB/s wr, 318 op/s 2024-06-06T07:20:24.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55513 9.06 MB/sec execute 209 sec latency 58.889 ms 2024-06-06T07:20:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:25 smithi073 ceph-mon[32303]: pgmap v606: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.6 MiB/s wr, 431 op/s 2024-06-06T07:20:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:25 smithi136 ceph-mon[35901]: pgmap v606: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.6 MiB/s wr, 431 op/s 2024-06-06T07:20:25.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55654 9.04 MB/sec execute 210 sec latency 58.660 ms 2024-06-06T07:20:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:25 smithi170 ceph-mon[36108]: pgmap v606: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.6 MiB/s wr, 431 op/s 2024-06-06T07:20:26.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55845 9.05 MB/sec execute 211 sec latency 53.779 ms 2024-06-06T07:20:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:27 smithi073 ceph-mon[32303]: pgmap v607: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:20:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:27 smithi136 ceph-mon[35901]: pgmap v607: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:20:27.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 55988 9.03 MB/sec execute 212 sec latency 105.001 ms 2024-06-06T07:20:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:27 smithi170 ceph-mon[36108]: pgmap v607: 129 pgs: 129 active+clean; 899 MiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:20:28.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56146 9.02 MB/sec execute 213 sec latency 86.801 ms 2024-06-06T07:20:29.750 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:29 smithi170 ceph-mon[36108]: pgmap v608: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 407 op/s 2024-06-06T07:20:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:29 smithi073 ceph-mon[32303]: pgmap v608: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 407 op/s 2024-06-06T07:20:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:29 smithi136 ceph-mon[35901]: pgmap v608: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.8 MiB/s wr, 407 op/s 2024-06-06T07:20:29.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56304 9.01 MB/sec execute 214 sec latency 70.176 ms 2024-06-06T07:20:30.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56480 9.03 MB/sec execute 215 sec latency 98.061 ms 2024-06-06T07:20:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:31 smithi073 ceph-mon[32303]: pgmap v609: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:31 smithi136 ceph-mon[35901]: pgmap v609: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:31.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56667 9.04 MB/sec execute 216 sec latency 70.986 ms 2024-06-06T07:20:31.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:31 smithi170 ceph-mon[36108]: pgmap v609: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:32.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56820 9.02 MB/sec execute 217 sec latency 78.208 ms 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 -- 172.21.15.73:0/218602060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc434108200 msgr2=0x7fc4341085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 --2- 172.21.15.73:0/218602060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc434108200 0x7fc4341085e0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7fc430009900 tx=0x7fc43002a410 comp rx=0 tx=0).stop 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 -- 172.21.15.73:0/218602060 shutdown_connections 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 --2- 172.21.15.73:0/218602060 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc43410d2d0 0x7fc434114ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 --2- 172.21.15.73:0/218602060 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc434108bb0 0x7fc43410cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.729 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 --2- 172.21.15.73:0/218602060 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc434108200 0x7fc4341085e0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 -- 172.21.15.73:0/218602060 >> 172.21.15.73:0/218602060 conn(0x7fc434075f30 msgr2=0x7fc434078350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.728+0000 7fc43bdee640 1 -- 172.21.15.73:0/218602060 shutdown_connections 2024-06-06T07:20:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.729+0000 7fc43bdee640 1 -- 172.21.15.73:0/218602060 wait complete. 2024-06-06T07:20:33.730 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 Processor -- start 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 -- start start 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 0x7fc43419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4341a2fe0 0x7fc4341a3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc434118290 con 0x7fc43410d2d0 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc434118110 con 0x7fc4341a2fe0 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43bdee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc434118410 con 0x7fc434108200 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc439b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc439389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 0x7fc43419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc439b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:60638/0 (socket says 172.21.15.73:60638) 2024-06-06T07:20:33.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc439b8a640 1 -- 172.21.15.73:0/3151619825 learned_addr learned my addr 172.21.15.73:0/3151619825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:33.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.730+0000 7fc43a38b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4341a2fe0 0x7fc4341a3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 -- 172.21.15.73:0/3151619825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4341a2fe0 msgr2=0x7fc4341a3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4341a2fe0 0x7fc4341a3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 -- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 msgr2=0x7fc43419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 0x7fc43419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 -- 172.21.15.73:0/3151619825 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4341a3ad0 con 0x7fc434108200 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439389640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 0x7fc43419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc439b8a640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc430005ab0 tx=0x7fc430003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc42affd640 1 -- 172.21.15.73:0/3151619825 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc43003f070 con 0x7fc434108200 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4341a3d60 con 0x7fc434108200 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc42affd640 1 -- 172.21.15.73:0/3151619825 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc430031b60 con 0x7fc434108200 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4341ab580 con 0x7fc434108200 2024-06-06T07:20:33.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.731+0000 7fc42affd640 1 -- 172.21.15.73:0/3151619825 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc430035030 con 0x7fc434108200 2024-06-06T07:20:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.734+0000 7fc42affd640 1 -- 172.21.15.73:0/3151619825 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc430042070 con 0x7fc434108200 2024-06-06T07:20:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.734+0000 7fc43ebd8740 1 Processor -- start 2024-06-06T07:20:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 -- start start 2024-06-06T07:20:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416170 0x56176c416550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416a90 0x56176c420e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56176c37a4f0 con 0x56176c416170 2024-06-06T07:20:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56176c318870 con 0x56176c421350 2024-06-06T07:20:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56176c3764b0 con 0x56176c416a90 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc43a38b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416170 0x56176c416550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc439389640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc439389640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:49160/0 (socket says 172.21.15.73:49160) 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc439389640 1 -- 172.21.15.73:0/615774033 learned_addr learned my addr 172.21.15.73:0/615774033 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.735+0000 7fc439b8a640 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416a90 0x56176c420e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 -- 172.21.15.73:0/615774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416a90 msgr2=0x56176c420e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416a90 0x56176c420e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.737 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 -- 172.21.15.73:0/615774033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416170 msgr2=0x56176c416550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416170 0x56176c416550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 -- 172.21.15.73:0/615774033 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176c3349b0 con 0x56176c421350 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc439389640 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4240114f0 tx=0x7fc4240119b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f969b729316e56d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc42affd640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 0x7fc4200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc41ffff640 1 -- 172.21.15.73:0/615774033 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424005020 con 0x56176c421350 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc41ffff640 1 -- 172.21.15.73:0/615774033 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4240047a0 con 0x56176c421350 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc41ffff640 1 -- 172.21.15.73:0/615774033 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42400f840 con 0x56176c421350 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc42affd640 1 -- 172.21.15.73:0/3151619825 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc4300efe70 con 0x7fc434108200 2024-06-06T07:20:33.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.736+0000 7fc43a38b640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 0x7fc4200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 -- 172.21.15.73:0/615774033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 msgr2=0x56176c423740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43a38b640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 0x7fc4200a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005fd0 tx=0x7fc42c009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc4240114f0 tx=0x7fc4240119b0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 -- 172.21.15.73:0/615774033 shutdown_connections 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c423740 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416a90 0x56176c420e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/615774033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416170 0x56176c416550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 -- 172.21.15.73:0/615774033 >> 172.21.15.73:0/615774033 conn(0x56176c412680 msgr2=0x56176c412a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 -- 172.21.15.73:0/615774033 shutdown_connections 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.737+0000 7fc43ebd8740 1 -- 172.21.15.73:0/615774033 wait complete. 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 -- start start 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 Processor -- start 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 0x56176c41e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 0x56176c41ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56176c423f80 con 0x56176c421350 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56176c303ae0 con 0x56176c416a90 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56176c424100 con 0x56176c416170 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43ebd8740 10 client.0 _ll_register_callbacks cb 0x56176c33c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc439b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc439b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:60644/0 (socket says 172.21.15.73:60644) 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc43a38b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 0x56176c41ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc439b8a640 1 -- 172.21.15.73:0/151758569 learned_addr learned my addr 172.21.15.73:0/151758569 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.738+0000 7fc439389640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 0x56176c41e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 -- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 msgr2=0x56176c41e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 -- 172.21.15.73:0/151758569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 msgr2=0x56176c41e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 0x56176c41e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 -- 172.21.15.73:0/151758569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176c08e210 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc439b8a640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc439389640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 0x56176c41e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43a38b640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 0x56176c41ecb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc42c011dd0 tx=0x7fc42c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61e68b1145f8dc0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc41d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c004b20 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4240047a0 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc42c004cc0 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc42c00f810 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43ebd8740 10 client.17406 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.739+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x56176c15a620 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4240047a0 con 0x56176c421350 2024-06-06T07:20:33.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fc42c026070 con 0x56176c421350 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 10 client.17406 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 10 client.17406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 20 client.17406 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c433c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 4 client.17406 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176c38df10 con 0x56176c4317a0 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.740+0000 7fc439b8a640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c433c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.741+0000 7fc41d7fa640 10 client.17406 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:33.745 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.741+0000 7fc439b8a640 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c433c20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ef9141da982cf6 server_cookie=eb9f41591a7d328d in_seq=0 out_seq=0 2024-06-06T07:20:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.742+0000 7fc41d7fa640 1 -- 172.21.15.73:0/151758569 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56176c38df10 con 0x56176c4317a0 2024-06-06T07:20:33.746 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.742+0000 7fc41d7fa640 10 client.17406 handle_command_reply: tid=0 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.742+0000 7fc43ebd8740 1 client.17406 shutdown 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.742+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 msgr2=0x56176c41ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.742+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 0x56176c41ecb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc42c011dd0 tx=0x7fc42c004ab0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 shutdown_connections 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c433c20 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc41d7fa640 0 client.17406 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c421350 0x56176c41ecb0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416a90 0x56176c41e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/151758569 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c41e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 >> 172.21.15.73:0/151758569 conn(0x56176c412680 msgr2=0x56176c418400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 shutdown_connections 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 1 -- 172.21.15.73:0/151758569 wait complete. 2024-06-06T07:20:33.747 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.743+0000 7fc43ebd8740 20 client.17406 trim_cache size 0 max 16384 2024-06-06T07:20:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 Processor -- start 2024-06-06T07:20:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 -- start start 2024-06-06T07:20:33.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c4394f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c41e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc418010090 con 0x56176c416a90 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc418011bd0 con 0x56176c421350 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc418010760 con 0x56176c416170 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc439389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc439389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34296/0 (socket says 172.21.15.73:34296) 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc439389640 1 -- 172.21.15.73:0/1962567138 learned_addr learned my addr 172.21.15.73:0/1962567138 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc439b8a640 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c4394f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.768+0000 7fc43a38b640 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c41e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 -- 172.21.15.73:0/1962567138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 msgr2=0x56176c4394f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c4394f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 -- 172.21.15.73:0/1962567138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 msgr2=0x56176c41e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c41e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 -- 172.21.15.73:0/1962567138 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176c424100 con 0x56176c416a90 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc439389640 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4240114f0 tx=0x7fc424005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ef5ea31e9f1d4b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc41f7fe640 1 -- 172.21.15.73:0/1962567138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424027070 con 0x56176c416a90 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc41f7fe640 1 -- 172.21.15.73:0/1962567138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc424004c60 con 0x56176c416a90 2024-06-06T07:20:33.772 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.769+0000 7fc41f7fe640 1 -- 172.21.15.73:0/1962567138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42400f620 con 0x56176c416a90 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.770+0000 7fc43ebd8740 1 -- 172.21.15.73:0/1962567138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 msgr2=0x56176c439a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.770+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4240114f0 tx=0x7fc424005850 comp rx=0 tx=0).stop 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 -- 172.21.15.73:0/1962567138 shutdown_connections 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c421350 0x56176c41e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c439a30 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/1962567138 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c416170 0x56176c4394f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 -- 172.21.15.73:0/1962567138 >> 172.21.15.73:0/1962567138 conn(0x56176c412680 msgr2=0x56176c4171d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 -- 172.21.15.73:0/1962567138 shutdown_connections 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 -- 172.21.15.73:0/1962567138 wait complete. 2024-06-06T07:20:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 -- start start 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 Processor -- start 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.771+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416170 0x56176c4293b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c4298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 0x56176c41a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc418011500 con 0x56176c416a90 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc418010e30 con 0x56176c416170 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc41800e090 con 0x56176c421350 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43ebd8740 10 client.0 _ll_register_callbacks cb 0x7fc42c009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc439389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c4298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc439389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c4298f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34298/0 (socket says 172.21.15.73:34298) 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc439389640 1 -- 172.21.15.73:0/2896175878 learned_addr learned my addr 172.21.15.73:0/2896175878 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc439b8a640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416170 0x56176c4293b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 0x56176c41a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 -- 172.21.15.73:0/2896175878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416170 msgr2=0x56176c4293b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416170 0x56176c4293b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 -- 172.21.15.73:0/2896175878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 msgr2=0x56176c4298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c4298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 -- 172.21.15.73:0/2896175878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176c429fe0 con 0x56176c421350 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc43a38b640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 0x56176c41a500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005750 tx=0x7fc42c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=147c9ad02f3dc870 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.772+0000 7fc41d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc42c00fe00 con 0x56176c421350 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc424004c60 con 0x56176c421350 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7fc42c01e6b0 con 0x56176c421350 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7fc42c042050 con 0x56176c421350 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc43ebd8740 10 client.26821 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x56176c49baa0 con 0x56176c421350 2024-06-06T07:20:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.773+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc424004c60 con 0x56176c421350 2024-06-06T07:20:33.778 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:33 smithi073 ceph-mon[32303]: pgmap v610: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7fc42c026050 con 0x56176c421350 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 10 client.26821 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 10 client.26821 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 20 client.26821 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c49e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 4 client.26821 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:20:33.779 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56176c387c20 con 0x56176c4317a0 2024-06-06T07:20:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.774+0000 7fc439b8a640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c49e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.775+0000 7fc439b8a640 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c49e220 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c21efe47f93b13f server_cookie=5680f3fcfe225d76 in_seq=0 out_seq=0 2024-06-06T07:20:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.775+0000 7fc41d7fa640 10 client.26821 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:33 smithi136 ceph-mon[35901]: pgmap v610: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:33.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 56954 9.00 MB/sec execute 218 sec latency 92.531 ms 2024-06-06T07:20:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:33 smithi170 ceph-mon[36108]: pgmap v610: 129 pgs: 129 active+clean; 915 MiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.5 MiB/s wr, 279 op/s 2024-06-06T07:20:33.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.846+0000 7fc41d7fa640 1 -- 172.21.15.73:0/2896175878 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56176c387c20 con 0x56176c4317a0 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.846+0000 7fc41d7fa640 10 client.26821 handle_command_reply: tid=0 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 client.26821 shutdown 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 msgr2=0x56176c41a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 0x56176c41a500 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005750 tx=0x7fc42c016000 comp rx=0 tx=0).stop 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 shutdown_connections 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56176c4317a0 0x56176c49e220 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56176c421350 0x56176c41a500 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56176c416a90 0x56176c4298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 --2- 172.21.15.73:0/2896175878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56176c416170 0x56176c4293b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 >> 172.21.15.73:0/2896175878 conn(0x56176c412680 msgr2=0x56176c41e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc41d7fa640 0 client.26821 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 shutdown_connections 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 1 -- 172.21.15.73:0/2896175878 wait complete. 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.847+0000 7fc43ebd8740 20 client.26821 trim_cache size 0 max 16384 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:20:33.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:20:33.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 msgr2=0x7fc4200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 0x7fc4200a52e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc42c005fd0 tx=0x7fc42c009500 comp rx=0 tx=0).stop 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 msgr2=0x7fc43419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc430005ab0 tx=0x7fc430003c10 comp rx=0 tx=0).stop 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 shutdown_connections 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc4341a2fe0 0x7fc4341a3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc43410d2d0 0x7fc43419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc4200a2e20 0x7fc4200a52e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 --2- 172.21.15.73:0/3151619825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc434108200 0x7fc43419e770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 >> 172.21.15.73:0/3151619825 conn(0x7fc434075f30 msgr2=0x7fc434076af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 shutdown_connections 2024-06-06T07:20:33.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:33.849+0000 7fc43bdee640 1 -- 172.21.15.73:0/3151619825 wait complete. 2024-06-06T07:20:34.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57128 9.03 MB/sec execute 219 sec latency 110.815 ms 2024-06-06T07:20:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:35 smithi073 ceph-mon[32303]: pgmap v611: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 421 op/s 2024-06-06T07:20:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:35 smithi136 ceph-mon[35901]: pgmap v611: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 421 op/s 2024-06-06T07:20:35.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57300 9.03 MB/sec execute 220 sec latency 72.518 ms 2024-06-06T07:20:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:35 smithi170 ceph-mon[36108]: pgmap v611: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.3 MiB/s wr, 421 op/s 2024-06-06T07:20:36.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57479 9.05 MB/sec execute 221 sec latency 61.706 ms 2024-06-06T07:20:37.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:37 smithi170 ceph-mon[36108]: pgmap v612: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-06-06T07:20:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:37 smithi073 ceph-mon[32303]: pgmap v612: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-06-06T07:20:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:37 smithi136 ceph-mon[35901]: pgmap v612: 129 pgs: 129 active+clean; 932 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 278 op/s 2024-06-06T07:20:37.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57613 9.02 MB/sec execute 222 sec latency 73.477 ms 2024-06-06T07:20:38.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57759 8.99 MB/sec execute 223 sec latency 243.114 ms 2024-06-06T07:20:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:39 smithi073 ceph-mon[32303]: pgmap v613: 129 pgs: 129 active+clean; 929 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:20:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:39 smithi136 ceph-mon[35901]: pgmap v613: 129 pgs: 129 active+clean; 929 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:20:39.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 57947 9.00 MB/sec execute 224 sec latency 78.122 ms 2024-06-06T07:20:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:39 smithi170 ceph-mon[36108]: pgmap v613: 129 pgs: 129 active+clean; 929 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:20:40.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 58106 8.98 MB/sec execute 225 sec latency 102.618 ms 2024-06-06T07:20:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:41 smithi073 ceph-mon[32303]: pgmap v614: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 287 op/s 2024-06-06T07:20:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:41 smithi136 ceph-mon[35901]: pgmap v614: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 287 op/s 2024-06-06T07:20:41.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 58283 8.97 MB/sec execute 226 sec latency 88.521 ms 2024-06-06T07:20:41.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:41 smithi170 ceph-mon[36108]: pgmap v614: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.2 MiB/s wr, 287 op/s 2024-06-06T07:20:42.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 58460 8.99 MB/sec execute 227 sec latency 73.108 ms 2024-06-06T07:20:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:43 smithi073 ceph-mon[32303]: pgmap v615: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-06-06T07:20:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:43 smithi136 ceph-mon[35901]: pgmap v615: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-06-06T07:20:43.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 58660 9.03 MB/sec execute 228 sec latency 102.114 ms 2024-06-06T07:20:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:43 smithi170 ceph-mon[36108]: pgmap v615: 129 pgs: 129 active+clean; 931 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.1 MiB/s wr, 285 op/s 2024-06-06T07:20:44.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 58852 9.03 MB/sec execute 229 sec latency 72.191 ms 2024-06-06T07:20:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:45 smithi073 ceph-mon[32303]: pgmap v616: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 422 op/s 2024-06-06T07:20:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:45 smithi136 ceph-mon[35901]: pgmap v616: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 422 op/s 2024-06-06T07:20:45.810 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59023 9.02 MB/sec execute 230 sec latency 71.709 ms 2024-06-06T07:20:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:45 smithi170 ceph-mon[36108]: pgmap v616: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 422 op/s 2024-06-06T07:20:46.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59186 9.03 MB/sec execute 231 sec latency 73.195 ms 2024-06-06T07:20:46.982 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 -- 172.21.15.73:0/2150589426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 msgr2=0x7f260406a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 --2- 172.21.15.73:0/2150589426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260406a630 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009900 tx=0x7f25f802a410 comp rx=0 tx=0).stop 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 -- 172.21.15.73:0/2150589426 shutdown_connections 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 --2- 172.21.15.73:0/2150589426 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f260406ab70 0x7f260410e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 --2- 172.21.15.73:0/2150589426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260406a630 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 --2- 172.21.15.73:0/2150589426 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2604107990 0x7f2604107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 -- 172.21.15.73:0/2150589426 >> 172.21.15.73:0/2150589426 conn(0x7f26040765d0 msgr2=0x7f26040769e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 -- 172.21.15.73:0/2150589426 shutdown_connections 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.981+0000 7f260a5e7640 1 -- 172.21.15.73:0/2150589426 wait complete. 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.982+0000 7f260a5e7640 1 Processor -- start 2024-06-06T07:20:46.983 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.982+0000 7f260a5e7640 1 -- start start 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260410f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 0x7f260410fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 0x7f260410ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2604113e50 con 0x7f260406a1d0 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2604113cd0 con 0x7f2604107990 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f260a5e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2604113fd0 con 0x7f260406ab70 2024-06-06T07:20:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2603fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260410f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f26037fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 0x7f260410fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2603fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260410f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56412/0 (socket says 172.21.15.73:56412) 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 0x7f260410ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2603fff640 1 -- 172.21.15.73:0/1734912825 learned_addr learned my addr 172.21.15.73:0/1734912825 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 -- 172.21.15.73:0/1734912825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 msgr2=0x7f260410fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 0x7f260410fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 -- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 msgr2=0x7f260410f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260410f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 -- 172.21.15.73:0/1734912825 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26041aa970 con 0x7f2604107990 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f26037fe640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 0x7f260410fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.983+0000 7f2608b84640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 0x7f260410ff80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e970 tx=0x7f25fc00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.984+0000 7f26017fa640 1 -- 172.21.15.73:0/1734912825 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc004d20 con 0x7f2604107990 2024-06-06T07:20:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.984+0000 7f26017fa640 1 -- 172.21.15.73:0/1734912825 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25fc005020 con 0x7f2604107990 2024-06-06T07:20:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.984+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26041aac60 con 0x7f2604107990 2024-06-06T07:20:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.984+0000 7f26017fa640 1 -- 172.21.15.73:0/1734912825 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc0116d0 con 0x7f2604107990 2024-06-06T07:20:46.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.984+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26041ab180 con 0x7f2604107990 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.986+0000 7f260d3d4740 1 Processor -- start 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f26017fa640 1 -- 172.21.15.73:0/1734912825 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f25fc0051d0 con 0x7f2604107990 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 -- start start 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12da550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 0x5582c12e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5582c123e4f0 con 0x5582c12e5350 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5582c11dc870 con 0x5582c12da170 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f260d3d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582c123a4b0 con 0x5582c12daa90 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f26037fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12da550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56428/0 (socket says 172.21.15.73:56428) 2024-06-06T07:20:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f26037fe640 1 -- 172.21.15.73:0/516565179 learned_addr learned my addr 172.21.15.73:0/516565179 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.987+0000 7f2603fff640 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 0x5582c12e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 -- 172.21.15.73:0/516565179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 msgr2=0x5582c12da550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12da550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 -- 172.21.15.73:0/516565179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 msgr2=0x5582c12e7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 -- 172.21.15.73:0/516565179 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c11f89b0 con 0x5582c12daa90 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f26037fe640 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f26017fa640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 0x7f25ec0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f2603fff640 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 0x5582c12e4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009c70 tx=0x7f25f4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7fd4acacc804999 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f26037fe640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 0x7f25ec0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f26017fa640 1 -- 172.21.15.73:0/1734912825 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f25fc0cf8c0 con 0x7f2604107990 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f25ea7fc640 1 -- 172.21.15.73:0/516565179 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4005220 con 0x5582c12daa90 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f25ea7fc640 1 -- 172.21.15.73:0/516565179 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f4004c50 con 0x5582c12daa90 2024-06-06T07:20:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.988+0000 7f25ea7fc640 1 -- 172.21.15.73:0/516565179 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f4019760 con 0x5582c12daa90 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f26037fe640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 0x7f25ec0a52e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5582c12d7aa0 tx=0x7f25f8000bc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- 172.21.15.73:0/516565179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 msgr2=0x5582c12e4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 0x5582c12e4e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f25f4009c70 tx=0x7f25f4011810 comp rx=0 tx=0).stop 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- 172.21.15.73:0/516565179 shutdown_connections 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12e5350 0x5582c12e7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12daa90 0x5582c12e4e10 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 --2- 172.21.15.73:0/516565179 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12da550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- 172.21.15.73:0/516565179 >> 172.21.15.73:0/516565179 conn(0x5582c12d6680 msgr2=0x5582c12d6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- 172.21.15.73:0/516565179 shutdown_connections 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- 172.21.15.73:0/516565179 wait complete. 2024-06-06T07:20:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.989+0000 7f260d3d4740 1 -- start start 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 Processor -- start 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e2260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c12e2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5582c12e7f80 con 0x5582c12daa90 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5582c11c7ae0 con 0x5582c12e5350 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5582c12e8100 con 0x5582c12da170 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f260d3d4740 10 client.0 _ll_register_callbacks cb 0x5582c12009b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56434/0 (socket says 172.21.15.73:56434) 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f26037fe640 1 -- 172.21.15.73:0/2055092021 learned_addr learned my addr 172.21.15.73:0/2055092021 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f2603fff640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.990+0000 7f2608b84640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c12e2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 -- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 msgr2=0x5582c12e2260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e2260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 -- 172.21.15.73:0/2055092021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 msgr2=0x5582c12e2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c12e2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 -- 172.21.15.73:0/2055092021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c0f52210 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f26037fe640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f25f802a930 tx=0x7f25f8005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15c768515631490 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f25e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f804f070 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f4004c50 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f25f80316a0 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f25f805b050 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f260d3d4740 10 client.17424 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.991+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5582c101e620 con 0x5582c12daa90 2024-06-06T07:20:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f4004c50 con 0x5582c12daa90 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f25f803d7a0 con 0x5582c12daa90 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 10 client.17424 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 20 client.17424 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c12f5850 0x5582c12f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 4 client.17424 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582c1251f10 con 0x5582c12f5850 2024-06-06T07:20:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.992+0000 7f2603fff640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c12f5850 0x5582c12f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.993+0000 7f25e37fe640 10 client.17424 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.993+0000 7f2603fff640 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c12f5850 0x5582c12f7d10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcb9fa6838e9b114 server_cookie=43e3ff514b8f8ca6 in_seq=0 out_seq=0 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.993+0000 7f25e37fe640 1 -- 172.21.15.73:0/2055092021 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5582c1251f10 con 0x5582c12f5850 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.993+0000 7f25e37fe640 10 client.17424 handle_command_reply: tid=0 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.993+0000 7f260d3d4740 1 client.17424 shutdown 2024-06-06T07:20:46.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 msgr2=0x5582c12e27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f25f802a930 tx=0x7f25f8005a20 comp rx=0 tx=0).stop 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 shutdown_connections 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c12f5850 0x5582c12f7d10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c12e2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e27a0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 --2- 172.21.15.73:0/2055092021 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e2260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 >> 172.21.15.73:0/2055092021 conn(0x5582c12d6680 msgr2=0x5582c12dc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f25e37fe640 0 client.17424 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 shutdown_connections 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 1 -- 172.21.15.73:0/2055092021 wait complete. 2024-06-06T07:20:46.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:46.994+0000 7f260d3d4740 20 client.17424 trim_cache size 0 max 16384 2024-06-06T07:20:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.019+0000 7f260d3d4740 1 Processor -- start 2024-06-06T07:20:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.019+0000 7f260d3d4740 1 -- start start 2024-06-06T07:20:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12d5f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12e5350 0x5582c12fdb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b5460 con 0x5582c12daa90 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b6fa0 con 0x5582c12da170 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f260d3d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b5b30 con 0x5582c12e5350 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56444/0 (socket says 172.21.15.73:56444) 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 -- 172.21.15.73:0/192570194 learned_addr learned my addr 172.21.15.73:0/192570194 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f2608b84640 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12e5350 0x5582c12fdb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f2603fff640 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12d5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 -- 172.21.15.73:0/192570194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12e5350 msgr2=0x5582c12fdb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12e5350 0x5582c12fdb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 -- 172.21.15.73:0/192570194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 msgr2=0x5582c12d5f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12d5f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 -- 172.21.15.73:0/192570194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582c12dcfe0 con 0x5582c12daa90 2024-06-06T07:20:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.020+0000 7f26037fe640 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009230 tx=0x7f25f8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b738fe6d20ae1ba3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.021+0000 7f25e9ffb640 1 -- 172.21.15.73:0/192570194 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8064070 con 0x5582c12daa90 2024-06-06T07:20:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.021+0000 7f25e9ffb640 1 -- 172.21.15.73:0/192570194 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f803d650 con 0x5582c12daa90 2024-06-06T07:20:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.021+0000 7f25e9ffb640 1 -- 172.21.15.73:0/192570194 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f804f040 con 0x5582c12daa90 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- 172.21.15.73:0/192570194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 msgr2=0x5582c12fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009230 tx=0x7f25f8005fd0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- 172.21.15.73:0/192570194 shutdown_connections 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12e5350 0x5582c12fdb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12fd620 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 --2- 172.21.15.73:0/192570194 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12da170 0x5582c12d5f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- 172.21.15.73:0/192570194 >> 172.21.15.73:0/192570194 conn(0x5582c12d6680 msgr2=0x5582c12dd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- 172.21.15.73:0/192570194 shutdown_connections 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- 172.21.15.73:0/192570194 wait complete. 2024-06-06T07:20:47.026 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 -- start start 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.022+0000 7f260d3d4740 1 Processor -- start 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c1358620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b68d0 con 0x5582c12daa90 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b6200 con 0x5582c12e5350 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f26041b3460 con 0x5582c12da170 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f260d3d4740 10 client.0 _ll_register_callbacks cb 0x7f25f8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c1358620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f26037fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56448/0 (socket says 172.21.15.73:56448) 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f26037fe640 1 -- 172.21.15.73:0/3612243930 learned_addr learned my addr 172.21.15.73:0/3612243930 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2603fff640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.027 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 -- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 msgr2=0x5582c12e8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 -- 172.21.15.73:0/3612243930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 msgr2=0x5582c12e9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 -- 172.21.15.73:0/3612243930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582c1358be0 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f2608b84640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c1358620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0ccc00 tx=0x7f25fc09bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7eee525ca542d61 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.023+0000 7f25e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25fc01f040 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f25fc00cb20 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25f803d650 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f25fc005600 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 10 client.26657 fetch_fsmap learned FSMap version 74 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5582c135ff60 con 0x5582c12e5350 2024-06-06T07:20:47.028 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25f803d650 con 0x5582c12e5350 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f25fc00f040 con 0x5582c12e5350 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 10 client.26657 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 10 client.26657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 20 client.26657 populate_metadata read hostname 'smithi073' 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c1360680 0x5582c1362aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 4 client.26657 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.024+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582c124bc20 con 0x5582c1360680 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.025+0000 7f2603fff640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c1360680 0x5582c1362aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.025+0000 7f25e37fe640 10 client.26657 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:47.029 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.025+0000 7f2603fff640 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c1360680 0x5582c1362aa0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7dd0679c06fc36 server_cookie=45a7b668f4d653f in_seq=0 out_seq=0 2024-06-06T07:20:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f25e37fe640 1 -- 172.21.15.73:0/3612243930 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5582c124bc20 con 0x5582c1360680 2024-06-06T07:20:47.091 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f25e37fe640 10 client.26657 handle_command_reply: tid=0 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 client.26657 shutdown 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 msgr2=0x5582c1358620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c1358620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f25fc0ccc00 tx=0x7f25fc09bbc0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 shutdown_connections 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5582c1360680 0x5582c1362aa0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5582c12e5350 0x5582c1358620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f25e37fe640 0 client.26657 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5582c12daa90 0x5582c12e9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 --2- 172.21.15.73:0/3612243930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5582c12da170 0x5582c12e8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 >> 172.21.15.73:0/3612243930 conn(0x5582c12d6680 msgr2=0x5582c12dbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 shutdown_connections 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.090+0000 7f260d3d4740 1 -- 172.21.15.73:0/3612243930 wait complete. 2024-06-06T07:20:47.092 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260d3d4740 20 client.26657 trim_cache size 0 max 16384 2024-06-06T07:20:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:20:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:20:47.093 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 msgr2=0x7f25ec0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 0x7f25ec0a52e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x5582c12d7aa0 tx=0x7f25f8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 msgr2=0x7f260410ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 0x7f260410ff80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f25fc00e970 tx=0x7f25fc00ee30 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 shutdown_connections 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2604107990 0x7f260410ff80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.091+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f25ec0a2e20 0x7f25ec0a52e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.092+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f260406ab70 0x7f260410fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.092+0000 7f260a5e7640 1 --2- 172.21.15.73:0/1734912825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f260406a1d0 0x7f260410f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.092+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 >> 172.21.15.73:0/1734912825 conn(0x7f26040765d0 msgr2=0x7f26040ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:20:47.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.092+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 shutdown_connections 2024-06-06T07:20:47.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:20:47.092+0000 7f260a5e7640 1 -- 172.21.15.73:0/1734912825 wait complete. 2024-06-06T07:20:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:47 smithi073 ceph-mon[32303]: pgmap v617: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-06-06T07:20:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:47 smithi136 ceph-mon[35901]: pgmap v617: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-06-06T07:20:47.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59352 9.03 MB/sec execute 232 sec latency 80.184 ms 2024-06-06T07:20:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:47 smithi170 ceph-mon[36108]: pgmap v617: 129 pgs: 129 active+clean; 939 MiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.6 MiB/s wr, 280 op/s 2024-06-06T07:20:48.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59504 9.03 MB/sec execute 233 sec latency 100.141 ms 2024-06-06T07:20:49.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:48 smithi073 ceph-mon[32303]: pgmap v618: 129 pgs: 129 active+clean; 950 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 410 op/s 2024-06-06T07:20:49.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:48 smithi136 ceph-mon[35901]: pgmap v618: 129 pgs: 129 active+clean; 950 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 410 op/s 2024-06-06T07:20:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:48 smithi170 ceph-mon[36108]: pgmap v618: 129 pgs: 129 active+clean; 950 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 6.8 MiB/s wr, 410 op/s 2024-06-06T07:20:49.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59662 9.03 MB/sec execute 234 sec latency 121.511 ms 2024-06-06T07:20:50.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 59856 9.07 MB/sec execute 235 sec latency 104.494 ms 2024-06-06T07:20:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:51 smithi170 ceph-mon[36108]: pgmap v619: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-06-06T07:20:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:51 smithi073 ceph-mon[32303]: pgmap v619: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-06-06T07:20:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:51 smithi136 ceph-mon[35901]: pgmap v619: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 4.8 MiB/s wr, 282 op/s 2024-06-06T07:20:51.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60032 9.06 MB/sec execute 236 sec latency 108.738 ms 2024-06-06T07:20:52.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60136 9.04 MB/sec execute 237 sec latency 135.725 ms 2024-06-06T07:20:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:53 smithi170 ceph-mon[36108]: pgmap v620: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:53 smithi073 ceph-mon[32303]: pgmap v620: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:53 smithi136 ceph-mon[35901]: pgmap v620: 129 pgs: 129 active+clean; 951 MiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.6 MiB/s wr, 271 op/s 2024-06-06T07:20:53.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60281 9.04 MB/sec execute 238 sec latency 343.081 ms 2024-06-06T07:20:54.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60454 9.03 MB/sec execute 239 sec latency 62.028 ms 2024-06-06T07:20:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:55 smithi170 ceph-mon[36108]: pgmap v621: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 405 op/s 2024-06-06T07:20:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:55 smithi073 ceph-mon[32303]: pgmap v621: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 405 op/s 2024-06-06T07:20:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:55 smithi136 ceph-mon[35901]: pgmap v621: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 405 op/s 2024-06-06T07:20:55.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60602 9.02 MB/sec execute 240 sec latency 107.251 ms 2024-06-06T07:20:56.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60742 9.01 MB/sec execute 241 sec latency 139.498 ms 2024-06-06T07:20:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:57 smithi170 ceph-mon[36108]: pgmap v622: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-06-06T07:20:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:57 smithi073 ceph-mon[32303]: pgmap v622: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-06-06T07:20:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:57 smithi136 ceph-mon[35901]: pgmap v622: 129 pgs: 129 active+clean; 969 MiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.0 MiB/s wr, 268 op/s 2024-06-06T07:20:57.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 60895 9.00 MB/sec execute 242 sec latency 109.757 ms 2024-06-06T07:20:58.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61048 8.99 MB/sec execute 243 sec latency 78.458 ms 2024-06-06T07:20:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:59 smithi170 ceph-mon[36108]: pgmap v623: 129 pgs: 129 active+clean; 973 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2024-06-06T07:20:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:20:59 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:20:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:59 smithi073 ceph-mon[32303]: pgmap v623: 129 pgs: 129 active+clean; 973 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2024-06-06T07:20:59.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:20:59 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:20:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:59 smithi136 ceph-mon[35901]: pgmap v623: 129 pgs: 129 active+clean; 973 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.2 MiB/s wr, 400 op/s 2024-06-06T07:20:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:20:59 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:20:59.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61191 8.97 MB/sec execute 244 sec latency 116.606 ms 2024-06-06T07:21:00.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- 172.21.15.73:0/849968114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 msgr2=0x7f421410d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 --2- 172.21.15.73:0/849968114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f421410d750 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f420c009cd0 tx=0x7f420c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- 172.21.15.73:0/849968114 shutdown_connections 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 --2- 172.21.15.73:0/849968114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f421410d750 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 --2- 172.21.15.73:0/849968114 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42141008d0 0x7f4214100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 --2- 172.21.15.73:0/849968114 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4214106890 0x7f4214106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- 172.21.15.73:0/849968114 >> 172.21.15.73:0/849968114 conn(0x7f4214077a20 msgr2=0x7f4214077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- 172.21.15.73:0/849968114 shutdown_connections 2024-06-06T07:21:00.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- 172.21.15.73:0/849968114 wait complete. 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 Processor -- start 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.225+0000 7f421a84c640 1 -- start start 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 0x7f4214198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4214106890 0x7f421419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.228 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4214110b60 con 0x7f4214101290 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42141109e0 con 0x7f4214106890 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f421a84c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4214110ce0 con 0x7f42141008d0 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43078/0 (socket says 172.21.15.73:43078) 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 -- 172.21.15.73:0/693242499 learned_addr learned my addr 172.21.15.73:0/693242499 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f4213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 0x7f4214198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f4218de9640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4214106890 0x7f421419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 -- 172.21.15.73:0/693242499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 msgr2=0x7f4214198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 0x7f4214198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 -- 172.21.15.73:0/693242499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4214106890 msgr2=0x7f421419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4214106890 0x7f421419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.229 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f42137fe640 1 -- 172.21.15.73:0/693242499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421419d130 con 0x7f4214101290 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.226+0000 7f4213fff640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 0x7f4214198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f42137fe640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f42117fa640 1 -- 172.21.15.73:0/693242499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420800cd80 con 0x7f4214101290 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f42117fa640 1 -- 172.21.15.73:0/693242499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42080108e0 con 0x7f4214101290 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f42117fa640 1 -- 172.21.15.73:0/693242499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208018920 con 0x7f4214101290 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421419d420 con 0x7f4214101290 2024-06-06T07:21:00.230 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.227+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42140fd790 con 0x7f4214101290 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.229+0000 7f42117fa640 1 -- 172.21.15.73:0/693242499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f4208010a90 con 0x7f4214101290 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 Processor -- start 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 -- start start 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968170 0x5562f9968550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.231 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9975740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562f98cc4f0 con 0x5562f9968a90 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562f986a870 con 0x5562f9968170 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.230+0000 7f421d631740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5562f98c84b0 con 0x5562f9973350 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43094/0 (socket says 172.21.15.73:43094) 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 -- 172.21.15.73:0/3758496613 learned_addr learned my addr 172.21.15.73:0/3758496613 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f42137fe640 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9975740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.232 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4218de9640 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968170 0x5562f9968550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f42117fa640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 0x7f41fc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 -- 172.21.15.73:0/3758496613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 msgr2=0x5562f9975740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9975740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 -- 172.21.15.73:0/3758496613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968170 msgr2=0x5562f9968550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968170 0x5562f9968550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 -- 172.21.15.73:0/3758496613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f98869b0 con 0x5562f9968a90 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4218de9640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 0x7f41fc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f42117fa640 1 -- 172.21.15.73:0/693242499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f42080cfb40 con 0x7f4214101290 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f4213fff640 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f42040096f0 tx=0x7f4204005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388c6718b1585588 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f41fa7fc640 1 -- 172.21.15.73:0/3758496613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420401e020 con 0x5562f9968a90 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.231+0000 7f41fa7fc640 1 -- 172.21.15.73:0/3758496613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f420400bd50 con 0x5562f9968a90 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f41fa7fc640 1 -- 172.21.15.73:0/3758496613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420401ad80 con 0x5562f9968a90 2024-06-06T07:21:00.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 -- 172.21.15.73:0/3758496613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 msgr2=0x5562f9972e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f42040096f0 tx=0x7f4204005f90 comp rx=0 tx=0).stop 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f4218de9640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 0x7f41fc0a52e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f420c004af0 tx=0x7f420c02a8e0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 -- 172.21.15.73:0/3758496613 shutdown_connections 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9975740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968a90 0x5562f9972e10 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 --2- 172.21.15.73:0/3758496613 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968170 0x5562f9968550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.232+0000 7f421d631740 1 -- 172.21.15.73:0/3758496613 >> 172.21.15.73:0/3758496613 conn(0x5562f9964680 msgr2=0x5562f9964a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.233+0000 7f421d631740 1 -- 172.21.15.73:0/3758496613 shutdown_connections 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.233+0000 7f421d631740 1 -- 172.21.15.73:0/3758496613 wait complete. 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.233+0000 7f421d631740 1 -- start start 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.233+0000 7f421d631740 1 Processor -- start 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f99707f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9973350 0x5562f9970d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5562f9975f80 con 0x5562f9973350 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5562f9855ae0 con 0x5562f9968a90 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5562f9976100 con 0x5562f9968170 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f421d631740 10 client.0 _ll_register_callbacks cb 0x5562f988e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f4213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f4213fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:38376/0 (socket says 172.21.15.73:38376) 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f4213fff640 1 -- 172.21.15.73:0/4178077307 learned_addr learned my addr 172.21.15.73:0/4178077307 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:00.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f4218de9640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9973350 0x5562f9970d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f99707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 -- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 msgr2=0x5562f99702b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 -- 172.21.15.73:0/4178077307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9973350 msgr2=0x5562f9970d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9973350 0x5562f9970d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 -- 172.21.15.73:0/4178077307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f95e0210 con 0x5562f9968a90 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f4213fff640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f42137fe640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f99707f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42080cce20 tx=0x7f4208098030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8479e7892cba5c08 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.234+0000 7f41f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42080cd2a0 con 0x5562f9968a90 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f4208010430 con 0x5562f9968a90 2024-06-06T07:21:00.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f420400bd50 con 0x5562f9968a90 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f4208010de0 con 0x5562f9968a90 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 10 client.26669 fetch_fsmap learned FSMap version 74 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5562f96ac620 con 0x5562f9968a90 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f420400bd50 con 0x5562f9968a90 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f42080979f0 con 0x5562f9968a90 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 10 client.26669 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 10 client.26669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 20 client.26669 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.235+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f9983440 0x5562f9985890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.236+0000 7f421d631740 4 client.26669 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:21:00.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.236+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562f98dff10 con 0x5562f9983440 2024-06-06T07:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.236+0000 7f4213fff640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f9983440 0x5562f9985890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.236+0000 7f4213fff640 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f9983440 0x5562f9985890 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c7a6e17c1bb2e56 server_cookie=d586ea12c2cace02 in_seq=0 out_seq=0 2024-06-06T07:21:00.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.236+0000 7f41f37fe640 10 client.26669 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f41f37fe640 1 -- 172.21.15.73:0/4178077307 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5562f98dff10 con 0x5562f9983440 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f41f37fe640 10 client.26669 handle_command_reply: tid=0 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 client.26669 shutdown 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 msgr2=0x5562f99707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f99707f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f42080cce20 tx=0x7f4208098030 comp rx=0 tx=0).stop 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 shutdown_connections 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f9983440 0x5562f9985890 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9973350 0x5562f9970d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f41f37fe640 0 client.26669 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f99707f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 --2- 172.21.15.73:0/4178077307 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9968170 0x5562f99702b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.237+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 >> 172.21.15.73:0/4178077307 conn(0x5562f9964680 msgr2=0x5562f996a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.238+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 shutdown_connections 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.238+0000 7f421d631740 1 -- 172.21.15.73:0/4178077307 wait complete. 2024-06-06T07:21:00.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.238+0000 7f421d631740 20 client.26669 trim_cache size 0 max 16384 2024-06-06T07:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.265+0000 7f421d631740 1 Processor -- start 2024-06-06T07:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.265+0000 7f421d631740 1 -- start start 2024-06-06T07:21:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f998b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f998b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42141acd50 con 0x5562f9968170 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42141ae890 con 0x5562f9968a90 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f421d631740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42141ad420 con 0x5562f9973350 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43116/0 (socket says 172.21.15.73:43116) 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 -- 172.21.15.73:0/3579101856 learned_addr learned my addr 172.21.15.73:0/3579101856 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f42137fe640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f998b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4218de9640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f998b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 -- 172.21.15.73:0/3579101856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 msgr2=0x5562f998b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f998b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 -- 172.21.15.73:0/3579101856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 msgr2=0x5562f998b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f998b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 -- 172.21.15.73:0/3579101856 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562f9963f60 con 0x5562f9968170 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f42137fe640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f998b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.266+0000 7f4213fff640 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f42040108e0 tx=0x7f4204009690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0bf9b71a3f0c5cc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.267+0000 7f41f9ffb640 1 -- 172.21.15.73:0/3579101856 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420401e040 con 0x5562f9968170 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.267+0000 7f41f9ffb640 1 -- 172.21.15.73:0/3579101856 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4204009d90 con 0x5562f9968170 2024-06-06T07:21:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.267+0000 7f41f9ffb640 1 -- 172.21.15.73:0/3579101856 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420402e420 con 0x5562f9968170 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- 172.21.15.73:0/3579101856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 msgr2=0x5562f99721f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f42040108e0 tx=0x7f4204009690 comp rx=0 tx=0).stop 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- 172.21.15.73:0/3579101856 shutdown_connections 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f998b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f998b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 --2- 172.21.15.73:0/3579101856 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f99721f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- 172.21.15.73:0/3579101856 >> 172.21.15.73:0/3579101856 conn(0x5562f9964680 msgr2=0x5562f9971ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- 172.21.15.73:0/3579101856 shutdown_connections 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- 172.21.15.73:0/3579101856 wait complete. 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.268+0000 7f421d631740 1 -- start start 2024-06-06T07:21:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 Processor -- start 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f9976cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f4213fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43128/0 (socket says 172.21.15.73:43128) 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f4213fff640 1 -- 172.21.15.73:0/1728521295 learned_addr learned my addr 172.21.15.73:0/1728521295 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9977210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42141ae1c0 con 0x5562f9968170 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42141adaf0 con 0x5562f9968a90 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42141aad50 con 0x5562f9973350 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f421d631740 10 client.0 _ll_register_callbacks cb 0x7f42080d32e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.269+0000 7f4218de9640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9977210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.274 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f42137fe640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f9976cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 -- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 msgr2=0x5562f9977210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9977210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 -- 172.21.15.73:0/1728521295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 msgr2=0x5562f9976cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f9976cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562f99778b0 con 0x5562f9968170 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4218de9640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9977210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f4213fff640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4204005760 tx=0x7f4204009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6587e3e0e2e570c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f41f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204005110 con 0x5562f9968170 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(133..133 src has 1..133) ==== 9554+0+0 (secure 0 0 0) 0x7f42040052b0 con 0x5562f9968170 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4204009d90 con 0x5562f9968170 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) ==== 24+0+0 (secure 0 0 0) 0x7f42040054b0 con 0x5562f9968170 2024-06-06T07:21:00.275 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f421d631740 10 client.17448 fetch_fsmap learned FSMap version 74 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.270+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) -- 0x5562f99ee030 con 0x5562f9968170 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4204009d90 con 0x5562f9968170 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 74) ==== 8362+0+0 (secure 0 0 0) 0x7f420402e420 con 0x5562f9968170 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 10 client.17448 fetch_fsmap finished waiting for FSMap version 74 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 20 client.17448 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f99ee4e0 0x5562f99f0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 4 client.17448 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5562f98d9c20 con 0x5562f99ee4e0 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.271+0000 7f42137fe640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f99ee4e0 0x5562f99f0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.272+0000 7f41f37fe640 10 client.17448 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:00.276 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.272+0000 7f42137fe640 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f99ee4e0 0x5562f99f0900 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=349c3f93a9b39466 server_cookie=9d2425206b74c9a0 in_seq=0 out_seq=0 2024-06-06T07:21:00.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.336+0000 7f41f37fe640 1 -- 172.21.15.73:0/1728521295 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5562f98d9c20 con 0x5562f99ee4e0 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.336+0000 7f41f37fe640 10 client.17448 handle_command_reply: tid=0 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.336+0000 7f421d631740 1 client.17448 shutdown 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.336+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 msgr2=0x5562f997b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.336+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f4204005760 tx=0x7f4204009f40 comp rx=0 tx=0).stop 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 shutdown_connections 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5562f99ee4e0 0x5562f99f0900 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5562f9973350 0x5562f9977210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5562f9968a90 0x5562f9976cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 --2- 172.21.15.73:0/1728521295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5562f9968170 0x5562f997b960 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 >> 172.21.15.73:0/1728521295 conn(0x5562f9964680 msgr2=0x5562f99637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f41f37fe640 0 client.17448 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 shutdown_connections 2024-06-06T07:21:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 1 -- 172.21.15.73:0/1728521295 wait complete. 2024-06-06T07:21:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.337+0000 7f421d631740 20 client.17448 trim_cache size 0 max 16384 2024-06-06T07:21:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:21:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:21:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 msgr2=0x7f41fc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 0x7f41fc0a52e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f420c004af0 tx=0x7f420c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 msgr2=0x7f4214198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f420800e9f0 tx=0x7f420800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 shutdown_connections 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4214106890 0x7f421419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4214101290 0x7f4214198680 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f41fc0a2e20 0x7f41fc0a52e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 --2- 172.21.15.73:0/693242499 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42141008d0 0x7f4214198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 >> 172.21.15.73:0/693242499 conn(0x7f4214077a20 msgr2=0x7f4214069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 shutdown_connections 2024-06-06T07:21:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:00.338+0000 7f421a84c640 1 -- 172.21.15.73:0/693242499 wait complete. 2024-06-06T07:21:00.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61356 8.97 MB/sec execute 245 sec latency 89.025 ms 2024-06-06T07:21:01.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:21:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: ignoring --setuser ceph since I am not root 2024-06-06T07:21:01.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:21:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: ignoring --setgroup ceph since I am not root 2024-06-06T07:21:01.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:21:00 smithi136 ceph-mds[73797]: -- 172.21.15.136:0/3771333662 <== mon.2 v2:172.21.15.136:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559eb7093a40 con 0x559eb7066c00 2024-06-06T07:21:01.052 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:21:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: starting mds.e at 2024-06-06T07:21:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:01 smithi170 ceph-mon[36108]: pgmap v624: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2024-06-06T07:21:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:01 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:21:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:01 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:21:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:01 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:21:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:01 smithi170 ceph-mon[36108]: daemon mds.e restarted 2024-06-06T07:21:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:01 smithi073 ceph-mon[32303]: pgmap v624: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2024-06-06T07:21:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:01 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:21:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:01 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:21:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:01 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:21:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:01 smithi073 ceph-mon[32303]: daemon mds.e restarted 2024-06-06T07:21:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:01 smithi136 ceph-mon[35901]: pgmap v624: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.1 MiB/s wr, 272 op/s 2024-06-06T07:21:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:01 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:21:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:01 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:21:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:01 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:21:01.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:01 smithi136 ceph-mon[35901]: daemon mds.e restarted 2024-06-06T07:21:01.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61542 8.97 MB/sec execute 246 sec latency 128.710 ms 2024-06-06T07:21:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:02 smithi170 ceph-mon[36108]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T07:21:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:02 smithi170 ceph-mon[36108]: mds.? [v2:172.21.15.136:6834/433813241,v1:172.21.15.136:6835/433813241] up:boot 2024-06-06T07:21:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:02 smithi170 ceph-mon[36108]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:21:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:02 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:21:02.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:02 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:02 smithi073 ceph-mon[32303]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T07:21:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:02 smithi073 ceph-mon[32303]: mds.? [v2:172.21.15.136:6834/433813241,v1:172.21.15.136:6835/433813241] up:boot 2024-06-06T07:21:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:02 smithi073 ceph-mon[32303]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:21:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:02 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:21:02.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:02 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:02 smithi136 ceph-mon[35901]: osdmap e134: 12 total, 12 up, 12 in 2024-06-06T07:21:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:02 smithi136 ceph-mon[35901]: mds.? [v2:172.21.15.136:6834/433813241,v1:172.21.15.136:6835/433813241] up:boot 2024-06-06T07:21:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:02 smithi136 ceph-mon[35901]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-06-06T07:21:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:02 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-06T07:21:02.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:02 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:02.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61749 9.00 MB/sec execute 247 sec latency 89.330 ms 2024-06-06T07:21:03.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 61921 8.99 MB/sec execute 248 sec latency 76.220 ms 2024-06-06T07:21:04.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:03 smithi073 ceph-mon[32303]: pgmap v626: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-06-06T07:21:04.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:03 smithi136 ceph-mon[35901]: pgmap v626: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-06-06T07:21:04.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:03 smithi170 ceph-mon[36108]: pgmap v626: 129 pgs: 129 active+clean; 972 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 322 op/s 2024-06-06T07:21:04.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62054 8.98 MB/sec execute 249 sec latency 121.382 ms 2024-06-06T07:21:05.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62214 8.97 MB/sec execute 250 sec latency 120.656 ms 2024-06-06T07:21:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:05 smithi073 ceph-mon[32303]: pgmap v627: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-06-06T07:21:06.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:05 smithi073 ceph-mon[32303]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T07:21:06.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:05 smithi136 ceph-mon[35901]: pgmap v627: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-06-06T07:21:06.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:05 smithi136 ceph-mon[35901]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T07:21:06.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:05 smithi170 ceph-mon[36108]: pgmap v627: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.3 MiB/s wr, 373 op/s 2024-06-06T07:21:06.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:05 smithi170 ceph-mon[36108]: osdmap e135: 12 total, 12 up, 12 in 2024-06-06T07:21:06.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62425 9.00 MB/sec execute 251 sec latency 60.788 ms 2024-06-06T07:21:07.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62607 9.00 MB/sec execute 252 sec latency 80.380 ms 2024-06-06T07:21:07.898 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:07 smithi170 ceph-mon[36108]: pgmap v629: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-06-06T07:21:08.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:07 smithi073 ceph-mon[32303]: pgmap v629: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-06-06T07:21:08.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:07 smithi136 ceph-mon[35901]: pgmap v629: 129 pgs: 129 active+clean; 974 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.4 MiB/s wr, 269 op/s 2024-06-06T07:21:08.811 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62777 9.00 MB/sec execute 253 sec latency 59.531 ms 2024-06-06T07:21:09.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 62935 9.02 MB/sec execute 254 sec latency 59.476 ms 2024-06-06T07:21:10.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:09 smithi073 ceph-mon[32303]: pgmap v630: 129 pgs: 129 active+clean; 985 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2024-06-06T07:21:10.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:09 smithi136 ceph-mon[35901]: pgmap v630: 129 pgs: 129 active+clean; 985 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2024-06-06T07:21:10.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:09 smithi170 ceph-mon[36108]: pgmap v630: 129 pgs: 129 active+clean; 985 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.5 MiB/s wr, 424 op/s 2024-06-06T07:21:10.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63137 9.04 MB/sec execute 255 sec latency 211.203 ms 2024-06-06T07:21:11.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63317 9.04 MB/sec execute 256 sec latency 78.482 ms 2024-06-06T07:21:12.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:11 smithi073 ceph-mon[32303]: pgmap v631: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 404 op/s 2024-06-06T07:21:12.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:11 smithi136 ceph-mon[35901]: pgmap v631: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 404 op/s 2024-06-06T07:21:12.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:11 smithi170 ceph-mon[36108]: pgmap v631: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.3 MiB/s wr, 404 op/s 2024-06-06T07:21:12.812 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63469 9.04 MB/sec execute 257 sec latency 130.397 ms 2024-06-06T07:21:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 -- 172.21.15.73:0/402969996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d410e360 msgr2=0x7f54d4110750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 --2- 172.21.15.73:0/402969996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d410e360 0x7f54d4110750 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009ca0 tx=0x7f54cc02a8c0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 -- 172.21.15.73:0/402969996 shutdown_connections 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 --2- 172.21.15.73:0/402969996 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d410e360 0x7f54d4110750 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 --2- 172.21.15.73:0/402969996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 --2- 172.21.15.73:0/402969996 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d41007b0 0x7f54d4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 -- 172.21.15.73:0/402969996 >> 172.21.15.73:0/402969996 conn(0x7f54d4077a20 msgr2=0x7f54d4077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 -- 172.21.15.73:0/402969996 shutdown_connections 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.465+0000 7f54d9438640 1 -- 172.21.15.73:0/402969996 wait complete. 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 Processor -- start 2024-06-06T07:21:13.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 -- start start 2024-06-06T07:21:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 0x7f54d410dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d410e360 0x7f54d410b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4113e40 con 0x7f54d41010d0 2024-06-06T07:21:13.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4113cc0 con 0x7f54d41007b0 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d9438640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4113fc0 con 0x7f54d410e360 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 0x7f54d410dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 0x7f54d410dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57850/0 (socket says 172.21.15.73:57850) 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d2ffd640 1 -- 172.21.15.73:0/180467527 learned_addr learned my addr 172.21.15.73:0/180467527 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.466+0000 7f54d37fe640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d410e360 0x7f54d410b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 -- 172.21.15.73:0/180467527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d410e360 msgr2=0x7f54d410b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d410e360 0x7f54d410b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 -- 172.21.15.73:0/180467527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 msgr2=0x7f54d410dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 0x7f54d410dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 -- 172.21.15.73:0/180467527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54d41aab40 con 0x7f54d41010d0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54d27fc640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410aee0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f54c800eab0 tx=0x7f54c800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.467+0000 7f54bbfff640 1 -- 172.21.15.73:0/180467527 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c800cc30 con 0x7f54d41010d0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.468+0000 7f54bbfff640 1 -- 172.21.15.73:0/180467527 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54c800cdd0 con 0x7f54d41010d0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.468+0000 7f54bbfff640 1 -- 172.21.15.73:0/180467527 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8018840 con 0x7f54d41010d0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.468+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54d41aad70 con 0x7f54d41010d0 2024-06-06T07:21:13.471 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.468+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54d41ab180 con 0x7f54d41010d0 2024-06-06T07:21:13.472 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.471+0000 7f54bbfff640 1 -- 172.21.15.73:0/180467527 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f54c80189e0 con 0x7f54d41010d0 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54bbfff640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 0x7f54bc0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 Processor -- start 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 -- start start 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 0x7f54bc0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54bbfff640 1 -- 172.21.15.73:0/180467527 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f54c80cfb80 con 0x7f54d41010d0 2024-06-06T07:21:13.474 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b44550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d139aa84f0 con 0x55d139b44170 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d139a46870 con 0x55d139b44a90 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.472+0000 7f54dc21e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d139aa44b0 con 0x55d139b4f350 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b44550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d27fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57858/0 (socket says 172.21.15.73:57858) 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:50322/0 (socket says 172.21.15.73:50322) 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d27fc640 1 -- 172.21.15.73:0/3842172300 learned_addr learned my addr 172.21.15.73:0/3842172300 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:13.475 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 0x7f54bc0a52e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009ce0 tx=0x7f54c4011810 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 -- 172.21.15.73:0/3842172300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 msgr2=0x55d139b4ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 -- 172.21.15.73:0/3842172300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 msgr2=0x55d139b44550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b44550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 -- 172.21.15.73:0/3842172300 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d139a629b0 con 0x55d139b4f350 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54d37fe640 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54cc006010 tx=0x7f54cc009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93608679feb9affb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.473+0000 7f54b97fa640 1 -- 172.21.15.73:0/3842172300 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc031a40 con 0x55d139b4f350 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.474+0000 7f54b97fa640 1 -- 172.21.15.73:0/3842172300 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54cc036030 con 0x55d139b4f350 2024-06-06T07:21:13.476 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.474+0000 7f54b97fa640 1 -- 172.21.15.73:0/3842172300 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc003970 con 0x55d139b4f350 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.474+0000 7f54dc21e740 1 -- 172.21.15.73:0/3842172300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 msgr2=0x55d139b51740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.474+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54cc006010 tx=0x7f54cc009a20 comp rx=0 tx=0).stop 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 -- 172.21.15.73:0/3842172300 shutdown_connections 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b51740 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.477 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3842172300 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b44550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.478 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 -- 172.21.15.73:0/3842172300 >> 172.21.15.73:0/3842172300 conn(0x55d139b40680 msgr2=0x55d139b40a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 -- 172.21.15.73:0/3842172300 shutdown_connections 2024-06-06T07:21:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.475+0000 7f54dc21e740 1 -- 172.21.15.73:0/3842172300 wait complete. 2024-06-06T07:21:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.476+0000 7f54dc21e740 1 -- start start 2024-06-06T07:21:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.476+0000 7f54dc21e740 1 Processor -- start 2024-06-06T07:21:13.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b4cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d139b51da0 con 0x55d139b44170 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d139a31ae0 con 0x55d139b44a90 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d139b51f20 con 0x55d139b4f350 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54dc21e740 10 client.0 _ll_register_callbacks cb 0x55d139a6a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54d2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54d2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47210/0 (socket says 172.21.15.73:47210) 2024-06-06T07:21:13.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54d2ffd640 1 -- 172.21.15.73:0/2869579390 learned_addr learned my addr 172.21.15.73:0/2869579390 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.477+0000 7f54d27fc640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 -- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 msgr2=0x55d139b4cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b4cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 -- 172.21.15.73:0/2869579390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 msgr2=0x55d139b4c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 -- 172.21.15.73:0/2869579390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1397bc210 con 0x55d139b44170 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d27fc640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f54c401ec80 tx=0x7f54c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2cd878c1bc93b30 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.478+0000 7f54b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4017040 con 0x55d139b44170 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc036030 con 0x55d139b44170 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f54c401e310 con 0x55d139b44170 2024-06-06T07:21:13.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f54c4030050 con 0x55d139b44170 2024-06-06T07:21:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54dc21e740 10 client.17466 fetch_fsmap learned FSMap version 76 2024-06-06T07:21:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d139888620 con 0x55d139b44170 2024-06-06T07:21:13.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.479+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54cc036030 con 0x55d139b44170 2024-06-06T07:21:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f54c4025020 con 0x55d139b44170 2024-06-06T07:21:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 10 client.17466 fetch_fsmap finished waiting for FSMap version 76 2024-06-06T07:21:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 10 client.17466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:13.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 20 client.17466 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139b54e10 0x55d139b572d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 4 client.17466 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d139abbf10 con 0x55d139b54e10 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.480+0000 7f54d27fc640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139b54e10 0x55d139b572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.481+0000 7f54d27fc640 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139b54e10 0x55d139b572d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6f1dc4225a3822a server_cookie=10453fd823c492 in_seq=0 out_seq=0 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.481+0000 7f54b6ff5640 10 client.17466 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:13.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54b6ff5640 1 -- 172.21.15.73:0/2869579390 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d139abbf10 con 0x55d139b54e10 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54b6ff5640 10 client.17466 handle_command_reply: tid=0 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 client.17466 shutdown 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 msgr2=0x55d139b4c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f54c401ec80 tx=0x7f54c4004730 comp rx=0 tx=0).stop 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 shutdown_connections 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139b54e10 0x55d139b572d0 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b4cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44a90 0x55d139b4c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 --2- 172.21.15.73:0/2869579390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44170 0x55d139b4c2e0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.482+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 >> 172.21.15.73:0/2869579390 conn(0x55d139b40680 msgr2=0x55d139b46400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.483+0000 7f54b6ff5640 0 client.17466 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.483+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 shutdown_connections 2024-06-06T07:21:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.483+0000 7f54dc21e740 1 -- 172.21.15.73:0/2869579390 wait complete. 2024-06-06T07:21:13.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.483+0000 7f54dc21e740 20 client.17466 trim_cache size 0 max 16384 2024-06-06T07:21:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.514+0000 7f54dc21e740 1 Processor -- start 2024-06-06T07:21:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.517+0000 7f54dc21e740 1 -- start start 2024-06-06T07:21:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.517+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44170 0x55d139b4cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b46280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b49540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54dc21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41acc60 con 0x55d139b44a90 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54dc21e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ae7a0 con 0x55d139b44170 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54dc21e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ad330 con 0x55d139b4f350 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54d27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b46280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54d27fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b46280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47220/0 (socket says 172.21.15.73:47220) 2024-06-06T07:21:13.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54d27fc640 1 -- 172.21.15.73:0/549626231 learned_addr learned my addr 172.21.15.73:0/549626231 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.518+0000 7f54d37fe640 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b49540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.519+0000 7f54d37fe640 1 -- 172.21.15.73:0/549626231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44170 msgr2=0x55d139b4cb80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.519+0000 7f54d37fe640 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44170 0x55d139b4cb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.519+0000 7f54d37fe640 1 -- 172.21.15.73:0/549626231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 msgr2=0x55d139b46280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.519+0000 7f54d37fe640 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b46280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.519+0000 7f54d37fe640 1 -- 172.21.15.73:0/549626231 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d139b539b0 con 0x55d139b4f350 2024-06-06T07:21:13.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.520+0000 7f54d37fe640 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b49540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009ca0 tx=0x7f54cc02b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ec8ff6840ccaef0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.521+0000 7f54b8ff9640 1 -- 172.21.15.73:0/549626231 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc052070 con 0x55d139b4f350 2024-06-06T07:21:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.521+0000 7f54b8ff9640 1 -- 172.21.15.73:0/549626231 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54cc036040 con 0x55d139b4f350 2024-06-06T07:21:13.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.521+0000 7f54b8ff9640 1 -- 172.21.15.73:0/549626231 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54cc030c60 con 0x55d139b4f350 2024-06-06T07:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.522+0000 7f54dc21e740 1 -- 172.21.15.73:0/549626231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 msgr2=0x55d139b49540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.522+0000 7f54dc21e740 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b49540 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f54cc009ca0 tx=0x7f54cc02b040 comp rx=0 tx=0).stop 2024-06-06T07:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.523+0000 7f54dc21e740 1 -- 172.21.15.73:0/549626231 shutdown_connections 2024-06-06T07:21:13.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.523+0000 7f54dc21e740 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b4f350 0x55d139b49540 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.523+0000 7f54dc21e740 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b46280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.523+0000 7f54dc21e740 1 --2- 172.21.15.73:0/549626231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b44170 0x55d139b4cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.523+0000 7f54dc21e740 1 -- 172.21.15.73:0/549626231 >> 172.21.15.73:0/549626231 conn(0x55d139b40680 msgr2=0x55d139b45b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.525 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.525+0000 7f54dc21e740 1 -- 172.21.15.73:0/549626231 shutdown_connections 2024-06-06T07:21:13.526 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.525+0000 7f54dc21e740 1 -- 172.21.15.73:0/549626231 wait complete. 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.525+0000 7f54dc21e740 1 -- start start 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 Processor -- start 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 0x55d139b5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 0x55d139b5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ae0d0 con 0x55d139b44a90 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f54d41ada00 con 0x55d139b4f350 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f54d4073420 con 0x55d139b44170 2024-06-06T07:21:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.526+0000 7f54dc21e740 10 client.0 _ll_register_callbacks cb 0x7f54c4009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 0x55d139b5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d37fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 0x55d139b5b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57876/0 (socket says 172.21.15.73:57876) 2024-06-06T07:21:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d37fe640 1 -- 172.21.15.73:0/3877375229 learned_addr learned my addr 172.21.15.73:0/3877375229 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 0x55d139b5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d27fc640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d2ffd640 1 -- 172.21.15.73:0/3877375229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 msgr2=0x55d139b5b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.527+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 0x55d139b5b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54d2ffd640 1 -- 172.21.15.73:0/3877375229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 msgr2=0x55d139b5afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b5afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54d2ffd640 1 -- 172.21.15.73:0/3877375229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d139bea9e0 con 0x55d139b44170 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54d27fc640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54d2ffd640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 0x55d139b5aa90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005820 tx=0x7f54c4009fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad91a2dcb272a877 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c4039060 con 0x55d139b44170 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f54c4027440 con 0x55d139b44170 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.528+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54cc036040 con 0x55d139b44170 2024-06-06T07:21:13.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.529+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f54c4040050 con 0x55d139b44170 2024-06-06T07:21:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.531+0000 7f54dc21e740 10 client.26878 fetch_fsmap learned FSMap version 76 2024-06-06T07:21:13.531 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.531+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=4+}) -- 0x55d139bebd30 con 0x55d139b44170 2024-06-06T07:21:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.531+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54c4019050 con 0x55d139b44170 2024-06-06T07:21:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.531+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 76) ==== 8362+0+0 (secure 0 0 0) 0x7f54c4025050 con 0x55d139b44170 2024-06-06T07:21:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.532+0000 7f54dc21e740 10 client.26878 fetch_fsmap finished waiting for FSMap version 76 2024-06-06T07:21:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54dc21e740 10 client.26878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54dc21e740 20 client.26878 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139bec410 0x55d139bee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54dc21e740 4 client.26878 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f54cc036040 con 0x55d139bec410 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.533+0000 7f54d27fc640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139bec410 0x55d139bee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.534+0000 7f54d27fc640 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139bec410 0x55d139bee830 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=624b89d9116190cc server_cookie=bd1dfe8383ebabb0 in_seq=0 out_seq=0 2024-06-06T07:21:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.534+0000 7f54b6ff5640 10 client.26878 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:13.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63623 9.05 MB/sec execute 258 sec latency 86.600 ms 2024-06-06T07:21:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.852+0000 7f54b6ff5640 1 -- 172.21.15.73:0/3877375229 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f54cc036040 con 0x55d139bec410 2024-06-06T07:21:13.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.852+0000 7f54b6ff5640 10 client.26878 handle_command_reply: tid=0 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.852+0000 7f54dc21e740 1 client.26878 shutdown 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 msgr2=0x55d139b5aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 0x55d139b5aa90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f54c4005820 tx=0x7f54c4009fa0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 shutdown_connections 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d139bec410 0x55d139bee830 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d139b4f350 0x55d139b5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d139b44a90 0x55d139b5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 --2- 172.21.15.73:0/3877375229 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d139b44170 0x55d139b5aa90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54b6ff5640 0 client.26878 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 >> 172.21.15.73:0/3877375229 conn(0x55d139b40680 msgr2=0x55d139b3f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 shutdown_connections 2024-06-06T07:21:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 1 -- 172.21.15.73:0/3877375229 wait complete. 2024-06-06T07:21:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54dc21e740 20 client.26878 trim_cache size 0 max 16384 2024-06-06T07:21:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:21:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:21:13.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:21:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 msgr2=0x7f54bc0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.853+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 0x7f54bc0a52e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009ce0 tx=0x7f54c4011810 comp rx=0 tx=0).stop 2024-06-06T07:21:13.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 msgr2=0x7f54d410aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410aee0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f54c800eab0 tx=0x7f54c800ef70 comp rx=0 tx=0).stop 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 shutdown_connections 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f54bc0a2e20 0x7f54bc0a52e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f54d410e360 0x7f54d410b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d41010d0 0x7f54d410aee0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 --2- 172.21.15.73:0/180467527 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f54d41007b0 0x7f54d410dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 >> 172.21.15.73:0/180467527 conn(0x7f54d4077a20 msgr2=0x7f54d410f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 shutdown_connections 2024-06-06T07:21:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:13.854+0000 7f54d9438640 1 -- 172.21.15.73:0/180467527 wait complete. 2024-06-06T07:21:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:13 smithi073 ceph-mon[32303]: pgmap v632: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2024-06-06T07:21:14.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:13 smithi073 ceph-mon[32303]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T07:21:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:13 smithi136 ceph-mon[35901]: pgmap v632: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2024-06-06T07:21:14.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:13 smithi136 ceph-mon[35901]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T07:21:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:13 smithi170 ceph-mon[36108]: pgmap v632: 129 pgs: 129 active+clean; 989 MiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.7 MiB/s wr, 361 op/s 2024-06-06T07:21:14.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:13 smithi170 ceph-mon[36108]: osdmap e136: 12 total, 12 up, 12 in 2024-06-06T07:21:14.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63764 9.05 MB/sec execute 259 sec latency 89.015 ms 2024-06-06T07:21:15.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 63914 9.04 MB/sec execute 260 sec latency 99.795 ms 2024-06-06T07:21:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:15 smithi170 ceph-mon[36108]: pgmap v634: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-06-06T07:21:16.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:15 smithi073 ceph-mon[32303]: pgmap v634: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-06-06T07:21:16.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:15 smithi136 ceph-mon[35901]: pgmap v634: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.1 MiB/s wr, 334 op/s 2024-06-06T07:21:16.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64056 9.02 MB/sec execute 261 sec latency 76.284 ms 2024-06-06T07:21:17.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64207 8.99 MB/sec execute 262 sec latency 115.874 ms 2024-06-06T07:21:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:17 smithi073 ceph-mon[32303]: pgmap v635: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 314 op/s 2024-06-06T07:21:18.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:17 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:17 smithi136 ceph-mon[35901]: pgmap v635: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 314 op/s 2024-06-06T07:21:18.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:17 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:18.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:17 smithi170 ceph-mon[36108]: pgmap v635: 129 pgs: 129 active+clean; 1010 MiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.7 MiB/s wr, 314 op/s 2024-06-06T07:21:18.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:17 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:21:18.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64363 8.98 MB/sec execute 263 sec latency 121.502 ms 2024-06-06T07:21:19.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64520 8.97 MB/sec execute 264 sec latency 76.174 ms 2024-06-06T07:21:20.026 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:19 smithi073 ceph-mon[32303]: pgmap v636: 129 pgs: 129 active+clean; 1022 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 364 op/s 2024-06-06T07:21:20.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:19 smithi136 ceph-mon[35901]: pgmap v636: 129 pgs: 129 active+clean; 1022 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 364 op/s 2024-06-06T07:21:20.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:19 smithi170 ceph-mon[36108]: pgmap v636: 129 pgs: 129 active+clean; 1022 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.7 MiB/s wr, 364 op/s 2024-06-06T07:21:20.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64688 8.98 MB/sec execute 265 sec latency 139.072 ms 2024-06-06T07:21:21.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:20 smithi073 ceph-mon[32303]: pgmap v637: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:21.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:20 smithi136 ceph-mon[35901]: pgmap v637: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:21.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:20 smithi170 ceph-mon[36108]: pgmap v637: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:21.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 64857 8.98 MB/sec execute 266 sec latency 96.350 ms 2024-06-06T07:21:22.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65031 8.98 MB/sec execute 267 sec latency 73.663 ms 2024-06-06T07:21:23.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:23 smithi170 ceph-mon[36108]: pgmap v638: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:23 smithi073 ceph-mon[32303]: pgmap v638: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:23 smithi136 ceph-mon[35901]: pgmap v638: 129 pgs: 129 active+clean; 1021 MiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.4 MiB/s wr, 347 op/s 2024-06-06T07:21:23.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65181 8.97 MB/sec execute 268 sec latency 156.457 ms 2024-06-06T07:21:24.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65371 8.97 MB/sec execute 269 sec latency 180.246 ms 2024-06-06T07:21:25.346 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:25 smithi170 ceph-mon[36108]: pgmap v639: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 457 op/s 2024-06-06T07:21:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:25 smithi073 ceph-mon[32303]: pgmap v639: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 457 op/s 2024-06-06T07:21:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:25 smithi136 ceph-mon[35901]: pgmap v639: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 457 op/s 2024-06-06T07:21:25.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65558 8.97 MB/sec execute 270 sec latency 98.735 ms 2024-06-06T07:21:26.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65719 8.96 MB/sec execute 271 sec latency 64.293 ms 2024-06-06T07:21:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 -- 172.21.15.73:0/1447544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc10bf70 msgr2=0x7f9adc10e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:26.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 --2- 172.21.15.73:0/1447544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc10bf70 0x7f9adc10e410 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009ca0 tx=0x7f9ad402a9e0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 -- 172.21.15.73:0/1447544 shutdown_connections 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 --2- 172.21.15.73:0/1447544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc10bf70 0x7f9adc10e410 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 --2- 172.21.15.73:0/1447544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc103550 0x7f9adc10ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 --2- 172.21.15.73:0/1447544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 -- 172.21.15.73:0/1447544 >> 172.21.15.73:0/1447544 conn(0x7f9adc077a20 msgr2=0x7f9adc077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 -- 172.21.15.73:0/1447544 shutdown_connections 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.992+0000 7f9ae4589640 1 -- 172.21.15.73:0/1447544 wait complete. 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 Processor -- start 2024-06-06T07:21:26.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 -- start start 2024-06-06T07:21:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc10bf70 0x7f9adc19c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc111c20 con 0x7f9adc103550 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc111aa0 con 0x7f9adc10bf70 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae4589640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc111da0 con 0x7f9adc102c30 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae1b24640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae1b24640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35424/0 (socket says 172.21.15.73:35424) 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.993+0000 7f9ae1b24640 1 -- 172.21.15.73:0/3042725674 learned_addr learned my addr 172.21.15.73:0/3042725674 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc10bf70 0x7f9adc19c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 -- 172.21.15.73:0/3042725674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 msgr2=0x7f9adc1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 -- 172.21.15.73:0/3042725674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc10bf70 msgr2=0x7f9adc19c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc10bf70 0x7f9adc19c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 -- 172.21.15.73:0/3042725674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9adc19d0d0 con 0x7f9adc103550 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9f0 tx=0x7f9acc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.994+0000 7f9ad37fe640 1 -- 172.21.15.73:0/3042725674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00cd00 con 0x7f9adc103550 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.995+0000 7f9ad37fe640 1 -- 172.21.15.73:0/3042725674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9acc0108e0 con 0x7f9adc103550 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.995+0000 7f9ad37fe640 1 -- 172.21.15.73:0/3042725674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc018850 con 0x7f9adc103550 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.995+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9adc19d3c0 con 0x7f9adc103550 2024-06-06T07:21:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.995+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9adc1a4c50 con 0x7f9adc103550 2024-06-06T07:21:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.998+0000 7f9ad37fe640 1 -- 172.21.15.73:0/3042725674 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9acc010a90 con 0x7f9adc103550 2024-06-06T07:21:26.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.999+0000 7f9ad37fe640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 0x7f9ac80a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.999+0000 7f9ad37fe640 1 -- 172.21.15.73:0/3042725674 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f9acc0cfa60 con 0x7f9adc103550 2024-06-06T07:21:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.999+0000 7f9ae7372740 1 Processor -- start 2024-06-06T07:21:27.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:26.999+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 0x7f9ac80a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 -- start start 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 0x7f9ac80a5330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009fc0 tx=0x7f9ad8005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x560356369550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356376740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5603562cd4f0 con 0x560356369a90 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56035626b870 con 0x560356374350 2024-06-06T07:21:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae7372740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603562c94b0 con 0x560356369170 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x560356369550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35430/0 (socket says 172.21.15.73:35430) 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 -- 172.21.15.73:0/580626976 learned_addr learned my addr 172.21.15.73:0/580626976 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356376740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 -- 172.21.15.73:0/580626976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 msgr2=0x560356369550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x560356369550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 -- 172.21.15.73:0/580626976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 msgr2=0x560356376740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356376740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.000+0000 7f9ae1b24640 1 -- 172.21.15.73:0/580626976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603562879b0 con 0x560356369a90 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.001+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0985f0 tx=0x7f9acc010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cc1657846d66aab server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.001+0000 7f9ad0bfe640 1 -- 172.21.15.73:0/580626976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc0cd020 con 0x560356369a90 2024-06-06T07:21:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.001+0000 7f9ad0bfe640 1 -- 172.21.15.73:0/580626976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9acc0b3070 con 0x560356369a90 2024-06-06T07:21:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.001+0000 7f9ad0bfe640 1 -- 172.21.15.73:0/580626976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9acc09b340 con 0x560356369a90 2024-06-06T07:21:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- 172.21.15.73:0/580626976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 msgr2=0x560356373e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0985f0 tx=0x7f9acc010f20 comp rx=0 tx=0).stop 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- 172.21.15.73:0/580626976 shutdown_connections 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356376740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356373e10 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 --2- 172.21.15.73:0/580626976 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x560356369550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- 172.21.15.73:0/580626976 >> 172.21.15.73:0/580626976 conn(0x560356365680 msgr2=0x560356365a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- 172.21.15.73:0/580626976 shutdown_connections 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- 172.21.15.73:0/580626976 wait complete. 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.002+0000 7f9ae7372740 1 -- start start 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 Processor -- start 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356371920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356371e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560356376f20 con 0x560356369a90 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560356256ae0 con 0x560356374350 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5603563770a0 con 0x560356369170 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:42816/0 (socket says 172.21.15.73:42816) 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae2325640 1 -- 172.21.15.73:0/3644566050 learned_addr learned my addr 172.21.15.73:0/3644566050 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae1b24640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356371920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356371e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.003+0000 7f9ae7372740 10 client.0 _ll_register_callbacks cb 0x56035628f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 -- 172.21.15.73:0/3644566050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 msgr2=0x560356371e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356371e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 -- 172.21.15.73:0/3644566050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 msgr2=0x560356371920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356371920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 -- 172.21.15.73:0/3644566050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560355fe1210 con 0x560356369170 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356371920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae2325640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8003830 tx=0x7f9ad8004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7d85da33c783379 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad800fb90 con 0x560356369170 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f9ad800fd30 con 0x560356369170 2024-06-06T07:21:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.004+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9acc0b3070 con 0x560356369170 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9ad8026410 con 0x560356369170 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 10 client.26890 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5603560ad620 con 0x560356369170 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ad8026a60 con 0x560356369170 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f9ad801a050 con 0x560356369170 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 10 client.26890 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 10 client.26890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 20 client.26890 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.005+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356386be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.006+0000 7f9ae7372740 4 client.26890 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:21:27.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.006+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f9acc0b3070 con 0x560356384790 2024-06-06T07:21:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.006+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356386be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.006+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356386be0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3abd9410fd923f77 server_cookie=c73229fec000bbe1 in_seq=0 out_seq=0 2024-06-06T07:21:27.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.006+0000 7f9ac5ffb640 10 client.26890 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.007+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/3644566050 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f9acc0b3070 con 0x560356384790 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.007+0000 7f9ac5ffb640 10 client.26890 handle_command_reply: tid=0 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 client.26890 shutdown 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 msgr2=0x5603563713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8003830 tx=0x7f9ad8004730 comp rx=0 tx=0).stop 2024-06-06T07:21:27.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 shutdown_connections 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356386be0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356374350 0x560356371e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ac5ffb640 0 client.26890 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356371920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 --2- 172.21.15.73:0/3644566050 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356369170 0x5603563713e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 >> 172.21.15.73:0/3644566050 conn(0x560356365680 msgr2=0x56035636b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 shutdown_connections 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 1 -- 172.21.15.73:0/3644566050 wait complete. 2024-06-06T07:21:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.008+0000 7f9ae7372740 20 client.26890 trim_cache size 0 max 16384 2024-06-06T07:21:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 Processor -- start 2024-06-06T07:21:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 -- start start 2024-06-06T07:21:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x56035636eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x5603563713a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1aefe0 con 0x560356369a90 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1b0b20 con 0x560356369170 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.033+0000 7f9ae7372740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1af6b0 con 0x560356374350 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:41272/0 (socket says 172.21.15.73:41272) 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 -- 172.21.15.73:0/2490374596 learned_addr learned my addr 172.21.15.73:0/2490374596 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x56035636eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 -- 172.21.15.73:0/2490374596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 msgr2=0x5603563713a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:21:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x5603563713a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 -- 172.21.15.73:0/2490374596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 msgr2=0x56035636eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x56035636eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 -- 172.21.15.73:0/2490374596 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56035636f930 con 0x560356369170 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x56035636eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.034+0000 7f9ae2325640 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011ca0 tx=0x7f9ad80046a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b73b1fa63856231 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.035+0000 7f9ac7fff640 1 -- 172.21.15.73:0/2490374596 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad801d3f0 con 0x560356369170 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.035+0000 7f9ac7fff640 1 -- 172.21.15.73:0/2490374596 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ad801d8e0 con 0x560356369170 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.035+0000 7f9ac7fff640 1 -- 172.21.15.73:0/2490374596 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8026a60 con 0x560356369170 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 -- 172.21.15.73:0/2490374596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 msgr2=0x56035636e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8011ca0 tx=0x7f9ad80046a0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 -- 172.21.15.73:0/2490374596 shutdown_connections 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x5603563713a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x56035636eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 --2- 172.21.15.73:0/2490374596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x56035636e640 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 -- 172.21.15.73:0/2490374596 >> 172.21.15.73:0/2490374596 conn(0x560356365680 msgr2=0x56035636fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 -- 172.21.15.73:0/2490374596 shutdown_connections 2024-06-06T07:21:27.038 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.036+0000 7f9ae7372740 1 -- 172.21.15.73:0/2490374596 wait complete. 2024-06-06T07:21:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 -- start start 2024-06-06T07:21:27.040 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 Processor -- start 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x560356389350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356389890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1b0450 con 0x560356369a90 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1afd80 con 0x560356369170 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9adc1acfe0 con 0x560356374350 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae7372740 10 client.0 _ll_register_callbacks cb 0x7f9ad8004130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae2b26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae2b26640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:42828/0 (socket says 172.21.15.73:42828) 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae2b26640 1 -- 172.21.15.73:0/898574152 learned_addr learned my addr 172.21.15.73:0/898574152 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.037+0000 7f9ae2325640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x560356389350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.041 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356389890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 -- 172.21.15.73:0/898574152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 msgr2=0x560356389350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x560356389350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 -- 172.21.15.73:0/898574152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 msgr2=0x560356389890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356389890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 -- 172.21.15.73:0/898574152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603563fe560 con 0x560356374350 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae1b24640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356389890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae2b26640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009ca0 tx=0x7f9ad4003ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c15ec5236459a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ac5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad4049020 con 0x560356374350 2024-06-06T07:21:27.042 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ad801d8e0 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.038+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f9ad402e590 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f9ad4054050 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 10 client.26893 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x5603563ff880 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ad801d8e0 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f9ad4003fb0 con 0x560356374350 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 10 client.26893 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 10 client.26893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 20 client.26893 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356402070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 4 client.26893 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603562e0f10 con 0x560356384790 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.039+0000 7f9ae2325640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356402070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:27.043 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.040+0000 7f9ae2325640 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356402070 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3cb3845d68c89ca server_cookie=7cc3ed2cd3025e3b in_seq=0 out_seq=0 2024-06-06T07:21:27.044 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.040+0000 7f9ac5ffb640 10 client.26893 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.092+0000 7f9ac5ffb640 1 -- 172.21.15.73:0/898574152 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603562e0f10 con 0x560356384790 2024-06-06T07:21:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.092+0000 7f9ac5ffb640 10 client.26893 handle_command_reply: tid=0 2024-06-06T07:21:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 client.26893 shutdown 2024-06-06T07:21:27.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 msgr2=0x560356389dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4009ca0 tx=0x7f9ad4003ba0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 shutdown_connections 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560356384790 0x560356402070 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560356374350 0x560356389dd0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ac5ffb640 0 client.26893 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560356369a90 0x560356389890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 --2- 172.21.15.73:0/898574152 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560356369170 0x560356389350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 >> 172.21.15.73:0/898574152 conn(0x560356365680 msgr2=0x5603563713a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 shutdown_connections 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.093+0000 7f9ae7372740 1 -- 172.21.15.73:0/898574152 wait complete. 2024-06-06T07:21:27.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae7372740 20 client.26893 trim_cache size 0 max 16384 2024-06-06T07:21:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:21:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:21:27.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:21:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 msgr2=0x7f9ac80a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 0x7f9ac80a5330 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9ad8009fc0 tx=0x7f9ad8005f90 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 msgr2=0x7f9adc198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9acc00e9f0 tx=0x7f9acc00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 shutdown_connections 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9adc10bf70 0x7f9adc19c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9adc103550 0x7f9adc198620 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9ac80a2e70 0x7f9ac80a5330 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 --2- 172.21.15.73:0/3042725674 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9adc102c30 0x7f9adc1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 >> 172.21.15.73:0/3042725674 conn(0x7f9adc077a20 msgr2=0x7f9adc0fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 shutdown_connections 2024-06-06T07:21:27.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:27.094+0000 7f9ae4589640 1 -- 172.21.15.73:0/3042725674 wait complete. 2024-06-06T07:21:27.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:27 smithi170 ceph-mon[36108]: pgmap v640: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-06-06T07:21:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:27 smithi073 ceph-mon[32303]: pgmap v640: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-06-06T07:21:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:27 smithi136 ceph-mon[35901]: pgmap v640: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.5 MiB/s wr, 295 op/s 2024-06-06T07:21:27.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 65893 8.98 MB/sec execute 272 sec latency 73.590 ms 2024-06-06T07:21:28.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66050 8.97 MB/sec execute 273 sec latency 85.720 ms 2024-06-06T07:21:29.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:29 smithi170 ceph-mon[36108]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2024-06-06T07:21:29.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:29 smithi073 ceph-mon[32303]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2024-06-06T07:21:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:29 smithi136 ceph-mon[35901]: pgmap v641: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.6 MiB/s wr, 425 op/s 2024-06-06T07:21:29.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66269 9.01 MB/sec execute 274 sec latency 82.979 ms 2024-06-06T07:21:30.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66454 9.02 MB/sec execute 275 sec latency 193.381 ms 2024-06-06T07:21:31.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:31 smithi170 ceph-mon[36108]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-06-06T07:21:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:31 smithi073 ceph-mon[32303]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-06-06T07:21:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:31 smithi136 ceph-mon[35901]: pgmap v642: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 281 op/s 2024-06-06T07:21:31.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66638 9.03 MB/sec execute 276 sec latency 63.881 ms 2024-06-06T07:21:32.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66791 9.03 MB/sec execute 277 sec latency 127.460 ms 2024-06-06T07:21:33.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:33 smithi170 ceph-mon[36108]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-06-06T07:21:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:33 smithi073 ceph-mon[32303]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-06-06T07:21:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:33 smithi136 ceph-mon[35901]: pgmap v643: 129 pgs: 129 active+clean; 1.0 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 277 op/s 2024-06-06T07:21:33.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 66931 9.03 MB/sec execute 278 sec latency 76.025 ms 2024-06-06T07:21:34.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67075 9.02 MB/sec execute 279 sec latency 89.633 ms 2024-06-06T07:21:35.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:35 smithi170 ceph-mon[36108]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:21:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:35 smithi073 ceph-mon[32303]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:21:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:35 smithi136 ceph-mon[35901]: pgmap v644: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:21:35.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67247 9.03 MB/sec execute 280 sec latency 109.670 ms 2024-06-06T07:21:36.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67403 9.02 MB/sec execute 281 sec latency 104.725 ms 2024-06-06T07:21:37.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:37 smithi170 ceph-mon[36108]: pgmap v645: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 258 op/s 2024-06-06T07:21:37.352 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:37 smithi136 ceph-mon[35901]: pgmap v645: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 258 op/s 2024-06-06T07:21:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:37 smithi073 ceph-mon[32303]: pgmap v645: 129 pgs: 129 active+clean; 1.0 GiB data, 6.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.2 MiB/s wr, 258 op/s 2024-06-06T07:21:37.813 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67549 9.00 MB/sec execute 282 sec latency 80.475 ms 2024-06-06T07:21:38.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67699 8.99 MB/sec execute 283 sec latency 137.657 ms 2024-06-06T07:21:39.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:39 smithi170 ceph-mon[36108]: pgmap v646: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 389 op/s 2024-06-06T07:21:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:39 smithi073 ceph-mon[32303]: pgmap v646: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 389 op/s 2024-06-06T07:21:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:39 smithi136 ceph-mon[35901]: pgmap v646: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.7 MiB/s wr, 389 op/s 2024-06-06T07:21:39.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 67858 8.99 MB/sec execute 284 sec latency 104.294 ms 2024-06-06T07:21:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/1412109483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24103550 msgr2=0x7f2a2410ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.219 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/1412109483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24103550 0x7f2a2410ba30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2a14009860 tx=0x7f2a1402a460 comp rx=0 tx=0).stop 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/1412109483 shutdown_connections 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/1412109483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a2410bf70 0x7f2a2410e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/1412109483 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24103550 0x7f2a2410ba30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/1412109483 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a24102c30 0x7f2a24103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/1412109483 >> 172.21.15.73:0/1412109483 conn(0x7f2a24077a20 msgr2=0x7f2a24077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/1412109483 shutdown_connections 2024-06-06T07:21:40.220 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.218+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/1412109483 wait complete. 2024-06-06T07:21:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.219+0000 7f2a2b6ee640 1 Processor -- start 2024-06-06T07:21:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.219+0000 7f2a2b6ee640 1 -- start start 2024-06-06T07:21:40.223 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2410bf70 0x7f2a241052f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410fa60 con 0x7f2a24103550 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410f8e0 con 0x7f2a24102c30 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2b6ee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a2410fbe0 con 0x7f2a2410bf70 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.224 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:37408/0 (socket says 172.21.15.73:37408) 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2948a640 1 -- 172.21.15.73:0/3699459808 learned_addr learned my addr 172.21.15.73:0/3699459808 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50548/0 (socket says 172.21.15.73:50548) 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2410bf70 0x7f2a241052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 -- 172.21.15.73:0/3699459808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2410bf70 msgr2=0x7f2a241052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2410bf70 0x7f2a241052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 -- 172.21.15.73:0/3699459808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 msgr2=0x7f2a2410b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.225 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a28c89640 1 -- 172.21.15.73:0/3699459808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a24105bb0 con 0x7f2a24103550 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.220+0000 7f2a2948a640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a28c89640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14030fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a1a7fc640 1 -- 172.21.15.73:0/3699459808 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14040070 con 0x7f2a24103550 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a24105e40 con 0x7f2a24103550 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a1a7fc640 1 -- 172.21.15.73:0/3699459808 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a14005610 con 0x7f2a24103550 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a1a7fc640 1 -- 172.21.15.73:0/3699459808 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a14004d70 con 0x7f2a24103550 2024-06-06T07:21:40.226 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.221+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a241b3bd0 con 0x7f2a24103550 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 Processor -- start 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 -- start start 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 0x55799e34ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e34d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55799e2a44f0 con 0x55799e340170 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799e242870 con 0x55799e340a90 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55799e2a04b0 con 0x55799e34b350 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a1a7fc640 1 -- 172.21.15.73:0/3699459808 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f2a14039040 con 0x7f2a24103550 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a29c8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.234 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e34d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a29c8b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50550/0 (socket says 172.21.15.73:50550) 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a29c8b640 1 -- 172.21.15.73:0/74368823 learned_addr learned my addr 172.21.15.73:0/74368823 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.224+0000 7f2a2948a640 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 0x55799e34ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 -- 172.21.15.73:0/74368823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 msgr2=0x55799e34d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e34d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 -- 172.21.15.73:0/74368823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 msgr2=0x55799e340550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 -- 172.21.15.73:0/74368823 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799e25e9b0 con 0x55799e340a90 2024-06-06T07:21:40.235 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a2948a640 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 0x55799e34ae10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a20011510 tx=0x7f2a200119d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23d7e1bf54eca646 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a0f7fe640 1 -- 172.21.15.73:0/74368823 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2001e070 con 0x55799e340a90 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a1a7fc640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 0x7f2a100a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a0f7fe640 1 -- 172.21.15.73:0/74368823 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a200035f0 con 0x55799e340a90 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 0x7f2a100a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.225+0000 7f2a1a7fc640 1 -- 172.21.15.73:0/3699459808 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f2a140f0740 con 0x7f2a24103550 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.226+0000 7f2a0f7fe640 1 -- 172.21.15.73:0/74368823 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2000f820 con 0x55799e340a90 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.226+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/74368823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 msgr2=0x55799e34ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.226+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 0x55799e34ae10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2a20011510 tx=0x7f2a200119d0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.236 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.226+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 0x7f2a100a5350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005f40 tx=0x7f2a1c0096a0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.226+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/74368823 shutdown_connections 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e34d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340a90 0x55799e34ae10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/74368823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e340550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/74368823 >> 172.21.15.73:0/74368823 conn(0x55799e33c680 msgr2=0x55799e33ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/74368823 shutdown_connections 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/74368823 wait complete. 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 -- start start 2024-06-06T07:21:40.237 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.227+0000 7f2a2e3fa740 1 Processor -- start 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e348280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e3487c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e348d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55799e34dfd0 con 0x55799e340a90 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55799e22dae0 con 0x55799e340170 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55799e34e150 con 0x55799e34b350 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2e3fa740 10 client.0 _ll_register_callbacks cb 0x55799e2669b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e3487c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e348280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e3487c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50554/0 (socket says 172.21.15.73:50554) 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a28c89640 1 -- 172.21.15.73:0/1425354692 learned_addr learned my addr 172.21.15.73:0/1425354692 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e348d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.238 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 -- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 msgr2=0x55799e348d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e348d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 -- 172.21.15.73:0/1425354692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 msgr2=0x55799e3487c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e3487c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 -- 172.21.15.73:0/1425354692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799dfb8210 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e348280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a200058c0 tx=0x7f2a200039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693f02c12aa142fd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.228+0000 7f2a0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a2001e070 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f2a20016040 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a200035f0 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f2a20009cd0 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 10 client.26726 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55799e084620 con 0x55799e340170 2024-06-06T07:21:40.239 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a200035f0 con 0x55799e340170 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f2a20004b80 con 0x55799e340170 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 10 client.26726 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 10 client.26726 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.229+0000 7f2a2e3fa740 20 client.26726 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e35b880 0x55799e35dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a2e3fa740 4 client.26726 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55799e2b7f10 con 0x55799e35b880 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a28c89640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e35b880 0x55799e35dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a28c89640 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e35b880 0x55799e35dd10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a980a70354a98aec server_cookie=a0aa71baa97a0167 in_seq=0 out_seq=0 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.230+0000 7f2a0cff9640 10 client.26726 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1425354692 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55799e2b7f10 con 0x55799e35b880 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a0cff9640 10 client.26726 handle_command_reply: tid=0 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 client.26726 shutdown 2024-06-06T07:21:40.240 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 msgr2=0x55799e348280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e348280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a200058c0 tx=0x7f2a200039d0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 shutdown_connections 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e35b880 0x55799e35dd10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e348d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e3487c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1425354692 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e348280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 >> 172.21.15.73:0/1425354692 conn(0x55799e33c680 msgr2=0x55799e342400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.231+0000 7f2a0cff9640 0 client.26726 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.232+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 shutdown_connections 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.232+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1425354692 wait complete. 2024-06-06T07:21:40.241 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.232+0000 7f2a2e3fa740 20 client.26726 trim_cache size 0 max 16384 2024-06-06T07:21:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 Processor -- start 2024-06-06T07:21:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 -- start start 2024-06-06T07:21:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e342280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e3427c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241bddb0 con 0x55799e340a90 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241bf8f0 con 0x55799e340170 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241be480 con 0x55799e34b350 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e342280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:37436/0 (socket says 172.21.15.73:37436) 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a29c8b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e3427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.257+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e342280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50564/0 (socket says 172.21.15.73:50564) 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 -- 172.21.15.73:0/1846712833 learned_addr learned my addr 172.21.15.73:0/1846712833 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 -- 172.21.15.73:0/1846712833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 msgr2=0x55799e3427c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e3427c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 -- 172.21.15.73:0/1846712833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 msgr2=0x55799e342280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e342280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 -- 172.21.15.73:0/1846712833 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55799e3467d0 con 0x55799e340170 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a20001eb0 tx=0x7f2a20009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb350f27a41b5b51 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a0effd640 1 -- 172.21.15.73:0/1846712833 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a200035f0 con 0x55799e340170 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a0effd640 1 -- 172.21.15.73:0/1846712833 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2001f040 con 0x55799e340170 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.258+0000 7f2a0effd640 1 -- 172.21.15.73:0/1846712833 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a20005520 con 0x55799e340170 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1846712833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 msgr2=0x55799e35dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2a20001eb0 tx=0x7f2a20009ba0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1846712833 shutdown_connections 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e34b350 0x55799e3427c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340a90 0x55799e342280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1846712833 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e340170 0x55799e35dcf0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1846712833 >> 172.21.15.73:0/1846712833 conn(0x55799e33c680 msgr2=0x55799e341b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1846712833 shutdown_connections 2024-06-06T07:21:40.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.259+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1846712833 wait complete. 2024-06-06T07:21:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.260+0000 7f2a2e3fa740 1 -- start start 2024-06-06T07:21:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.260+0000 7f2a2e3fa740 1 Processor -- start 2024-06-06T07:21:40.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.260+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e34e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.260+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 0x55799e34e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2e3fa740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e34b350 0x55799e34ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e34e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241bf220 con 0x55799e340170 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 0x55799e34e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241beb50 con 0x55799e34b350 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2e3fa740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2a241bbdb0 con 0x55799e340a90 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2948a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e34e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50570/0 (socket says 172.21.15.73:50570) 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2948a640 1 -- 172.21.15.73:0/1437647844 learned_addr learned my addr 172.21.15.73:0/1437647844 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a29c8b640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e34b350 0x55799e34ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.267 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a2e3fa740 10 client.0 _ll_register_callbacks cb 0x7f2a2001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 -- 172.21.15.73:0/1437647844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e34b350 msgr2=0x55799e34ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e34b350 0x55799e34ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 -- 172.21.15.73:0/1437647844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 msgr2=0x55799e34e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e34e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 -- 172.21.15.73:0/1437647844 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55799e3c4790 con 0x55799e340a90 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a28c89640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 0x55799e34e8a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a1402aa50 tx=0x7f2a140f6530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba2a80428fede6df server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a0cff9640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.261+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a140d5060 con 0x55799e340a90 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7f2a140a70b0 con 0x55799e340a90 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a2001f040 con 0x55799e340a90 2024-06-06T07:21:40.268 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f2a140ef070 con 0x55799e340a90 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 10 client.26911 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55799e3c5aa0 con 0x55799e340a90 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a2001f040 con 0x55799e340a90 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f2a140b8e60 con 0x55799e340a90 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 10 client.26911 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 10 client.26911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 20 client.26911 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e3c6180 0x55799e3c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 4 client.26911 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.262+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55799e2b1c20 con 0x55799e3c6180 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.263+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e3c6180 0x55799e3c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.263+0000 7f2a2948a640 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e3c6180 0x55799e3c85a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1b2922afe6da91 server_cookie=ed7a4a67396ffc80 in_seq=0 out_seq=0 2024-06-06T07:21:40.269 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.263+0000 7f2a0cff9640 10 client.26911 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a0cff9640 1 -- 172.21.15.73:0/1437647844 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55799e2b1c20 con 0x55799e3c6180 2024-06-06T07:21:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a0cff9640 10 client.26911 handle_command_reply: tid=0 2024-06-06T07:21:40.336 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 client.26911 shutdown 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 msgr2=0x55799e34e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 0x55799e34e8a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2a1402aa50 tx=0x7f2a140f6530 comp rx=0 tx=0).stop 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 shutdown_connections 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55799e3c6180 0x55799e3c85a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55799e34b350 0x55799e34ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55799e340a90 0x55799e34e8a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 --2- 172.21.15.73:0/1437647844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55799e340170 0x55799e34e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 >> 172.21.15.73:0/1437647844 conn(0x55799e33c680 msgr2=0x55799e3423b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.337 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a0cff9640 0 client.26911 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.334+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 shutdown_connections 2024-06-06T07:21:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.335+0000 7f2a2e3fa740 1 -- 172.21.15.73:0/1437647844 wait complete. 2024-06-06T07:21:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.335+0000 7f2a2e3fa740 20 client.26911 trim_cache size 0 max 16384 2024-06-06T07:21:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:21:40.338 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:21:40.339 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:21:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.348+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 msgr2=0x7f2a100a5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.348+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 0x7f2a100a5350 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c005f40 tx=0x7f2a1c0096a0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.348+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 msgr2=0x7f2a2410b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.348+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f2a14000c00 tx=0x7f2a14030fa0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 shutdown_connections 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f2a100a2e90 0x7f2a100a5350 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2a2410bf70 0x7f2a241052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a24103550 0x7f2a2410b7c0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 --2- 172.21.15.73:0/3699459808 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2a24102c30 0x7f2a2410b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 >> 172.21.15.73:0/3699459808 conn(0x7f2a24077a20 msgr2=0x7f2a240fd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 shutdown_connections 2024-06-06T07:21:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:40.349+0000 7f2a2b6ee640 1 -- 172.21.15.73:0/3699459808 wait complete. 2024-06-06T07:21:40.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68036 8.99 MB/sec execute 285 sec latency 75.217 ms 2024-06-06T07:21:41.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:41 smithi170 ceph-mon[36108]: pgmap v647: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 262 op/s 2024-06-06T07:21:41.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:41 smithi073 ceph-mon[32303]: pgmap v647: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 262 op/s 2024-06-06T07:21:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:41 smithi136 ceph-mon[35901]: pgmap v647: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.7 MiB/s wr, 262 op/s 2024-06-06T07:21:41.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68175 8.98 MB/sec execute 286 sec latency 342.259 ms 2024-06-06T07:21:42.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68336 8.99 MB/sec execute 287 sec latency 71.600 ms 2024-06-06T07:21:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:43 smithi073 ceph-mon[32303]: pgmap v648: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 258 op/s 2024-06-06T07:21:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:43 smithi136 ceph-mon[35901]: pgmap v648: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 258 op/s 2024-06-06T07:21:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:43 smithi170 ceph-mon[36108]: pgmap v648: 129 pgs: 129 active+clean; 1.0 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.6 MiB/s wr, 258 op/s 2024-06-06T07:21:43.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68499 8.99 MB/sec execute 288 sec latency 68.051 ms 2024-06-06T07:21:44.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68647 8.97 MB/sec execute 289 sec latency 68.785 ms 2024-06-06T07:21:45.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:45 smithi073 ceph-mon[32303]: pgmap v649: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2024-06-06T07:21:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:45 smithi136 ceph-mon[35901]: pgmap v649: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2024-06-06T07:21:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:45 smithi170 ceph-mon[36108]: pgmap v649: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 425 op/s 2024-06-06T07:21:45.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 68816 8.96 MB/sec execute 290 sec latency 111.885 ms 2024-06-06T07:21:46.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69016 8.97 MB/sec execute 291 sec latency 74.243 ms 2024-06-06T07:21:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:47 smithi073 ceph-mon[32303]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-06-06T07:21:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:47 smithi136 ceph-mon[35901]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-06-06T07:21:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:47 smithi170 ceph-mon[36108]: pgmap v650: 129 pgs: 129 active+clean; 1.0 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 4.8 MiB/s wr, 299 op/s 2024-06-06T07:21:47.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69188 8.96 MB/sec execute 292 sec latency 133.489 ms 2024-06-06T07:21:48.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69317 8.94 MB/sec execute 293 sec latency 80.065 ms 2024-06-06T07:21:49.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:48 smithi073 ceph-mon[32303]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 420 op/s 2024-06-06T07:21:49.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:48 smithi136 ceph-mon[35901]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 420 op/s 2024-06-06T07:21:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:48 smithi170 ceph-mon[36108]: pgmap v651: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 420 op/s 2024-06-06T07:21:49.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69502 8.97 MB/sec execute 294 sec latency 65.154 ms 2024-06-06T07:21:50.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69722 8.99 MB/sec execute 295 sec latency 62.797 ms 2024-06-06T07:21:51.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:51 smithi170 ceph-mon[36108]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-06-06T07:21:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:51 smithi073 ceph-mon[32303]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-06-06T07:21:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:51 smithi136 ceph-mon[35901]: pgmap v652: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 298 op/s 2024-06-06T07:21:51.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 69912 9.02 MB/sec execute 296 sec latency 110.637 ms 2024-06-06T07:21:52.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70052 9.00 MB/sec execute 297 sec latency 58.085 ms 2024-06-06T07:21:53.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:53 smithi170 ceph-mon[36108]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-06-06T07:21:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 -- 172.21.15.73:0/2140956468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 msgr2=0x7fd91810d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 --2- 172.21.15.73:0/2140956468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91810d780 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fd910009cd0 tx=0x7fd91002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 -- 172.21.15.73:0/2140956468 shutdown_connections 2024-06-06T07:21:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 --2- 172.21.15.73:0/2140956468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91810d780 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.480 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 --2- 172.21.15.73:0/2140956468 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd918100900 0x7fd918100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 --2- 172.21.15.73:0/2140956468 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9181068c0 0x7fd918106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.479+0000 7fd91cbce640 1 -- 172.21.15.73:0/2140956468 >> 172.21.15.73:0/2140956468 conn(0x7fd9180779f0 msgr2=0x7fd918077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.481+0000 7fd91cbce640 1 -- 172.21.15.73:0/2140956468 shutdown_connections 2024-06-06T07:21:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.481+0000 7fd91cbce640 1 -- 172.21.15.73:0/2140956468 wait complete. 2024-06-06T07:21:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.482+0000 7fd91cbce640 1 Processor -- start 2024-06-06T07:21:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.483+0000 7fd91cbce640 1 -- start start 2024-06-06T07:21:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.483+0000 7fd91cbce640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.483+0000 7fd91cbce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91659c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:34084/0 (socket says 172.21.15.73:34084) 2024-06-06T07:21:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91659c640 1 -- 172.21.15.73:0/3262979199 learned_addr learned my addr 172.21.15.73:0/3262979199 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9181068c0 0x7fd91819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd915d9b640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111cd0 con 0x7fd9181012c0 2024-06-06T07:21:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111b50 con 0x7fd9181068c0 2024-06-06T07:21:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.484+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd918111e50 con 0x7fd918100900 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.485+0000 7fd91659c640 1 -- 172.21.15.73:0/3262979199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9181068c0 msgr2=0x7fd91819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.485+0000 7fd91659c640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9181068c0 0x7fd91819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.485+0000 7fd91659c640 1 -- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 msgr2=0x7fd91819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.486+0000 7fd91659c640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.486+0000 7fd91659c640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd91819f290 con 0x7fd918100900 2024-06-06T07:21:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.486+0000 7fd915d9b640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:21:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.487+0000 7fd91659c640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9f0 tx=0x7fd90c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.488+0000 7fd9077fe640 1 -- 172.21.15.73:0/3262979199 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c00cd80 con 0x7fd918100900 2024-06-06T07:21:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.488+0000 7fd9077fe640 1 -- 172.21.15.73:0/3262979199 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c0108e0 con 0x7fd918100900 2024-06-06T07:21:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.488+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd91819f580 con 0x7fd918100900 2024-06-06T07:21:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.488+0000 7fd9077fe640 1 -- 172.21.15.73:0/3262979199 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c018920 con 0x7fd918100900 2024-06-06T07:21:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.488+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9181a6e10 con 0x7fd918100900 2024-06-06T07:21:53.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.492+0000 7fd9077fe640 1 -- 172.21.15.73:0/3262979199 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd90c010a90 con 0x7fd918100900 2024-06-06T07:21:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.492+0000 7fd91f9bb740 1 Processor -- start 2024-06-06T07:21:53.493 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 -- start start 2024-06-06T07:21:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 0x557357f42490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f55620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557357ea64f0 con 0x557357f55b60 2024-06-06T07:21:53.494 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557357e44870 con 0x557357f429d0 2024-06-06T07:21:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557357ea24b0 con 0x557357f420b0 2024-06-06T07:21:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.493+0000 7fd915d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34588/0 (socket says 172.21.15.73:34588) 2024-06-06T07:21:53.503 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 -- 172.21.15.73:0/3147418005 learned_addr learned my addr 172.21.15.73:0/3147418005 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd916d9d640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 0x557357f42490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd9077fe640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 0x7fd9000a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 -- 172.21.15.73:0/3147418005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 msgr2=0x557357f42490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 0x557357f42490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 -- 172.21.15.73:0/3147418005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 msgr2=0x557357f55620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd91659c640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f55620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f55620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 -- 172.21.15.73:0/3147418005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557357e609b0 con 0x557357f55b60 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd916d9d640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 0x557357f42490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd9077fe640 1 -- 172.21.15.73:0/3262979199 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7fd90c0cfac0 con 0x7fd918100900 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.494+0000 7fd915d9b640 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd9080114f0 tx=0x7fd9080119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13da99e219ef7758 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.495+0000 7fd904ff9640 1 -- 172.21.15.73:0/3147418005 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90801e070 con 0x557357f55b60 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.495+0000 7fd904ff9640 1 -- 172.21.15.73:0/3147418005 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9080035f0 con 0x557357f55b60 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.495+0000 7fd904ff9640 1 -- 172.21.15.73:0/3147418005 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90800f820 con 0x557357f55b60 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd91f9bb740 1 -- 172.21.15.73:0/3147418005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 msgr2=0x557357f57f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fd9080114f0 tx=0x7fd9080119b0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd916d9d640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 0x7fd9000a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd91f9bb740 1 -- 172.21.15.73:0/3147418005 shutdown_connections 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f57f50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.496+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f55620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/3147418005 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f420b0 0x557357f42490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 -- 172.21.15.73:0/3147418005 >> 172.21.15.73:0/3147418005 conn(0x557357f3e680 msgr2=0x557357f3ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 -- 172.21.15.73:0/3147418005 shutdown_connections 2024-06-06T07:21:53.506 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 -- 172.21.15.73:0/3147418005 wait complete. 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd916d9d640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 0x7fd9000a53c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd910004ab0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 -- start start 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.497+0000 7fd91f9bb740 1 Processor -- start 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f530e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f53620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557357f58730 con 0x557357f420b0 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x557357e2fae0 con 0x557357f429d0 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x557357f588b0 con 0x557357f55b60 2024-06-06T07:21:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91f9bb740 10 client.0 _ll_register_callbacks cb 0x557357e689b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34604/0 (socket says 172.21.15.73:34604) 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.498+0000 7fd91659c640 1 -- 172.21.15.73:0/894199023 learned_addr learned my addr 172.21.15.73:0/894199023 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd915d9b640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f530e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 -- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 msgr2=0x557357f53620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f53620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 -- 172.21.15.73:0/894199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 msgr2=0x557357f530e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f530e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 -- 172.21.15.73:0/894199023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557357bba210 con 0x557357f420b0 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd91659c640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd90c097850 tx=0x7fd90c010de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00a55f5acfd41d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.499+0000 7fd8fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0cd230 con 0x557357f420b0 2024-06-06T07:21:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7fd90c010630 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9080035f0 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd90c09ff20 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd91f9bb740 10 client.17517 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.500+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x557357c86620 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9080035f0 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7fd90c097a60 con 0x557357f420b0 2024-06-06T07:21:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 10 client.17517 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 20 client.17517 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357f662f0 0x557357f68720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 4 client.17517 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.501+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557357eb9f10 con 0x557357f662f0 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.502+0000 7fd915d9b640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357f662f0 0x557357f68720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.503+0000 7fd915d9b640 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357f662f0 0x557357f68720 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7afbcb68663cdb server_cookie=8da9f94bedb650d3 in_seq=0 out_seq=0 2024-06-06T07:21:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.503+0000 7fd8fdffb640 10 client.17517 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd8fdffb640 1 -- 172.21.15.73:0/894199023 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557357eb9f10 con 0x557357f662f0 2024-06-06T07:21:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd8fdffb640 10 client.17517 handle_command_reply: tid=0 2024-06-06T07:21:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 client.17517 shutdown 2024-06-06T07:21:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 msgr2=0x557357f52ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fd90c097850 tx=0x7fd90c010de0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 shutdown_connections 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357f662f0 0x557357f68720 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.504+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f53620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f530e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd8fdffb640 0 client.17517 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/894199023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f52ba0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 >> 172.21.15.73:0/894199023 conn(0x557357f3e680 msgr2=0x557357f44400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 shutdown_connections 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 1 -- 172.21.15.73:0/894199023 wait complete. 2024-06-06T07:21:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.505+0000 7fd91f9bb740 20 client.17517 trim_cache size 0 max 16384 2024-06-06T07:21:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:53 smithi073 ceph-mon[32303]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 Processor -- start 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 -- start start 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f3d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f6e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181acd10 con 0x557357f420b0 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181ae850 con 0x557357f429d0 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91f9bb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181ad3e0 con 0x557357f55b60 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91659c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34618/0 (socket says 172.21.15.73:34618) 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd91659c640 1 -- 172.21.15.73:0/267992240 learned_addr learned my addr 172.21.15.73:0/267992240 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd915d9b640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f3d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.528+0000 7fd916d9d640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f6e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 -- 172.21.15.73:0/267992240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 msgr2=0x557357f6e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f6e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 -- 172.21.15.73:0/267992240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 msgr2=0x557357f3d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f3d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 -- 172.21.15.73:0/267992240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557357f51b50 con 0x557357f420b0 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd915d9b640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f3d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd91659c640 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd90c097fd0 tx=0x7fd90c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1bc07a5b8473d8d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd8fffff640 1 -- 172.21.15.73:0/267992240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0a9080 con 0x557357f420b0 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd8fffff640 1 -- 172.21.15.73:0/267992240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c09b1c0 con 0x557357f420b0 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.529+0000 7fd8fffff640 1 -- 172.21.15.73:0/267992240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd90c0ba500 con 0x557357f420b0 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 -- 172.21.15.73:0/267992240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 msgr2=0x557357f53440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fd90c097fd0 tx=0x7fd90c009310 comp rx=0 tx=0).stop 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 -- 172.21.15.73:0/267992240 shutdown_connections 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f55b60 0x557357f6e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f429d0 0x557357f3d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.533 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.530+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/267992240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f420b0 0x557357f53440 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.531+0000 7fd91f9bb740 1 -- 172.21.15.73:0/267992240 >> 172.21.15.73:0/267992240 conn(0x557357f3e680 msgr2=0x557357f3ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.531+0000 7fd91f9bb740 1 -- 172.21.15.73:0/267992240 shutdown_connections 2024-06-06T07:21:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.531+0000 7fd91f9bb740 1 -- 172.21.15.73:0/267992240 wait complete. 2024-06-06T07:21:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.531+0000 7fd91f9bb740 1 -- start start 2024-06-06T07:21:53.534 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.532+0000 7fd91f9bb740 1 Processor -- start 2024-06-06T07:21:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.532+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 0x557357f5de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.532+0000 7fd91f9bb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f429d0 0x557357f5e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.532+0000 7fd91659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 0x557357f5de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.536 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.532+0000 7fd91659c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 0x557357f5de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:58538/0 (socket says 172.21.15.73:58538) 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd91659c640 1 -- 172.21.15.73:0/585176769 learned_addr learned my addr 172.21.15.73:0/585176769 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f4f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181ae180 con 0x557357f55b60 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd915d9b640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f429d0 0x557357f5e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181adab0 con 0x557357f420b0 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9181aad30 con 0x557357f429d0 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.533+0000 7fd916d9d640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f4f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd91f9bb740 10 client.0 _ll_register_callbacks cb 0x7fd90c009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 -- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f429d0 msgr2=0x557357f5e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f429d0 0x557357f5e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.537 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 -- 172.21.15.73:0/585176769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 msgr2=0x557357f5de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 0x557357f5de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557357f582c0 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd916d9d640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f4f5e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd910006010 tx=0x7fd9100095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bb0a58725d9742 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.534+0000 7fd8fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd910003c40 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(136..136 src has 1..136) ==== 9554+0+0 (secure 0 0 0) 0x7fd910003de0 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd90c09b1c0 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fd91005a050 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd91f9bb740 10 client.17523 fetch_fsmap learned FSMap version 77 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.535+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x557357fd09f0 con 0x557357f55b60 2024-06-06T07:21:53.538 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd90c09b1c0 con 0x557357f55b60 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7fd910030070 con 0x557357f55b60 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 10 client.17523 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 10 client.17523 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 20 client.17523 populate_metadata read hostname 'smithi073' 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357fd1110 0x557357fd3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 4 client.17523 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557357eb3c20 con 0x557357fd1110 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.536+0000 7fd91659c640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357fd1110 0x557357fd3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.537+0000 7fd8fdffb640 10 client.17523 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:53.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.537+0000 7fd91659c640 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357fd1110 0x557357fd3530 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fade18502aa80c3c server_cookie=7b7501ab8faf04f4 in_seq=0 out_seq=0 2024-06-06T07:21:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:53 smithi136 ceph-mon[35901]: pgmap v653: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.3 MiB/s wr, 296 op/s 2024-06-06T07:21:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd8fdffb640 1 -- 172.21.15.73:0/585176769 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557357eb3c20 con 0x557357fd1110 2024-06-06T07:21:53.731 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd8fdffb640 10 client.17523 handle_command_reply: tid=0 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 client.17523 shutdown 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 msgr2=0x557357f4f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f4f5e0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fd910006010 tx=0x7fd9100095c0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 shutdown_connections 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x557357fd1110 0x557357fd3530 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557357f55b60 0x557357f4f5e0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x557357f429d0 0x557357f5e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 --2- 172.21.15.73:0/585176769 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x557357f420b0 0x557357f5de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 >> 172.21.15.73:0/585176769 conn(0x557357f3e680 msgr2=0x557357f443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd8fdffb640 0 client.17523 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:21:53.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 shutdown_connections 2024-06-06T07:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.730+0000 7fd91f9bb740 1 -- 172.21.15.73:0/585176769 wait complete. 2024-06-06T07:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91f9bb740 20 client.17523 trim_cache size 0 max 16384 2024-06-06T07:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:21:53.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 msgr2=0x7fd9000a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 0x7fd9000a53c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd910009ca0 tx=0x7fd910004ab0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 msgr2=0x7fd91819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd90c00e9f0 tx=0x7fd90c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 shutdown_connections 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9181068c0 0x7fd91819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9181012c0 0x7fd91819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9000a2f00 0x7fd9000a53c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 --2- 172.21.15.73:0/3262979199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd918100900 0x7fd91819a2a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:21:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.731+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 >> 172.21.15.73:0/3262979199 conn(0x7fd9180779f0 msgr2=0x7fd91810c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:21:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.732+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 shutdown_connections 2024-06-06T07:21:53.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:21:53.732+0000 7fd91cbce640 1 -- 172.21.15.73:0/3262979199 wait complete. 2024-06-06T07:21:53.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70196 9.01 MB/sec execute 298 sec latency 100.677 ms 2024-06-06T07:21:54.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70350 9.00 MB/sec execute 299 sec latency 77.027 ms 2024-06-06T07:21:55.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:55 smithi170 ceph-mon[36108]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 418 op/s 2024-06-06T07:21:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:55 smithi073 ceph-mon[32303]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 418 op/s 2024-06-06T07:21:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:55 smithi136 ceph-mon[35901]: pgmap v654: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 418 op/s 2024-06-06T07:21:55.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70510 9.00 MB/sec execute 300 sec latency 89.228 ms 2024-06-06T07:21:56.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70673 9.00 MB/sec execute 301 sec latency 193.311 ms 2024-06-06T07:21:57.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:57 smithi170 ceph-mon[36108]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 252 op/s 2024-06-06T07:21:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:57 smithi073 ceph-mon[32303]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 252 op/s 2024-06-06T07:21:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:57 smithi136 ceph-mon[35901]: pgmap v655: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.8 MiB/s wr, 252 op/s 2024-06-06T07:21:57.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70824 9.00 MB/sec execute 302 sec latency 73.084 ms 2024-06-06T07:21:58.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 70967 9.00 MB/sec execute 303 sec latency 108.756 ms 2024-06-06T07:21:59.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:21:59 smithi170 ceph-mon[36108]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 393 op/s 2024-06-06T07:21:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:21:59 smithi073 ceph-mon[32303]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 393 op/s 2024-06-06T07:21:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:21:59 smithi136 ceph-mon[35901]: pgmap v656: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.3 MiB/s wr, 393 op/s 2024-06-06T07:21:59.814 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71125 8.99 MB/sec execute 304 sec latency 121.268 ms 2024-06-06T07:22:00.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71267 8.98 MB/sec execute 305 sec latency 146.609 ms 2024-06-06T07:22:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:01 smithi170 ceph-mon[36108]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-06-06T07:22:01.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:01 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:22:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:01 smithi073 ceph-mon[32303]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-06-06T07:22:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:01 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:22:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:01 smithi136 ceph-mon[35901]: pgmap v657: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.5 MiB/s wr, 275 op/s 2024-06-06T07:22:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:01 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:22:01.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71388 8.97 MB/sec execute 306 sec latency 201.325 ms 2024-06-06T07:22:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:02 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:02 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:02 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:02 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71558 8.97 MB/sec execute 307 sec latency 76.657 ms 2024-06-06T07:22:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:02 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:02 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:02 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:02 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:03.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:02 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:03.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:02 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:03.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:02 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:03.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:02 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:03 smithi136 ceph-mon[35901]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 266 op/s 2024-06-06T07:22:03.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71711 8.97 MB/sec execute 308 sec latency 74.357 ms 2024-06-06T07:22:03.830 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:03 smithi073 ceph-mon[32303]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 266 op/s 2024-06-06T07:22:03.846 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:03 smithi170 ceph-mon[36108]: pgmap v658: 129 pgs: 129 active+clean; 1.1 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.4 MiB/s wr, 266 op/s 2024-06-06T07:22:04.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71810 8.95 MB/sec execute 309 sec latency 220.210 ms 2024-06-06T07:22:05.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.7 MiB/s wr, 411 op/s 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:04 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.7 MiB/s wr, 411 op/s 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:04 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: pgmap v659: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.7 MiB/s wr, 411 op/s 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: osdmap e137: 12 total, 12 up, 12 in 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:04 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:05.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 71948 8.95 MB/sec execute 310 sec latency 238.145 ms 2024-06-06T07:22:06.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:06 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:06 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:06 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:06.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72108 8.95 MB/sec execute 311 sec latency 171.320 ms 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 -- 172.21.15.73:0/3742221896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4103e10 msgr2=0x7f58b41041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 --2- 172.21.15.73:0/3742221896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4103e10 0x7f58b41041f0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f589c009860 tx=0x7f589c02a340 comp rx=0 tx=0).stop 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 -- 172.21.15.73:0/3742221896 shutdown_connections 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 --2- 172.21.15.73:0/3742221896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4108ee0 0x7f58b4110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 --2- 172.21.15.73:0/3742221896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b41047c0 0x7f58b4108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 --2- 172.21.15.73:0/3742221896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4103e10 0x7f58b41041f0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 -- 172.21.15.73:0/3742221896 >> 172.21.15.73:0/3742221896 conn(0x7f58b40ff6b0 msgr2=0x7f58b4101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 -- 172.21.15.73:0/3742221896 shutdown_connections 2024-06-06T07:22:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.859+0000 7f58b94cd640 1 -- 172.21.15.73:0/3742221896 wait complete. 2024-06-06T07:22:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.860+0000 7f58b94cd640 1 Processor -- start 2024-06-06T07:22:06.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 -- start start 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4108ee0 0x7f58b419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b419ebd0 0x7f58b419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113e40 con 0x7f58b4103e10 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113cc0 con 0x7f58b419ebd0 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.861+0000 7f58b94cd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58b4113fc0 con 0x7f58b4108ee0 2024-06-06T07:22:06.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4108ee0 0x7f58b419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38434/0 (socket says 172.21.15.73:38434) 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 -- 172.21.15.73:0/875197409 learned_addr learned my addr 172.21.15.73:0/875197409 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b37fe640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b419ebd0 0x7f58b419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 -- 172.21.15.73:0/875197409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4108ee0 msgr2=0x7f58b419a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4108ee0 0x7f58b419a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 -- 172.21.15.73:0/875197409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b419ebd0 msgr2=0x7f58b419f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b419ebd0 0x7f58b419f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.864 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.862+0000 7f58b2ffd640 1 -- 172.21.15.73:0/875197409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58b419f6c0 con 0x7f58b4103e10 2024-06-06T07:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f589c035040 tx=0x7f589c030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b0ff9640 1 -- 172.21.15.73:0/875197409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c040020 con 0x7f58b4103e10 2024-06-06T07:22:06.865 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b0ff9640 1 -- 172.21.15.73:0/875197409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c038040 con 0x7f58b4103e10 2024-06-06T07:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b0ff9640 1 -- 172.21.15.73:0/875197409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c037070 con 0x7f58b4103e10 2024-06-06T07:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58b419f950 con 0x7f58b4103e10 2024-06-06T07:22:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.863+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58b41a70c0 con 0x7f58b4103e10 2024-06-06T07:22:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.866+0000 7f58b0ff9640 1 -- 172.21.15.73:0/875197409 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f589c043020 con 0x7f58b4103e10 2024-06-06T07:22:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58bc1db740 1 Processor -- start 2024-06-06T07:22:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58b0ff9640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 0x7f58a00a53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58bc1db740 1 -- start start 2024-06-06T07:22:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58b0ff9640 1 -- 172.21.15.73:0/875197409 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f589c0ef100 con 0x7f58b4103e10 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58b37fe640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 0x7f58a00a53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.867+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec1a2a0 0x55587ec24620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58bc1db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55587eb754f0 con 0x55587ec24b60 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58bc1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55587eb13870 con 0x55587ec1a2a0 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58bc1db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55587eb714b0 con 0x55587ec19980 2024-06-06T07:22:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:54336/0 (socket says 172.21.15.73:54336) 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 -- 172.21.15.73:0/919271068 learned_addr learned my addr 172.21.15.73:0/919271068 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec1a2a0 0x55587ec24620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b37fe640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 -- 172.21.15.73:0/919271068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec1a2a0 msgr2=0x55587ec24620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec1a2a0 0x55587ec24620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 -- 172.21.15.73:0/919271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 msgr2=0x55587ec26f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec26f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 -- 172.21.15.73:0/919271068 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587eb2f9b0 con 0x55587ec19980 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b37fe640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec26f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:22:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f589c0b7730 tx=0x7f589c02a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc29f01c7a015fa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58b37fe640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 0x7f58a00a53c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005fd0 tx=0x7f58ac00f7d0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.868+0000 7f58a97fa640 1 -- 172.21.15.73:0/919271068 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0d3020 con 0x55587ec19980 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.869+0000 7f58a97fa640 1 -- 172.21.15.73:0/919271068 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c0bb9d0 con 0x55587ec19980 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.869+0000 7f58a97fa640 1 -- 172.21.15.73:0/919271068 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0b7940 con 0x55587ec19980 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.869+0000 7f58bc1db740 1 -- 172.21.15.73:0/919271068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 msgr2=0x55587ec19d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.869+0000 7f58bc1db740 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f589c0b7730 tx=0x7f589c02a340 comp rx=0 tx=0).stop 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.869+0000 7f58bc1db740 1 -- 172.21.15.73:0/919271068 shutdown_connections 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec26f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec1a2a0 0x55587ec24620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- 172.21.15.73:0/919271068 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec19980 0x55587ec19d60 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- 172.21.15.73:0/919271068 >> 172.21.15.73:0/919271068 conn(0x55587ec15e90 msgr2=0x55587ec16270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- 172.21.15.73:0/919271068 shutdown_connections 2024-06-06T07:22:06.874 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- 172.21.15.73:0/919271068 wait complete. 2024-06-06T07:22:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- start start 2024-06-06T07:22:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 Processor -- start 2024-06-06T07:22:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec21bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec22650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55587ec27550 con 0x55587ec1a2a0 2024-06-06T07:22:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55587eafeae0 con 0x55587ec19980 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55587ec276d0 con 0x55587ec24b60 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58bc1db740 10 client.0 _ll_register_callbacks cb 0x55587eb379b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38448/0 (socket says 172.21.15.73:38448) 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec21bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58aa7fc640 1 -- 172.21.15.73:0/2071589596 learned_addr learned my addr 172.21.15.73:0/2071589596 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.870+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec22650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 -- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 msgr2=0x55587ec22650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec22650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 -- 172.21.15.73:0/2071589596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 msgr2=0x55587ec21bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec21bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 -- 172.21.15.73:0/2071589596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587e889210 con 0x55587ec1a2a0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec21bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f58a40126e0 tx=0x7f58a4012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268535e7cd17b63a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f589a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4012190 con 0x55587ec1a2a0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f58a400f920 con 0x55587ec1a2a0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f589c0bb9d0 con 0x55587ec1a2a0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f58a400fb20 con 0x55587ec1a2a0 2024-06-06T07:22:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58bc1db740 10 client.17535 fetch_fsmap learned FSMap version 77 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.871+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55587e955620 con 0x55587ec1a2a0 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c0bb9d0 con 0x55587ec1a2a0 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f58a4010cb0 con 0x55587ec1a2a0 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 10 client.17535 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 10 client.17535 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 20 client.17535 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587ec36fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 4 client.17535 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55587eb88f10 con 0x55587ec34b90 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.872+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587ec36fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.873+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587ec36fe0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eebd3c99d8ba198a server_cookie=53fa3dbd53ef2e04 in_seq=0 out_seq=0 2024-06-06T07:22:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.873+0000 7f589a7fc640 10 client.17535 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f589a7fc640 1 -- 172.21.15.73:0/2071589596 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55587eb88f10 con 0x55587ec34b90 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f589a7fc640 10 client.17535 handle_command_reply: tid=0 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 client.17535 shutdown 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 msgr2=0x55587ec22110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f58a40126e0 tx=0x7f58a4012ba0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 shutdown_connections 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587ec36fe0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f589a7fc640 0 client.17535 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec22110 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 --2- 172.21.15.73:0/2071589596 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec21bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.874+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 >> 172.21.15.73:0/2071589596 conn(0x55587ec15e90 msgr2=0x55587ec1bc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.884 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.875+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 shutdown_connections 2024-06-06T07:22:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.875+0000 7f58bc1db740 1 -- 172.21.15.73:0/2071589596 wait complete. 2024-06-06T07:22:06.885 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.875+0000 7f58bc1db740 20 client.17535 trim_cache size 0 max 16384 2024-06-06T07:22:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.895+0000 7f58bc1db740 1 Processor -- start 2024-06-06T07:22:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.895+0000 7f58bc1db740 1 -- start start 2024-06-06T07:22:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec1c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec3c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41ace30 con 0x55587ec24b60 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41ae970 con 0x55587ec19980 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58bc1db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41ad500 con 0x55587ec1a2a0 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58b37fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec3c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec1c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:54354/0 (socket says 172.21.15.73:54354) 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec1c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:33998/0 (socket says 172.21.15.73:33998) 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 -- 172.21.15.73:0/3252170741 learned_addr learned my addr 172.21.15.73:0/3252170741 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58b37fe640 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec3c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38454/0 (socket says 172.21.15.73:38454) 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 -- 172.21.15.73:0/3252170741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 msgr2=0x55587ec1c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec1c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 -- 172.21.15.73:0/3252170741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 msgr2=0x55587ec3c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec3c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 -- 172.21.15.73:0/3252170741 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587ec1a0d0 con 0x55587ec1a2a0 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.896+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58a401c570 tx=0x7f58a400f810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1dc4e7e45dc81c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.897+0000 7f58a8ff9640 1 -- 172.21.15.73:0/3252170741 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a4012190 con 0x55587ec1a2a0 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.897+0000 7f58a8ff9640 1 -- 172.21.15.73:0/3252170741 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f58a4005590 con 0x55587ec1a2a0 2024-06-06T07:22:06.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.897+0000 7f58a8ff9640 1 -- 172.21.15.73:0/3252170741 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58a401cac0 con 0x55587ec1a2a0 2024-06-06T07:22:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.897+0000 7f58bc1db740 1 -- 172.21.15.73:0/3252170741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 msgr2=0x55587ec23b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.902 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.897+0000 7f58bc1db740 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f58a401c570 tx=0x7f58a400f810 comp rx=0 tx=0).stop 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 -- 172.21.15.73:0/3252170741 shutdown_connections 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec24b60 0x55587ec3c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec1a2a0 0x55587ec23b10 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 --2- 172.21.15.73:0/3252170741 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec1c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 -- 172.21.15.73:0/3252170741 >> 172.21.15.73:0/3252170741 conn(0x55587ec15e90 msgr2=0x55587ec23400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 -- 172.21.15.73:0/3252170741 shutdown_connections 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 -- 172.21.15.73:0/3252170741 wait complete. 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 -- start start 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 Processor -- start 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.898+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec2ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.903 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec28730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41ae2a0 con 0x55587ec1a2a0 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41adbd0 con 0x55587ec19980 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f58b41aae30 con 0x55587ec24b60 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58bc1db740 10 client.0 _ll_register_callbacks cb 0x7f58a4009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec2ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58aa7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38468/0 (socket says 172.21.15.73:38468) 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58aa7fc640 1 -- 172.21.15.73:0/1115190610 learned_addr learned my addr 172.21.15.73:0/1115190610 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b37fe640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec28730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 -- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 msgr2=0x55587ec28730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec28730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 -- 172.21.15.73:0/1115190610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 msgr2=0x55587ec281f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:06.904 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 -- 172.21.15.73:0/1115190610 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587ec28dd0 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f58b2ffd640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec2ce80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f589c0b7730 tx=0x7f589c0c3f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b51e4906da2277ff server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.899+0000 7f589a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589c0d3040 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(137..137 src has 1..137) ==== 9642+0+0 (secure 0 0 0) 0x7f589c0b7940 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58a4005590 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f589c0bb9d0 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 10 client.26762 fetch_fsmap learned FSMap version 77 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=4+}) -- 0x55587ec9f780 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f589c0bb2d0 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 77) ==== 8366+0+0 (secure 0 0 0) 0x7f589c0db1b0 con 0x55587ec19980 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 10 client.26762 fetch_fsmap finished waiting for FSMap version 77 2024-06-06T07:22:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 10 client.26762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 20 client.26762 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587eca1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 4 client.26762 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.900+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f58a4005590 con 0x55587ec34b90 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.901+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587eca1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.901+0000 7f589a7fc640 10 client.26762 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.901+0000 7f58aa7fc640 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587eca1c70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47d772bc0437195 server_cookie=83f3b0ca57eacb48 in_seq=0 out_seq=0 2024-06-06T07:22:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f589a7fc640 1 -- 172.21.15.73:0/1115190610 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f58a4005590 con 0x55587ec34b90 2024-06-06T07:22:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f589a7fc640 10 client.26762 handle_command_reply: tid=0 2024-06-06T07:22:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 client.26762 shutdown 2024-06-06T07:22:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 msgr2=0x55587ec2ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec2ce80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f589c0b7730 tx=0x7f589c0c3f40 comp rx=0 tx=0).stop 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 shutdown_connections 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55587ec34b90 0x55587eca1c70 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55587ec24b60 0x55587ec28730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55587ec1a2a0 0x55587ec281f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 --2- 172.21.15.73:0/1115190610 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55587ec19980 0x55587ec2ce80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 >> 172.21.15.73:0/1115190610 conn(0x55587ec15e90 msgr2=0x55587ec0c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.968+0000 7f589a7fc640 0 client.26762 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.969+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 shutdown_connections 2024-06-06T07:22:06.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.969+0000 7f58bc1db740 1 -- 172.21.15.73:0/1115190610 wait complete. 2024-06-06T07:22:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.969+0000 7f58bc1db740 20 client.26762 trim_cache size 0 max 16384 2024-06-06T07:22:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:22:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:22:06.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 msgr2=0x7f58a00a53c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 0x7f58a00a53c0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005fd0 tx=0x7f58ac00f7d0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 msgr2=0x7f58b419a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f589c035040 tx=0x7f589c030e00 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 shutdown_connections 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f58b419ebd0 0x7f58b419f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f58a00a2f00 0x7f58a00a53c0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f58b4108ee0 0x7f58b419a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 --2- 172.21.15.73:0/875197409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58b4103e10 0x7f58b419a360 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.970+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 >> 172.21.15.73:0/875197409 conn(0x7f58b40ff6b0 msgr2=0x7f58b4101a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:06.972 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.971+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 shutdown_connections 2024-06-06T07:22:06.973 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:06.971+0000 7f58b94cd640 1 -- 172.21.15.73:0/875197409 wait complete. 2024-06-06T07:22:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:07 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:22:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:07 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:22:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:07 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:22:07.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:07 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:22:07.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:07 smithi073 ceph-mon[32303]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-06-06T07:22:07.297 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:07 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:22:07.297 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:07 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:22:07.297 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:07 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:22:07.297 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:07 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:22:07.297 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:07 smithi136 ceph-mon[35901]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-06-06T07:22:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:07 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi170 to 176.1M 2024-06-06T07:22:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:07 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi170 to 184736563: error parsing value: Value '184736563' is below minimum 939524096 2024-06-06T07:22:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:07 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi073 to 176.1M 2024-06-06T07:22:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:07 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi073 to 184711475: error parsing value: Value '184711475' is below minimum 939524096 2024-06-06T07:22:07.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:07 smithi170 ceph-mon[36108]: pgmap v661: 129 pgs: 129 active+clean; 1.1 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.8 MiB/s wr, 347 op/s 2024-06-06T07:22:07.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72277 8.93 MB/sec execute 312 sec latency 90.018 ms 2024-06-06T07:22:08.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72432 8.94 MB/sec execute 313 sec latency 95.237 ms 2024-06-06T07:22:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:22:09.275 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:08 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:09.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:22:09.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:08 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: pgmap v662: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-06T07:22:09.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:22:09.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:22:09.348 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:08 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:22:09.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72585 8.93 MB/sec execute 314 sec latency 85.594 ms 2024-06-06T07:22:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:09 smithi073 ceph-mon[32303]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:22:10.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:09 smithi073 ceph-mon[32303]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:22:10.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:09 smithi136 ceph-mon[35901]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:22:10.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:09 smithi136 ceph-mon[35901]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:22:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:09 smithi170 ceph-mon[36108]: Adjusting osd_memory_target on smithi136 to 176.1M 2024-06-06T07:22:10.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:09 smithi170 ceph-mon[36108]: Unable to set osd_memory_target on smithi136 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-06T07:22:10.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72766 8.93 MB/sec execute 315 sec latency 58.963 ms 2024-06-06T07:22:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:10 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:11.274 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:10 smithi073 ceph-mon[32303]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:10 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:11.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:10 smithi136 ceph-mon[35901]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:10 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:11.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:10 smithi170 ceph-mon[36108]: pgmap v663: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:11.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 72950 8.95 MB/sec execute 316 sec latency 84.903 ms 2024-06-06T07:22:12.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73126 8.96 MB/sec execute 317 sec latency 87.731 ms 2024-06-06T07:22:13.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:13 smithi170 ceph-mon[36108]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:13.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:13 smithi170 ceph-mon[36108]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T07:22:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:13 smithi073 ceph-mon[32303]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:13 smithi073 ceph-mon[32303]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T07:22:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:13 smithi136 ceph-mon[35901]: pgmap v664: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 336 op/s 2024-06-06T07:22:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:13 smithi136 ceph-mon[35901]: osdmap e138: 12 total, 12 up, 12 in 2024-06-06T07:22:13.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73274 8.94 MB/sec execute 318 sec latency 89.817 ms 2024-06-06T07:22:14.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73422 8.94 MB/sec execute 319 sec latency 79.963 ms 2024-06-06T07:22:15.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:15 smithi170 ceph-mon[36108]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-06-06T07:22:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:15 smithi073 ceph-mon[32303]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-06-06T07:22:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:15 smithi136 ceph-mon[35901]: pgmap v666: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 341 op/s 2024-06-06T07:22:15.815 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73584 8.94 MB/sec execute 320 sec latency 107.848 ms 2024-06-06T07:22:16.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73734 8.94 MB/sec execute 321 sec latency 117.900 ms 2024-06-06T07:22:17.356 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:17 smithi073 ceph-mon[32303]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 324 op/s 2024-06-06T07:22:17.356 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:17 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:17 smithi136 ceph-mon[35901]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 324 op/s 2024-06-06T07:22:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:17 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:17.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:17 smithi170 ceph-mon[36108]: pgmap v667: 129 pgs: 129 active+clean; 1.1 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.6 MiB/s wr, 324 op/s 2024-06-06T07:22:17.599 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:17 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:17.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 73883 8.94 MB/sec execute 322 sec latency 108.431 ms 2024-06-06T07:22:18.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74024 8.94 MB/sec execute 323 sec latency 91.937 ms 2024-06-06T07:22:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:19 smithi073 ceph-mon[32303]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-06-06T07:22:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:19 smithi136 ceph-mon[35901]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-06-06T07:22:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:19 smithi170 ceph-mon[36108]: pgmap v668: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.7 MiB/s wr, 286 op/s 2024-06-06T07:22:19.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74188 8.94 MB/sec execute 324 sec latency 85.930 ms 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 -- 172.21.15.73:0/306818392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c4102370 msgr2=0x7f42c410e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 --2- 172.21.15.73:0/306818392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c4102370 0x7f42c410e880 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009cd0 tx=0x7f42bc02a910 comp rx=0 tx=0).stop 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 -- 172.21.15.73:0/306818392 shutdown_connections 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 --2- 172.21.15.73:0/306818392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c4102370 0x7f42c410e880 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 --2- 172.21.15.73:0/306818392 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c41019b0 0x7f42c4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 --2- 172.21.15.73:0/306818392 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41079c0 0x7f42c4107da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 -- 172.21.15.73:0/306818392 >> 172.21.15.73:0/306818392 conn(0x7f42c4078e30 msgr2=0x7f42c4079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.107 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 -- 172.21.15.73:0/306818392 shutdown_connections 2024-06-06T07:22:20.108 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.106+0000 7f42cb04d640 1 -- 172.21.15.73:0/306818392 wait complete. 2024-06-06T07:22:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 Processor -- start 2024-06-06T07:22:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 -- start start 2024-06-06T07:22:20.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c4102370 0x7f42c4073ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c41079c0 0x7f42c40742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4111ca0 con 0x7f42c41079c0 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4111b20 con 0x7f42c41019b0 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42cb04d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4111e20 con 0x7f42c4102370 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.107+0000 7f42c8de9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57874/0 (socket says 172.21.15.73:57874) 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 -- 172.21.15.73:0/2500734830 learned_addr learned my addr 172.21.15.73:0/2500734830 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c3fff640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c4102370 0x7f42c4073ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c95ea640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c41079c0 0x7f42c40742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 -- 172.21.15.73:0/2500734830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c4102370 msgr2=0x7f42c4073ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c4102370 0x7f42c4073ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 -- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c41079c0 msgr2=0x7f42c40742e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c41079c0 0x7f42c40742e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 -- 172.21.15.73:0/2500734830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c41aaae0 con 0x7f42c41019b0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.108+0000 7f42c8de9640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d950 tx=0x7f42b800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.109+0000 7f42c1ffb640 1 -- 172.21.15.73:0/2500734830 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8017070 con 0x7f42c41019b0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.109+0000 7f42c1ffb640 1 -- 172.21.15.73:0/2500734830 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42b8009c30 con 0x7f42c41019b0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.109+0000 7f42c1ffb640 1 -- 172.21.15.73:0/2500734830 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b800f620 con 0x7f42c41019b0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.109+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c41aad10 con 0x7f42c41019b0 2024-06-06T07:22:20.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.109+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c41ab1b0 con 0x7f42c41019b0 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 Processor -- start 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 -- start start 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42c1ffb640 1 -- 172.21.15.73:0/2500734830 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f42b80109a0 con 0x7f42c41019b0 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f49ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f3f64f0 con 0x55cf7f49d350 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f394870 con 0x55cf7f492a90 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f3f24b0 con 0x55cf7f492170 2024-06-06T07:22:20.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42c95ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42c95ea640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36674/0 (socket says 172.21.15.73:36674) 2024-06-06T07:22:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42c95ea640 1 -- 172.21.15.73:0/3284243262 learned_addr learned my addr 172.21.15.73:0/3284243262 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:20.114 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.112+0000 7f42c3fff640 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 -- 172.21.15.73:0/3284243262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 msgr2=0x55cf7f49ce10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f49ce10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.115 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 -- 172.21.15.73:0/3284243262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 msgr2=0x55cf7f49f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 -- 172.21.15.73:0/3284243262 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7f3b09b0 con 0x55cf7f492170 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c3fff640 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c1ffb640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 0x7f42b40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42bc035000 tx=0x7f42bc009830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f1cf7e5714e09 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42c1ffb640 1 -- 172.21.15.73:0/2500734830 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7f42b80995e0 con 0x7f42c41019b0 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.113+0000 7f42aeffd640 1 -- 172.21.15.73:0/3284243262 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc031d10 con 0x55cf7f492170 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42aeffd640 1 -- 172.21.15.73:0/3284243262 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42bc036030 con 0x55cf7f492170 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42aeffd640 1 -- 172.21.15.73:0/3284243262 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0042a0 con 0x55cf7f492170 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42c8de9640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 0x7f42b40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3284243262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 msgr2=0x55cf7f492550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42c8de9640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 0x7f42b40a52e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55cf7f493710 tx=0x7f42b8010f70 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.116 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.114+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42bc035000 tx=0x7f42bc009830 comp rx=0 tx=0).stop 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3284243262 shutdown_connections 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f49ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3284243262 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492170 0x55cf7f492550 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3284243262 >> 172.21.15.73:0/3284243262 conn(0x55cf7f48e680 msgr2=0x55cf7f48ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3284243262 shutdown_connections 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3284243262 wait complete. 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- start start 2024-06-06T07:22:20.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 Processor -- start 2024-06-06T07:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f49a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f49ff20 con 0x55cf7f49d350 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f37fae0 con 0x55cf7f492170 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55cf7f4a00a0 con 0x55cf7f492a90 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42cd9fd740 10 client.0 _ll_register_callbacks cb 0x55cf7f3b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.115+0000 7f42c95ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36680/0 (socket says 172.21.15.73:36680) 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 -- 172.21.15.73:0/1897432719 learned_addr learned my addr 172.21.15.73:0/1897432719 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f49a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 -- 172.21.15.73:0/1897432719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 msgr2=0x55cf7f49a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f49a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 -- 172.21.15.73:0/1897432719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 msgr2=0x55cf7f49adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 -- 172.21.15.73:0/1897432719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7f10a210 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42c3fff640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009e30 tx=0x7f42b0011630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c462e286de83a84f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b0019070 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7f42b0015660 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.116+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42bc036030 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f42b0015860 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 10 client.26953 fetch_fsmap learned FSMap version 79 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55cf7f1d6620 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42bc036030 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 79) ==== 8374+0+0 (secure 0 0 0) 0x7f42b0014ab0 con 0x55cf7f492a90 2024-06-06T07:22:20.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 10 client.26953 fetch_fsmap finished waiting for FSMap version 79 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 10 client.26953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 20 client.26953 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f4afa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 4 client.26953 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf7f409f10 con 0x55cf7f4ad610 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.117+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f4afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.118+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f4afa00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d08caba8449c17f server_cookie=aaf58d5d1254306f in_seq=0 out_seq=0 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.118+0000 7f42a3fff640 10 client.26953 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42a3fff640 1 -- 172.21.15.73:0/1897432719 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cf7f409f10 con 0x55cf7f4ad610 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42a3fff640 10 client.26953 handle_command_reply: tid=0 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 client.26953 shutdown 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 msgr2=0x55cf7f49a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42b0009e30 tx=0x7f42b0011630 comp rx=0 tx=0).stop 2024-06-06T07:22:20.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 shutdown_connections 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f4afa00 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f49adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42a3fff640 0 client.26953 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f49a870 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1897432719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f49a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 >> 172.21.15.73:0/1897432719 conn(0x55cf7f48e680 msgr2=0x55cf7f4943b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 shutdown_connections 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1897432719 wait complete. 2024-06-06T07:22:20.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.119+0000 7f42cd9fd740 20 client.26953 trim_cache size 0 max 16384 2024-06-06T07:22:20.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.141+0000 7f42cd9fd740 1 Processor -- start 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 -- start start 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f48d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f49d350 0x55cf7f4b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b5420 con 0x55cf7f492170 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b6f60 con 0x55cf7f492a90 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b5af0 con 0x55cf7f49d350 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44076/0 (socket says 172.21.15.73:44076) 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 -- 172.21.15.73:0/1490687993 learned_addr learned my addr 172.21.15.73:0/1490687993 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:20.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c3fff640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f48d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f49d350 0x55cf7f4b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 -- 172.21.15.73:0/1490687993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f49d350 msgr2=0x55cf7f4b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f49d350 0x55cf7f4b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 -- 172.21.15.73:0/1490687993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 msgr2=0x55cf7f48d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f48d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c95ea640 1 -- 172.21.15.73:0/1490687993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf7f4afd70 con 0x55cf7f492170 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.142+0000 7f42c3fff640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f48d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.143+0000 7f42c95ea640 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f42bc003db0 tx=0x7f42bc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20efcefa3c3c6759 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.143+0000 7f42ae7fc640 1 -- 172.21.15.73:0/1490687993 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc04a040 con 0x55cf7f492170 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.143+0000 7f42ae7fc640 1 -- 172.21.15.73:0/1490687993 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42bc036030 con 0x55cf7f492170 2024-06-06T07:22:20.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.143+0000 7f42ae7fc640 1 -- 172.21.15.73:0/1490687993 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0305d0 con 0x55cf7f492170 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1490687993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 msgr2=0x55cf7f49abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f42bc003db0 tx=0x7f42bc005ef0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1490687993 shutdown_connections 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f49d350 0x55cf7f4b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492a90 0x55cf7f48d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/1490687993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f492170 0x55cf7f49abd0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.147 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1490687993 >> 172.21.15.73:0/1490687993 conn(0x55cf7f48e680 msgr2=0x55cf7f48cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1490687993 shutdown_connections 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- 172.21.15.73:0/1490687993 wait complete. 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.144+0000 7f42cd9fd740 1 -- start start 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 Processor -- start 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f4a53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f4a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f4a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b6890 con 0x55cf7f49d350 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b61c0 con 0x55cf7f492170 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41b3420 con 0x55cf7f492a90 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.145+0000 7f42cd9fd740 10 client.0 _ll_register_callbacks cb 0x7f42b0009370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f4a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c3fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f4a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36700/0 (socket says 172.21.15.73:36700) 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c3fff640 1 -- 172.21.15.73:0/3719058992 learned_addr learned my addr 172.21.15.73:0/3719058992 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f4a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f4a53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 -- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 msgr2=0x55cf7f4a58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f4a58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 -- 172.21.15.73:0/3719058992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 msgr2=0x55cf7f4a53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f4a53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 -- 172.21.15.73:0/3719058992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf7f4a1020 con 0x55cf7f49d350 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42c95ea640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f4a08b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009bf0 tx=0x7f42bc003de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c663f3277e54dbe8 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:20.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.146+0000 7f42a3fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:22:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc0314e0 con 0x55cf7f49d350 2024-06-06T07:22:20.150 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7f42bc04a040 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42bc036030 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) ==== 24+0+0 (secure 0 0 0) 0x7f42bc031680 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42cd9fd740 10 client.17559 fetch_fsmap learned FSMap version 79 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=4+}) -- 0x55cf7f517d30 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f42bc031820 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.147+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 79) ==== 8374+0+0 (secure 0 0 0) 0x7f42bc050050 con 0x55cf7f49d350 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 10 client.17559 fetch_fsmap finished waiting for FSMap version 79 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 10 client.17559 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 20 client.17559 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f51a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 4 client.17559 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f42bc036030 con 0x55cf7f4ad610 2024-06-06T07:22:20.151 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42c3fff640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f51a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42a3fff640 10 client.17559 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:20.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.148+0000 7f42c3fff640 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f51a010 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0d5f4b9da17d5d server_cookie=d6a67f27bf35a63b in_seq=0 out_seq=0 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.279+0000 7f42a3fff640 1 -- 172.21.15.73:0/3719058992 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f42bc036030 con 0x55cf7f4ad610 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.279+0000 7f42a3fff640 10 client.17559 handle_command_reply: tid=0 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 client.17559 shutdown 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 msgr2=0x55cf7f4a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f4a08b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f42bc009bf0 tx=0x7f42bc003de0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 shutdown_connections 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55cf7f4ad610 0x55cf7f51a010 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cf7f49d350 0x55cf7f4a08b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55cf7f492a90 0x55cf7f4a58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 --2- 172.21.15.73:0/3719058992 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55cf7f492170 0x55cf7f4a53a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 >> 172.21.15.73:0/3719058992 conn(0x55cf7f48e680 msgr2=0x55cf7f48d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.280+0000 7f42a3fff640 0 client.17559 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.281+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 shutdown_connections 2024-06-06T07:22:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.281+0000 7f42cd9fd740 1 -- 172.21.15.73:0/3719058992 wait complete. 2024-06-06T07:22:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.281+0000 7f42cd9fd740 20 client.17559 trim_cache size 0 max 16384 2024-06-06T07:22:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:22:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:22:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.282+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 msgr2=0x7f42b40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.282+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 0x7f42b40a52e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x55cf7f493710 tx=0x7f42b8010f70 comp rx=0 tx=0).stop 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.282+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 msgr2=0x7f42c40758a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.282+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f42b800d950 tx=0x7f42b800de10 comp rx=0 tx=0).stop 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 shutdown_connections 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42c41079c0 0x7f42c40742e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f42b40a2e20 0x7f42b40a52e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f42c4102370 0x7f42c4073ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 --2- 172.21.15.73:0/2500734830 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f42c41019b0 0x7f42c40758a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 >> 172.21.15.73:0/2500734830 conn(0x7f42c4078e30 msgr2=0x7f42c40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 shutdown_connections 2024-06-06T07:22:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:20.283+0000 7f42cb04d640 1 -- 172.21.15.73:0/2500734830 wait complete. 2024-06-06T07:22:20.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74353 8.94 MB/sec execute 325 sec latency 96.796 ms 2024-06-06T07:22:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:21 smithi073 ceph-mon[32303]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:21 smithi136 ceph-mon[35901]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:21 smithi170 ceph-mon[36108]: pgmap v669: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:21.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74490 8.93 MB/sec execute 326 sec latency 97.517 ms 2024-06-06T07:22:22.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74648 8.92 MB/sec execute 327 sec latency 78.274 ms 2024-06-06T07:22:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:23 smithi073 ceph-mon[32303]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:23 smithi136 ceph-mon[35901]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:23 smithi170 ceph-mon[36108]: pgmap v670: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.1 MiB/s wr, 309 op/s 2024-06-06T07:22:23.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74817 8.94 MB/sec execute 328 sec latency 99.022 ms 2024-06-06T07:22:24.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 74950 8.93 MB/sec execute 329 sec latency 117.360 ms 2024-06-06T07:22:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:25 smithi073 ceph-mon[32303]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 438 op/s 2024-06-06T07:22:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:25 smithi136 ceph-mon[35901]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 438 op/s 2024-06-06T07:22:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:25 smithi170 ceph-mon[36108]: pgmap v671: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.8 MiB/s wr, 438 op/s 2024-06-06T07:22:25.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75122 8.93 MB/sec execute 330 sec latency 70.498 ms 2024-06-06T07:22:26.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75304 8.93 MB/sec execute 331 sec latency 90.659 ms 2024-06-06T07:22:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:27 smithi073 ceph-mon[32303]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 278 op/s 2024-06-06T07:22:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:27 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:27 smithi136 ceph-mon[35901]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 278 op/s 2024-06-06T07:22:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:27 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:27 smithi170 ceph-mon[36108]: pgmap v672: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.8 MiB/s wr, 278 op/s 2024-06-06T07:22:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:27 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:22:27.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75456 8.92 MB/sec execute 332 sec latency 83.232 ms 2024-06-06T07:22:28.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75608 8.91 MB/sec execute 333 sec latency 137.126 ms 2024-06-06T07:22:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:29 smithi073 ceph-mon[32303]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 390 op/s 2024-06-06T07:22:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:29 smithi136 ceph-mon[35901]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 390 op/s 2024-06-06T07:22:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:29 smithi170 ceph-mon[36108]: pgmap v673: 129 pgs: 129 active+clean; 1.1 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 390 op/s 2024-06-06T07:22:29.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75739 8.91 MB/sec execute 334 sec latency 311.861 ms 2024-06-06T07:22:30.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 75898 8.91 MB/sec execute 335 sec latency 72.476 ms 2024-06-06T07:22:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:31 smithi073 ceph-mon[32303]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-06-06T07:22:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:31 smithi136 ceph-mon[35901]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-06-06T07:22:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:31 smithi170 ceph-mon[36108]: pgmap v674: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.1 MiB/s wr, 310 op/s 2024-06-06T07:22:31.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76082 8.92 MB/sec execute 336 sec latency 111.513 ms 2024-06-06T07:22:32.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76234 8.92 MB/sec execute 337 sec latency 132.941 ms 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.413+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1010460920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101410 msgr2=0x7fb1c8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.413+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1010460920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101410 0x7fb1c8101890 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009930 tx=0x7fb1bc02a420 comp rx=0 tx=0).stop 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1010460920 shutdown_connections 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1010460920 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101dd0 0x7fb1c810d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1010460920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101410 0x7fb1c8101890 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1010460920 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1010460920 >> 172.21.15.73:0/1010460920 conn(0x7fb1c80769a0 msgr2=0x7fb1c80fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1010460920 shutdown_connections 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1010460920 wait complete. 2024-06-06T07:22:33.416 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.414+0000 7fb1ce6cd640 1 Processor -- start 2024-06-06T07:22:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 -- start start 2024-06-06T07:22:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101410 0x7fb1c819a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.418 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 0x7fb1c819a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c819ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111e50 con 0x7fb1c8101dd0 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111cd0 con 0x7fb1c8106b40 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ce6cd640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c8111fd0 con 0x7fb1c8101410 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 0x7fb1c819a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 0x7fb1c819a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39614/0 (socket says 172.21.15.73:39614) 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1000805704 learned_addr learned my addr 172.21.15.73:0/1000805704 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c819ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.415+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101410 0x7fb1c819a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 -- 172.21.15.73:0/1000805704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101410 msgr2=0x7fb1c819a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101410 0x7fb1c819a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.419 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 -- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 msgr2=0x7fb1c819a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 0x7fb1c819a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 -- 172.21.15.73:0/1000805704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c819f3c0 con 0x7fb1c8106b40 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c819ec90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fd0 tx=0x7fb1c000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1c57fa640 1 -- 172.21.15.73:0/1000805704 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c000ce20 con 0x7fb1c8106b40 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c819f6b0 con 0x7fb1c8106b40 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1c57fa640 1 -- 172.21.15.73:0/1000805704 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1c0005020 con 0x7fb1c8106b40 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1c57fa640 1 -- 172.21.15.73:0/1000805704 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1c0010690 con 0x7fb1c8106b40 2024-06-06T07:22:33.420 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.416+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c81a6f40 con 0x7fb1c8106b40 2024-06-06T07:22:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1c57fa640 1 -- 172.21.15.73:0/1000805704 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb1c0010830 con 0x7fb1c8106b40 2024-06-06T07:22:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 Processor -- start 2024-06-06T07:22:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 -- start start 2024-06-06T07:22:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502279550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.422 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 0x556502283e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5565021dd4f0 con 0x556502279a90 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55650217b870 con 0x556502279170 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.420+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5565021d94b0 con 0x556502284350 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.423 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:46408/0 (socket says 172.21.15.73:46408) 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1306362343 learned_addr learned my addr 172.21.15.73:0/1306362343 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502279550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 0x556502283e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c57fa640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 0x7fb1b00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1306362343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 msgr2=0x556502279550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502279550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1306362343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 msgr2=0x556502283e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 0x7fb1b00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 0x556502283e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c57fa640 1 -- 172.21.15.73:0/1000805704 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb1c00cff20 con 0x7fb1c8106b40 2024-06-06T07:22:33.424 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1306362343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565021979b0 con 0x556502284350 2024-06-06T07:22:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.421+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 0x556502283e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:33.425 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.422+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009a50 tx=0x7fb1bc009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=562d79caae7974ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.422+0000 7fb1ccc6a640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 0x7fb1b00a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1c009bd30 tx=0x7fb1c00b7040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.422+0000 7fb1ae7fc640 1 -- 172.21.15.73:0/1306362343 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc035070 con 0x556502284350 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.422+0000 7fb1ae7fc640 1 -- 172.21.15.73:0/1306362343 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1bc032bb0 con 0x556502284350 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.423+0000 7fb1ae7fc640 1 -- 172.21.15.73:0/1306362343 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc005020 con 0x556502284350 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1306362343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 msgr2=0x556502286740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009a50 tx=0x7fb1bc009260 comp rx=0 tx=0).stop 2024-06-06T07:22:33.426 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1306362343 shutdown_connections 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502284350 0x556502286740 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502279a90 0x556502283e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1306362343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502279550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.424+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1306362343 >> 172.21.15.73:0/1306362343 conn(0x556502275680 msgr2=0x556502275a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.425+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1306362343 shutdown_connections 2024-06-06T07:22:33.427 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.425+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1306362343 wait complete. 2024-06-06T07:22:33.428 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.425+0000 7fb1d13d9740 1 -- start start 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.426+0000 7fb1d13d9740 1 Processor -- start 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.426+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.426+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502281790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.426+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.426+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53006/0 (socket says 172.21.15.73:53006) 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 -- 172.21.15.73:0/661310934 learned_addr learned my addr 172.21.15.73:0/661310934 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502281cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502281790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556502286f10 con 0x556502284350 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x556502166ae0 con 0x556502279170 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x556502287090 con 0x556502279a90 2024-06-06T07:22:33.429 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1d13d9740 10 client.0 _ll_register_callbacks cb 0x55650219f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:33.430 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502281cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 -- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 msgr2=0x556502281790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502281790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.431 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 -- 172.21.15.73:0/661310934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 msgr2=0x556502281cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502281cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.427+0000 7fb1c77fe640 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556501ef1210 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc004290 tx=0x7fb1bc005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a748d04d1da6560a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1bc035050 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb1bc0314a0 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1bc032bb0 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb1bc0316a0 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1d13d9740 10 client.26789 fetch_fsmap learned FSMap version 80 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x556501fbd620 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.428+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1bc031840 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fb1bc04e030 con 0x556502279170 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 10 client.26789 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:22:33.432 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 10 client.26789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 20 client.26789 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502296690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 4 client.26789 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb1bc032bb0 con 0x5565022942a0 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502296690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1a77fe640 10 client.26789 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:33.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.429+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502296690 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d165b5419346c7b server_cookie=3befec6b99565c00 in_seq=0 out_seq=0 2024-06-06T07:22:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.431+0000 7fb1a77fe640 1 -- 172.21.15.73:0/661310934 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fb1bc032bb0 con 0x5565022942a0 2024-06-06T07:22:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.431+0000 7fb1a77fe640 10 client.26789 handle_command_reply: tid=0 2024-06-06T07:22:33.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.431+0000 7fb1d13d9740 1 client.26789 shutdown 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 msgr2=0x556502281250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc004290 tx=0x7fb1bc005a20 comp rx=0 tx=0).stop 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 shutdown_connections 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502296690 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502281cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1a77fe640 0 client.26789 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502281790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/661310934 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502281250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 >> 172.21.15.73:0/661310934 conn(0x556502275680 msgr2=0x55650227b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 shutdown_connections 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 1 -- 172.21.15.73:0/661310934 wait complete. 2024-06-06T07:22:33.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.432+0000 7fb1d13d9740 20 client.26789 trim_cache size 0 max 16384 2024-06-06T07:22:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.458+0000 7fb1d13d9740 1 Processor -- start 2024-06-06T07:22:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.458+0000 7fb1d13d9740 1 -- start start 2024-06-06T07:22:33.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x55650227e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x55650227e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53012/0 (socket says 172.21.15.73:53012) 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1241549941 learned_addr learned my addr 172.21.15.73:0/1241549941 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81accc0 con 0x556502284350 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ae800 con 0x556502279170 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ad390 con 0x556502279a90 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x55650227e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x55650227e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1241549941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 msgr2=0x55650227bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1241549941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 msgr2=0x55650227e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x55650227e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1241549941 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556502276410 con 0x556502279a90 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.459+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x55650227e4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556502287050 tx=0x7fb1b8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86ad08c879857925 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.460+0000 7fb1adffb640 1 -- 172.21.15.73:0/1241549941 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b80306c0 con 0x556502279a90 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.460+0000 7fb1adffb640 1 -- 172.21.15.73:0/1241549941 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b8030d10 con 0x556502279a90 2024-06-06T07:22:33.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.460+0000 7fb1adffb640 1 -- 172.21.15.73:0/1241549941 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b802fce0 con 0x556502279a90 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 msgr2=0x55650227e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x55650227e4b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x556502287050 tx=0x7fb1b8005f90 comp rx=0 tx=0).stop 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 shutdown_connections 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x55650227e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x55650227e4b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1241549941 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x55650227bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 >> 172.21.15.73:0/1241549941 conn(0x556502275680 msgr2=0x55650227c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 shutdown_connections 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1241549941 wait complete. 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 -- start start 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.461+0000 7fb1d13d9740 1 Processor -- start 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502299f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502296e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ae130 con 0x556502284350 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81ada60 con 0x556502279170 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c81aace0 con 0x556502279a90 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502299f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c77fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53020/0 (socket says 172.21.15.73:53020) 2024-06-06T07:22:33.467 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c77fe640 1 -- 172.21.15.73:0/1983659708 learned_addr learned my addr 172.21.15.73:0/1983659708 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502296e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1d13d9740 10 client.0 _ll_register_callbacks cb 0x7fb1b8009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1983659708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 msgr2=0x556502299a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1983659708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 msgr2=0x556502296e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502296e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c7fff640 1 -- 172.21.15.73:0/1983659708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565022976a0 con 0x556502279a90 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.462+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1c7fff640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502299f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5565022822d0 tx=0x7fb1b8009c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e7246326713179 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:33.468 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8045060 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1b8030d10 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb1b802f490 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb1b804c050 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 10 client.26977 fetch_fsmap learned FSMap version 80 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5565022fe800 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1b8030d10 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fb1b80304c0 con 0x556502279a90 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 10 client.26977 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 10 client.26977 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 20 client.26977 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:33.469 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502300f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 4 client.26977 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:22:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.463+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5565021f0f10 con 0x5565022942a0 2024-06-06T07:22:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.464+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502300f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.464+0000 7fb1a77fe640 10 client.26977 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:33.470 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.464+0000 7fb1c77fe640 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502300f40 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc9a992ce115c0c8 server_cookie=106d97aa544a694a in_seq=0 out_seq=0 2024-06-06T07:22:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:33 smithi073 ceph-mon[32303]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-06-06T07:22:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1a77fe640 1 -- 172.21.15.73:0/1983659708 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5565021f0f10 con 0x5565022942a0 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1a77fe640 10 client.26977 handle_command_reply: tid=0 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 client.26977 shutdown 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 msgr2=0x556502299f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502299f50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x5565022822d0 tx=0x7fb1b8009c10 comp rx=0 tx=0).stop 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 shutdown_connections 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5565022942a0 0x556502300f40 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556502284350 0x556502296e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x556502279a90 0x556502299f50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.545+0000 7fb1d13d9740 1 --2- 172.21.15.73:0/1983659708 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x556502279170 0x556502299a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.546+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 >> 172.21.15.73:0/1983659708 conn(0x556502275680 msgr2=0x556502282a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.546+0000 7fb1a77fe640 0 client.26977 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.546+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 shutdown_connections 2024-06-06T07:22:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.546+0000 7fb1d13d9740 1 -- 172.21.15.73:0/1983659708 wait complete. 2024-06-06T07:22:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.546+0000 7fb1d13d9740 20 client.26977 trim_cache size 0 max 16384 2024-06-06T07:22:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:22:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:22:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 msgr2=0x7fb1b00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 0x7fb1b00a5260 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1c009bd30 tx=0x7fb1c00b7040 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 msgr2=0x7fb1c819ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c819ec90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb1c0009fd0 tx=0x7fb1c000ef10 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 shutdown_connections 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb1c8106b40 0x7fb1c819ec90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1c8101dd0 0x7fb1c819a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb1b00a2da0 0x7fb1b00a5260 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 --2- 172.21.15.73:0/1000805704 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb1c8101410 0x7fb1c819a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.547+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 >> 172.21.15.73:0/1000805704 conn(0x7fb1c80769a0 msgr2=0x7fb1c8103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.548+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 shutdown_connections 2024-06-06T07:22:33.551 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:33.548+0000 7fb1ce6cd640 1 -- 172.21.15.73:0/1000805704 wait complete. 2024-06-06T07:22:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:33 smithi136 ceph-mon[35901]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-06-06T07:22:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:33 smithi170 ceph-mon[36108]: pgmap v675: 129 pgs: 129 active+clean; 1.1 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.7 MiB/s wr, 288 op/s 2024-06-06T07:22:33.816 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76366 8.91 MB/sec execute 338 sec latency 188.166 ms 2024-06-06T07:22:34.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76524 8.91 MB/sec execute 339 sec latency 70.174 ms 2024-06-06T07:22:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:35 smithi073 ceph-mon[32303]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-06-06T07:22:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:35 smithi136 ceph-mon[35901]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-06-06T07:22:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:35 smithi170 ceph-mon[36108]: pgmap v676: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.1 MiB/s wr, 424 op/s 2024-06-06T07:22:35.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76657 8.90 MB/sec execute 340 sec latency 108.460 ms 2024-06-06T07:22:36.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76830 8.92 MB/sec execute 341 sec latency 146.869 ms 2024-06-06T07:22:37.398 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:37 smithi170 ceph-mon[36108]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-06-06T07:22:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:37 smithi073 ceph-mon[32303]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-06-06T07:22:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:37 smithi136 ceph-mon[35901]: pgmap v677: 129 pgs: 129 active+clean; 1.1 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 267 op/s 2024-06-06T07:22:37.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 76972 8.91 MB/sec execute 342 sec latency 115.718 ms 2024-06-06T07:22:38.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77136 8.92 MB/sec execute 343 sec latency 55.478 ms 2024-06-06T07:22:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:39 smithi073 ceph-mon[32303]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-06-06T07:22:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:39 smithi136 ceph-mon[35901]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-06-06T07:22:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:39 smithi170 ceph-mon[36108]: pgmap v678: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 388 op/s 2024-06-06T07:22:39.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77306 8.91 MB/sec execute 344 sec latency 80.951 ms 2024-06-06T07:22:40.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77476 8.90 MB/sec execute 345 sec latency 240.703 ms 2024-06-06T07:22:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:41 smithi073 ceph-mon[32303]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 293 op/s 2024-06-06T07:22:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:41 smithi136 ceph-mon[35901]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 293 op/s 2024-06-06T07:22:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:41 smithi170 ceph-mon[36108]: pgmap v679: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 5.2 MiB/s wr, 293 op/s 2024-06-06T07:22:41.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77611 8.89 MB/sec execute 346 sec latency 198.187 ms 2024-06-06T07:22:42.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77741 8.88 MB/sec execute 347 sec latency 170.894 ms 2024-06-06T07:22:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:43 smithi073 ceph-mon[32303]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-06-06T07:22:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:43 smithi136 ceph-mon[35901]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-06-06T07:22:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:43 smithi170 ceph-mon[36108]: pgmap v680: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 272 op/s 2024-06-06T07:22:43.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 77899 8.89 MB/sec execute 348 sec latency 87.172 ms 2024-06-06T07:22:44.817 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78052 8.89 MB/sec execute 349 sec latency 68.569 ms 2024-06-06T07:22:45.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:45 smithi073 ceph-mon[32303]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2024-06-06T07:22:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:45 smithi136 ceph-mon[35901]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2024-06-06T07:22:45.598 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:45 smithi170 ceph-mon[36108]: pgmap v681: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 7.1 MiB/s wr, 400 op/s 2024-06-06T07:22:45.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78252 8.92 MB/sec execute 350 sec latency 115.010 ms 2024-06-06T07:22:46.694 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 -- 172.21.15.73:0/1244579344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb308103e10 msgr2=0x7fb3081041f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 --2- 172.21.15.73:0/1244579344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb308103e10 0x7fb3081041f0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc009900 tx=0x7fb2fc02a410 comp rx=0 tx=0).stop 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 -- 172.21.15.73:0/1244579344 shutdown_connections 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 --2- 172.21.15.73:0/1244579344 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308108ee0 0x7fb308110ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 --2- 172.21.15.73:0/1244579344 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb3081047c0 0x7fb308108800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 --2- 172.21.15.73:0/1244579344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb308103e10 0x7fb3081041f0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 -- 172.21.15.73:0/1244579344 >> 172.21.15.73:0/1244579344 conn(0x7fb3080ff650 msgr2=0x7fb308101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 -- 172.21.15.73:0/1244579344 shutdown_connections 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.693+0000 7fb30ec38640 1 -- 172.21.15.73:0/1244579344 wait complete. 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.694+0000 7fb30ec38640 1 Processor -- start 2024-06-06T07:22:46.695 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.694+0000 7fb30ec38640 1 -- start start 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.694+0000 7fb30ec38640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 0x7fb30819a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.694+0000 7fb30ec38640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb308108ee0 0x7fb30819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.694+0000 7fb30ec38640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30ec38640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113e40 con 0x7fb30819ec40 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54300/0 (socket says 172.21.15.73:54300) 2024-06-06T07:22:46.697 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 -- 172.21.15.73:0/3670462850 learned_addr learned my addr 172.21.15.73:0/3670462850 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb307fff640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb308108ee0 0x7fb30819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113cc0 con 0x7fb308108ee0 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 0x7fb30819a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb308113fc0 con 0x7fb308103e10 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 -- 172.21.15.73:0/3670462850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 msgr2=0x7fb30819a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 0x7fb30819a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 -- 172.21.15.73:0/3670462850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb308108ee0 msgr2=0x7fb30819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb308108ee0 0x7fb30819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30d1d5640 1 -- 172.21.15.73:0/3670462850 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30819f730 con 0x7fb30819ec40 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.695+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 0x7fb30819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb30d1d5640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb30000eb10 tx=0x7fb30000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.698 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb305ffb640 1 -- 172.21.15.73:0/3670462850 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300004b20 con 0x7fb30819ec40 2024-06-06T07:22:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb305ffb640 1 -- 172.21.15.73:0/3670462850 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb300004cc0 con 0x7fb30819ec40 2024-06-06T07:22:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb305ffb640 1 -- 172.21.15.73:0/3670462850 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb300010690 con 0x7fb30819ec40 2024-06-06T07:22:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30819fa20 con 0x7fb30819ec40 2024-06-06T07:22:46.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.696+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3081a71c0 con 0x7fb30819ec40 2024-06-06T07:22:46.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.699+0000 7fb305ffb640 1 -- 172.21.15.73:0/3670462850 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb300005020 con 0x7fb30819ec40 2024-06-06T07:22:46.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 Processor -- start 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 -- start start 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e027194170 0x55e027194550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0270f84f0 con 0x55e027194a90 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e027096870 con 0x55e027194170 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb311a28740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e0270f44b0 con 0x55e02719f350 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb305ffb640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 0x7fb2f80a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e027194170 0x55e027194550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb30c9d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:47526/0 (socket says 172.21.15.73:47526) 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb30c9d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54304/0 (socket says 172.21.15.73:54304) 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 -- 172.21.15.73:0/2344650343 learned_addr learned my addr 172.21.15.73:0/2344650343 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb305ffb640 1 -- 172.21.15.73:0/3670462850 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb3000cfd70 con 0x7fb30819ec40 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb30d1d5640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 0x7fb2f80a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 -- 172.21.15.73:0/2344650343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e027194170 msgr2=0x55e027194550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.700+0000 7fb307fff640 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e027194170 0x55e027194550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb307fff640 1 -- 172.21.15.73:0/2344650343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 msgr2=0x55e02719ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb307fff640 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb307fff640 1 -- 172.21.15.73:0/2344650343 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0270b29b0 con 0x55e02719f350 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb307fff640 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4012750 tx=0x7fb2f400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=356a65aae2c89e85 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb2f2ffd640 1 -- 172.21.15.73:0/2344650343 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4005020 con 0x55e02719f350 2024-06-06T07:22:46.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb2f2ffd640 1 -- 172.21.15.73:0/2344650343 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f40047a0 con 0x55e02719f350 2024-06-06T07:22:46.704 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.701+0000 7fb2f2ffd640 1 -- 172.21.15.73:0/2344650343 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f400f9a0 con 0x55e02719f350 2024-06-06T07:22:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- 172.21.15.73:0/2344650343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 msgr2=0x55e0271a1740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4012750 tx=0x7fb2f400b390 comp rx=0 tx=0).stop 2024-06-06T07:22:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb30d1d5640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 0x7fb2f80a5260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cd160 tx=0x7fb300005b40 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.707 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- 172.21.15.73:0/2344650343 shutdown_connections 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719f350 0x55e0271a1740 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 --2- 172.21.15.73:0/2344650343 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e027194170 0x55e027194550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- 172.21.15.73:0/2344650343 >> 172.21.15.73:0/2344650343 conn(0x55e027190680 msgr2=0x55e027190a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- 172.21.15.73:0/2344650343 shutdown_connections 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- 172.21.15.73:0/2344650343 wait complete. 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.702+0000 7fb311a28740 1 -- start start 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 Processor -- start 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e02719c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e02719cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e0271a1f80 con 0x55e027194a90 2024-06-06T07:22:46.708 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55e027081ae0 con 0x55e02719f350 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55e0271a2100 con 0x55e027194170 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb311a28740 10 client.0 _ll_register_callbacks cb 0x55e0270ba9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb30c9d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb30c9d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54316/0 (socket says 172.21.15.73:54316) 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb30c9d4640 1 -- 172.21.15.73:0/1123413851 learned_addr learned my addr 172.21.15.73:0/1123413851 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb307fff640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e02719c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.703+0000 7fb307fff640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e02719cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 -- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 msgr2=0x55e02719c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e02719c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 -- 172.21.15.73:0/1123413851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 msgr2=0x55e02719cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e02719cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.709 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 -- 172.21.15.73:0/1123413851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e026e0c210 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc02a930 tx=0x7fb2fc031810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5278404d62831f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc035040 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2f40047a0 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb2fc004240 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb2fc004440 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb311a28740 10 client.17589 fetch_fsmap learned FSMap version 80 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.704+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55e026ed8620 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2f40047a0 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fb2fc004c00 con 0x55e027194a90 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 10 client.17589 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:22:46.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 10 client.17589 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 20 client.17589 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0271af530 0x55e0271b1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 4 client.17589 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e02710bf10 con 0x55e0271af530 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.705+0000 7fb307fff640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0271af530 0x55e0271b1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.706+0000 7fb2e7fff640 10 client.17589 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.706+0000 7fb307fff640 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0271af530 0x55e0271b1940 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21028043d28a2a4 server_cookie=267338901340b45d in_seq=0 out_seq=0 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.706+0000 7fb2e7fff640 1 -- 172.21.15.73:0/1123413851 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e02710bf10 con 0x55e0271af530 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.706+0000 7fb2e7fff640 10 client.17589 handle_command_reply: tid=0 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.706+0000 7fb311a28740 1 client.17589 shutdown 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 msgr2=0x55e02719c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc02a930 tx=0x7fb2fc031810 comp rx=0 tx=0).stop 2024-06-06T07:22:46.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 shutdown_connections 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0271af530 0x55e0271b1940 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e02719cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194a90 0x55e02719c890 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb2e7fff640 0 client.17589 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 --2- 172.21.15.73:0/1123413851 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e02719c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 >> 172.21.15.73:0/1123413851 conn(0x55e027190680 msgr2=0x55e027196400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 shutdown_connections 2024-06-06T07:22:46.712 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 1 -- 172.21.15.73:0/1123413851 wait complete. 2024-06-06T07:22:46.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.707+0000 7fb311a28740 20 client.17589 trim_cache size 0 max 16384 2024-06-06T07:22:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.731+0000 7fb311a28740 1 Processor -- start 2024-06-06T07:22:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.731+0000 7fb311a28740 1 -- start start 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e027196b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 0x55e02719ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719cf60 0x55e0271b7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081b10a0 con 0x55e02719f350 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081b2be0 con 0x55e02719cf60 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb311a28740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081b1770 con 0x55e027194170 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb307fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e027196b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 0x55e02719ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb307fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e027196b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:47536/0 (socket says 172.21.15.73:47536) 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb307fff640 1 -- 172.21.15.73:0/156618478 learned_addr learned my addr 172.21.15.73:0/156618478 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb307fff640 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719cf60 0x55e0271b7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 -- 172.21.15.73:0/156618478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 msgr2=0x55e027196b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e027196b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 -- 172.21.15.73:0/156618478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719cf60 msgr2=0x55e0271b7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719cf60 0x55e0271b7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.732+0000 7fb30c9d4640 1 -- 172.21.15.73:0/156618478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e02719a8a0 con 0x55e02719f350 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.733+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 0x55e02719ca20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55e02719d3e0 tx=0x7fb2fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18669cef8ff84669 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.733+0000 7fb2f27fc640 1 -- 172.21.15.73:0/156618478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc035030 con 0x55e02719f350 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.733+0000 7fb2f27fc640 1 -- 172.21.15.73:0/156618478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2fc004fa0 con 0x55e02719f350 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.733+0000 7fb2f27fc640 1 -- 172.21.15.73:0/156618478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc003df0 con 0x55e02719f350 2024-06-06T07:22:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- 172.21.15.73:0/156618478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 msgr2=0x55e02719ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 0x55e02719ca20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x55e02719d3e0 tx=0x7fb2fc0046c0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- 172.21.15.73:0/156618478 shutdown_connections 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719cf60 0x55e0271b7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e02719f350 0x55e02719ca20 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 --2- 172.21.15.73:0/156618478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e027194170 0x55e027196b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- 172.21.15.73:0/156618478 >> 172.21.15.73:0/156618478 conn(0x55e027190680 msgr2=0x55e02719c310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- 172.21.15.73:0/156618478 shutdown_connections 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- 172.21.15.73:0/156618478 wait complete. 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 -- start start 2024-06-06T07:22:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.734+0000 7fb311a28740 1 Processor -- start 2024-06-06T07:22:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719cf60 0x55e0271a39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e027219220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081b2510 con 0x55e027194170 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081b1e40 con 0x55e02719f350 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb3081af0a0 con 0x55e02719cf60 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb311a28740 10 client.0 _ll_register_callbacks cb 0x7fb2fc005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54342/0 (socket says 172.21.15.73:54342) 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 -- 172.21.15.73:0/3812385640 learned_addr learned my addr 172.21.15.73:0/3812385640 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719cf60 0x55e0271a39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e027219220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 -- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719cf60 msgr2=0x55e0271a39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719cf60 0x55e0271a39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 -- 172.21.15.73:0/3812385640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 msgr2=0x55e027219220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e027219220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.735+0000 7fb307fff640 1 -- 172.21.15.73:0/3812385640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e027219760 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb307fff640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4015040 tx=0x7fb2fc009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59ab106074ff4cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb2e7fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2f4004250 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fb2f40043f0 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2fc004fa0 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fb2f4040050 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb311a28740 10 client.17595 fetch_fsmap learned FSMap version 80 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.736+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x55e0272230b0 con 0x55e027194170 2024-06-06T07:22:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2fc004fa0 con 0x55e027194170 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fb2f40050c0 con 0x55e027194170 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 10 client.17595 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 10 client.17595 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 20 client.17595 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0272237d0 0x55e027225bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 4 client.17595 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e027105c20 con 0x55e0272237d0 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0272237d0 0x55e027225bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb2e7fff640 10 client.17595 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.737+0000 7fb30c9d4640 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0272237d0 0x55e027225bc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64c6bc55d3d3e0d4 server_cookie=ee60f5f585e7801e in_seq=0 out_seq=0 2024-06-06T07:22:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb2e7fff640 1 -- 172.21.15.73:0/3812385640 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e027105c20 con 0x55e0272237d0 2024-06-06T07:22:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb2e7fff640 10 client.17595 handle_command_reply: tid=0 2024-06-06T07:22:46.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 client.17595 shutdown 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 msgr2=0x55e0271a34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb2f4015040 tx=0x7fb2fc009610 comp rx=0 tx=0).stop 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 shutdown_connections 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55e0272237d0 0x55e027225bc0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb2e7fff640 0 client.17595 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55e02719f350 0x55e027219220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55e02719cf60 0x55e0271a39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 --2- 172.21.15.73:0/3812385640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e027194170 0x55e0271a34a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 >> 172.21.15.73:0/3812385640 conn(0x55e027190680 msgr2=0x55e0271aa1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 shutdown_connections 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 1 -- 172.21.15.73:0/3812385640 wait complete. 2024-06-06T07:22:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.810+0000 7fb311a28740 20 client.17595 trim_cache size 0 max 16384 2024-06-06T07:22:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:22:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:22:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 msgr2=0x7fb2f80a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 0x7fb2f80a5260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb3000cd160 tx=0x7fb300005b40 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 msgr2=0x7fb30819f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fb30000eb10 tx=0x7fb30000c2d0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 shutdown_connections 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb30819ec40 0x7fb30819f0f0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb308108ee0 0x7fb30819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb2f80a2da0 0x7fb2f80a5260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 --2- 172.21.15.73:0/3670462850 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb308103e10 0x7fb30819a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.811+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 >> 172.21.15.73:0/3670462850 conn(0x7fb3080ff650 msgr2=0x7fb308100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.812+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 shutdown_connections 2024-06-06T07:22:46.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:46.812+0000 7fb30ec38640 1 -- 172.21.15.73:0/3670462850 wait complete. 2024-06-06T07:22:46.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78425 8.90 MB/sec execute 351 sec latency 123.775 ms 2024-06-06T07:22:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:47 smithi073 ceph-mon[32303]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-06-06T07:22:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:47 smithi136 ceph-mon[35901]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-06-06T07:22:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:47 smithi170 ceph-mon[36108]: pgmap v682: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 4.8 MiB/s wr, 265 op/s 2024-06-06T07:22:47.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78580 8.89 MB/sec execute 352 sec latency 95.707 ms 2024-06-06T07:22:48.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78717 8.88 MB/sec execute 353 sec latency 87.828 ms 2024-06-06T07:22:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:49 smithi073 ceph-mon[32303]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 393 op/s 2024-06-06T07:22:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:49 smithi136 ceph-mon[35901]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 393 op/s 2024-06-06T07:22:49.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:49 smithi170 ceph-mon[36108]: pgmap v683: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.9 MiB/s wr, 393 op/s 2024-06-06T07:22:49.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 78870 8.87 MB/sec execute 354 sec latency 115.122 ms 2024-06-06T07:22:50.818 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79047 8.89 MB/sec execute 355 sec latency 83.144 ms 2024-06-06T07:22:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:51 smithi073 ceph-mon[32303]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-06-06T07:22:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:51 smithi136 ceph-mon[35901]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-06-06T07:22:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:51 smithi170 ceph-mon[36108]: pgmap v684: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 290 op/s 2024-06-06T07:22:51.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79213 8.89 MB/sec execute 356 sec latency 159.542 ms 2024-06-06T07:22:52.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79371 8.89 MB/sec execute 357 sec latency 65.948 ms 2024-06-06T07:22:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:53 smithi073 ceph-mon[32303]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:22:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:53 smithi136 ceph-mon[35901]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:22:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:53 smithi170 ceph-mon[36108]: pgmap v685: 129 pgs: 129 active+clean; 1.2 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.7 MiB/s wr, 274 op/s 2024-06-06T07:22:53.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79535 8.90 MB/sec execute 358 sec latency 62.052 ms 2024-06-06T07:22:54.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79716 8.89 MB/sec execute 359 sec latency 68.662 ms 2024-06-06T07:22:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:55 smithi073 ceph-mon[32303]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 401 op/s 2024-06-06T07:22:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:55 smithi136 ceph-mon[35901]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 401 op/s 2024-06-06T07:22:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:55 smithi170 ceph-mon[36108]: pgmap v686: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.3 MiB/s wr, 401 op/s 2024-06-06T07:22:55.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 79861 8.89 MB/sec execute 360 sec latency 101.458 ms 2024-06-06T07:22:56.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80007 8.88 MB/sec execute 361 sec latency 82.790 ms 2024-06-06T07:22:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:57 smithi073 ceph-mon[32303]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 273 op/s 2024-06-06T07:22:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:57 smithi136 ceph-mon[35901]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 273 op/s 2024-06-06T07:22:57.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:57 smithi170 ceph-mon[36108]: pgmap v687: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 273 op/s 2024-06-06T07:22:57.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80160 8.88 MB/sec execute 362 sec latency 110.733 ms 2024-06-06T07:22:58.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80357 8.90 MB/sec execute 363 sec latency 113.347 ms 2024-06-06T07:22:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:22:59 smithi073 ceph-mon[32303]: pgmap v688: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2024-06-06T07:22:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:22:59 smithi136 ceph-mon[35901]: pgmap v688: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2024-06-06T07:22:59.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:22:59 smithi170 ceph-mon[36108]: pgmap v688: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 7.4 MiB/s wr, 401 op/s 2024-06-06T07:22:59.820 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80523 8.90 MB/sec execute 364 sec latency 88.558 ms 2024-06-06T07:22:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 -- 172.21.15.73:0/1053172655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe80806a710 msgr2=0x7fe80806ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 --2- 172.21.15.73:0/1053172655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe80806a710 0x7fe80806ab90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8009900 tx=0x7fe7f802a400 comp rx=0 tx=0).stop 2024-06-06T07:22:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 -- 172.21.15.73:0/1053172655 shutdown_connections 2024-06-06T07:22:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 --2- 172.21.15.73:0/1053172655 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe808104390 0x7fe808104810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 --2- 172.21.15.73:0/1053172655 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe80806a710 0x7fe80806ab90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 --2- 172.21.15.73:0/1053172655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe80806a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 -- 172.21.15.73:0/1053172655 >> 172.21.15.73:0/1053172655 conn(0x7fe8080766a0 msgr2=0x7fe808076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.946+0000 7fe810153640 1 -- 172.21.15.73:0/1053172655 shutdown_connections 2024-06-06T07:22:59.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.947+0000 7fe810153640 1 -- 172.21.15.73:0/1053172655 wait complete. 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.947+0000 7fe810153640 1 Processor -- start 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.947+0000 7fe810153640 1 -- start start 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe80806a710 0x7fe80819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe8081a2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe808118160 con 0x7fe8081069f0 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe808117fe0 con 0x7fe80806a710 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe810153640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe8081182e0 con 0x7fe808104390 2024-06-06T07:22:59.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80e6f0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe8081a2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:51628/0 (socket says 172.21.15.73:51628) 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 -- 172.21.15.73:0/3762864550 learned_addr learned my addr 172.21.15.73:0/3762864550 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 -- 172.21.15.73:0/3762864550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe80806a710 msgr2=0x7fe80819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe80806a710 0x7fe80819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 -- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 msgr2=0x7fe8081a2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.948+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe8081a2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe80d6ee640 1 -- 172.21.15.73:0/3762864550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8081a35e0 con 0x7fe808104390 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe8081a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005940 tx=0x7fe7f80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe7feffd640 1 -- 172.21.15.73:0/3762864550 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f803f030 con 0x7fe808104390 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8081a3870 con 0x7fe808104390 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8081ab160 con 0x7fe808104390 2024-06-06T07:22:59.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe7feffd640 1 -- 172.21.15.73:0/3762864550 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f80044c0 con 0x7fe808104390 2024-06-06T07:22:59.952 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.949+0000 7fe7feffd640 1 -- 172.21.15.73:0/3762864550 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f803f030 con 0x7fe808104390 2024-06-06T07:22:59.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.952+0000 7fe7feffd640 1 -- 172.21.15.73:0/3762864550 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe7f8042030 con 0x7fe808104390 2024-06-06T07:22:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.953+0000 7fe7feffd640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 0x7fe7f40a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.953+0000 7fe80deef640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 0x7fe7f40a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.954+0000 7fe7feffd640 1 -- 172.21.15.73:0/3762864550 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fe7f8040070 con 0x7fe808104390 2024-06-06T07:22:59.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.954+0000 7fe80deef640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 0x7fe7f40a5670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe804009270 tx=0x7fe8040091e0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.956+0000 7fe812e5e740 1 Processor -- start 2024-06-06T07:22:59.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.956+0000 7fe812e5e740 1 -- start start 2024-06-06T07:22:59.958 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff20e350 0x564aff210740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564aff1674f0 con 0x564aff203170 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564aff105870 con 0x564aff203a90 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe812e5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564aff1634b0 con 0x564aff20e350 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52332/0 (socket says 172.21.15.73:52332) 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 -- 172.21.15.73:0/4266365129 learned_addr learned my addr 172.21.15.73:0/4266365129 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:59.959 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff20e350 0x564aff210740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 -- 172.21.15.73:0/4266365129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff20e350 msgr2=0x564aff210740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff20e350 0x564aff210740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 -- 172.21.15.73:0/4266365129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 msgr2=0x564aff20de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 -- 172.21.15.73:0/4266365129 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aff1219b0 con 0x564aff203170 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.957+0000 7fe80deef640 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe804010560 tx=0x7fe804011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff336670924b4793 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.958+0000 7fe7f3fff640 1 -- 172.21.15.73:0/4266365129 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804020070 con 0x564aff203170 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.958+0000 7fe7f3fff640 1 -- 172.21.15.73:0/4266365129 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe804004d20 con 0x564aff203170 2024-06-06T07:22:59.960 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.958+0000 7fe7f3fff640 1 -- 172.21.15.73:0/4266365129 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804005400 con 0x564aff203170 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- 172.21.15.73:0/4266365129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 msgr2=0x564aff203550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe804010560 tx=0x7fe804011000 comp rx=0 tx=0).stop 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- 172.21.15.73:0/4266365129 shutdown_connections 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff20e350 0x564aff210740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4266365129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203170 0x564aff203550 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- 172.21.15.73:0/4266365129 >> 172.21.15.73:0/4266365129 conn(0x564aff1ff680 msgr2=0x564aff1ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:59.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- 172.21.15.73:0/4266365129 shutdown_connections 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- 172.21.15.73:0/4266365129 wait complete. 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 -- start start 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.959+0000 7fe812e5e740 1 Processor -- start 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff20bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564aff210f80 con 0x564aff20e350 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x564aff0f0ae0 con 0x564aff203a90 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564aff211100 con 0x564aff203170 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe812e5e740 10 client.0 _ll_register_callbacks cb 0x564aff1299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:51652/0 (socket says 172.21.15.73:51652) 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 -- 172.21.15.73:0/4231919875 learned_addr learned my addr 172.21.15.73:0/4231919875 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff20bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 -- 172.21.15.73:0/4231919875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 msgr2=0x564aff20b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 -- 172.21.15.73:0/4231919875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 msgr2=0x564aff20bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff20bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80deef640 1 -- 172.21.15.73:0/4231919875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afee7b210 con 0x564aff203170 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.960+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff20bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe80deef640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe80400f010 tx=0x7fe80401cbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7d982c652cedf30 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe7f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:22:59.968 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe804033060 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fe804005310 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe804004d20 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe80403a050 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe812e5e740 10 client.27007 fetch_fsmap learned FSMap version 80 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x564afef47620 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.961+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8040059c0 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fe80402e6a0 con 0x564aff203170 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 10 client.27007 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 10 client.27007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 20 client.27007 populate_metadata read hostname 'smithi073' 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff2207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 4 client.27007 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:22:59.969 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe804004d20 con 0x564aff21e3b0 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff2207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe7f17fa640 10 client.27007 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.962+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff2207a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642ddfdd6ed27be2 server_cookie=6000eec3ba74b9bb in_seq=0 out_seq=0 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe7f17fa640 1 -- 172.21.15.73:0/4231919875 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe804004d20 con 0x564aff21e3b0 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe7f17fa640 10 client.27007 handle_command_reply: tid=0 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 client.27007 shutdown 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 msgr2=0x564aff20b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe80400f010 tx=0x7fe80401cbc0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 shutdown_connections 2024-06-06T07:22:59.970 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff2207a0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff20bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe7f17fa640 0 client.27007 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff20b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 --2- 172.21.15.73:0/4231919875 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20b220 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 >> 172.21.15.73:0/4231919875 conn(0x564aff1ff680 msgr2=0x564aff205400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 shutdown_connections 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.963+0000 7fe812e5e740 1 -- 172.21.15.73:0/4231919875 wait complete. 2024-06-06T07:22:59.971 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.964+0000 7fe812e5e740 20 client.27007 trim_cache size 0 max 16384 2024-06-06T07:22:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 Processor -- start 2024-06-06T07:22:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 -- start start 2024-06-06T07:22:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 0x564aff225da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 0x564aff2262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec010090 con 0x564aff203a90 2024-06-06T07:22:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec011bd0 con 0x564aff20e350 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.989+0000 7fe812e5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec010760 con 0x564aff203170 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 0x564aff225da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80deef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80deef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:51660/0 (socket says 172.21.15.73:51660) 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80deef640 1 -- 172.21.15.73:0/2832230099 learned_addr learned my addr 172.21.15.73:0/2832230099 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 0x564aff2262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 -- 172.21.15.73:0/2832230099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 msgr2=0x564aff20bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 -- 172.21.15.73:0/2832230099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 msgr2=0x564aff2262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 0x564aff2262e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 -- 172.21.15.73:0/2832230099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aff2038c0 con 0x564aff203a90 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80e6f0640 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 0x564aff2262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.990+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 0x564aff225da0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ec7d0 tx=0x7fe7f80b7240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ba79ff3454f3b80 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.991+0000 7fe7f37fe640 1 -- 172.21.15.73:0/2832230099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8035040 con 0x564aff203a90 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.991+0000 7fe7f37fe640 1 -- 172.21.15.73:0/2832230099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f80b9640 con 0x564aff203a90 2024-06-06T07:22:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.991+0000 7fe7f37fe640 1 -- 172.21.15.73:0/2832230099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f803f040 con 0x564aff203a90 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.991+0000 7fe812e5e740 1 -- 172.21.15.73:0/2832230099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 msgr2=0x564aff225da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.991+0000 7fe812e5e740 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 0x564aff225da0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ec7d0 tx=0x7fe7f80b7240 comp rx=0 tx=0).stop 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 -- 172.21.15.73:0/2832230099 shutdown_connections 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff20e350 0x564aff2262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff203a90 0x564aff225da0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 --2- 172.21.15.73:0/2832230099 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff20bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 -- 172.21.15.73:0/2832230099 >> 172.21.15.73:0/2832230099 conn(0x564aff1ff680 msgr2=0x564aff209af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 -- 172.21.15.73:0/2832230099 shutdown_connections 2024-06-06T07:22:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 -- 172.21.15.73:0/2832230099 wait complete. 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 -- start start 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.992+0000 7fe812e5e740 1 Processor -- start 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff211340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff211dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec011500 con 0x564aff20e350 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec010e30 con 0x564aff203a90 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ec00e090 con 0x564aff203170 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe812e5e740 10 client.0 _ll_register_callbacks cb 0x7fe7f8005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80e6f0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff211dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:35690/0 (socket says 172.21.15.73:35690) 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80deef640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff211340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 -- 172.21.15.73:0/324884268 learned_addr learned my addr 172.21.15.73:0/324884268 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 -- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 msgr2=0x564aff211340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff211340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 -- 172.21.15.73:0/324884268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 msgr2=0x564aff211dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff211dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 -- 172.21.15.73:0/324884268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aff286e10 con 0x564aff203a90 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe80d6ee640 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ecdd0 tx=0x7fe7f80c42d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86fe9a49ce8e8bc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe7f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.993+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7f803f070 con 0x564aff203a90 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(138..138 src has 1..138) ==== 9554+0+0 (secure 0 0 0) 0x7fe7f80eca50 con 0x564aff203a90 2024-06-06T07:22:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7f80b9640 con 0x564aff203a90 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fe7f8035070 con 0x564aff203a90 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 10 client.26828 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x564aff288120 con 0x564aff203a90 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe7f80b9640 con 0x564aff203a90 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fe7f80ba9f0 con 0x564aff203a90 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 10 client.26828 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 20 client.26828 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff28a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 4 client.26828 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:23:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564aff17af10 con 0x564aff21e3b0 2024-06-06T07:23:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.994+0000 7fe80deef640 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff28a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.995+0000 7fe80deef640 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff28a850 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b49d04e691aba9 server_cookie=b94be3dee58dc6b8 in_seq=0 out_seq=0 2024-06-06T07:23:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:22:59.995+0000 7fe7f17fa640 10 client.26828 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:00.108 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.108+0000 7fe7f17fa640 1 -- 172.21.15.73:0/324884268 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564aff17af10 con 0x564aff21e3b0 2024-06-06T07:23:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.108+0000 7fe7f17fa640 10 client.26828 handle_command_reply: tid=0 2024-06-06T07:23:00.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.108+0000 7fe812e5e740 1 client.26828 shutdown 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.108+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 msgr2=0x564aff211880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.108+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe7f80ecdd0 tx=0x7fe7f80c42d0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 shutdown_connections 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564aff21e3b0 0x564aff28a850 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564aff20e350 0x564aff211dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564aff203a90 0x564aff211880 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 --2- 172.21.15.73:0/324884268 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564aff203170 0x564aff211340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe7f17fa640 0 client.26828 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 >> 172.21.15.73:0/324884268 conn(0x564aff1ff680 msgr2=0x564aff1fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:00.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 shutdown_connections 2024-06-06T07:23:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 1 -- 172.21.15.73:0/324884268 wait complete. 2024-06-06T07:23:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.109+0000 7fe812e5e740 20 client.26828 trim_cache size 0 max 16384 2024-06-06T07:23:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:23:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:23:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:23:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 msgr2=0x7fe7f40a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 0x7fe7f40a5670 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe804009270 tx=0x7fe8040091e0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 msgr2=0x7fe80819eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe7f8005940 tx=0x7fe7f80040c0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 shutdown_connections 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8081069f0 0x7fe8081a2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe7f40a31b0 0x7fe7f40a5670 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe808104390 0x7fe80819eb80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 --2- 172.21.15.73:0/3762864550 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe80806a710 0x7fe80819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 >> 172.21.15.73:0/3762864550 conn(0x7fe8080766a0 msgr2=0x7fe8081030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 shutdown_connections 2024-06-06T07:23:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:00.110+0000 7fe810153640 1 -- 172.21.15.73:0/3762864550 wait complete. 2024-06-06T07:23:00.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80600 8.88 MB/sec execute 365 sec latency 569.567 ms 2024-06-06T07:23:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:01 smithi073 ceph-mon[32303]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 276 op/s 2024-06-06T07:23:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:01 smithi136 ceph-mon[35901]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 276 op/s 2024-06-06T07:23:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:01 smithi170 ceph-mon[36108]: pgmap v689: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.3 MiB/s wr, 276 op/s 2024-06-06T07:23:01.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80682 8.86 MB/sec execute 366 sec latency 839.195 ms 2024-06-06T07:23:02.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 80857 8.88 MB/sec execute 367 sec latency 1629.277 ms 2024-06-06T07:23:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:03 smithi073 ceph-mon[32303]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-06-06T07:23:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:03 smithi136 ceph-mon[35901]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-06-06T07:23:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:03 smithi170 ceph-mon[36108]: pgmap v690: 129 pgs: 129 active+clean; 1.2 GiB data, 7.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 5.0 MiB/s wr, 259 op/s 2024-06-06T07:23:03.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81032 8.88 MB/sec execute 368 sec latency 2629.315 ms 2024-06-06T07:23:04.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81138 8.87 MB/sec execute 369 sec latency 3629.398 ms 2024-06-06T07:23:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:05 smithi073 ceph-mon[32303]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 415 op/s 2024-06-06T07:23:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:05 smithi073 ceph-mon[32303]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T07:23:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:05 smithi136 ceph-mon[35901]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 415 op/s 2024-06-06T07:23:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:05 smithi136 ceph-mon[35901]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T07:23:05.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:05 smithi170 ceph-mon[36108]: pgmap v691: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.4 MiB/s wr, 415 op/s 2024-06-06T07:23:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:05 smithi170 ceph-mon[36108]: osdmap e139: 12 total, 12 up, 12 in 2024-06-06T07:23:05.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81259 8.86 MB/sec execute 370 sec latency 4629.435 ms 2024-06-06T07:23:06.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81441 8.86 MB/sec execute 371 sec latency 5629.469 ms 2024-06-06T07:23:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:07 smithi073 ceph-mon[32303]: pgmap v693: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 346 op/s 2024-06-06T07:23:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:07 smithi136 ceph-mon[35901]: pgmap v693: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 346 op/s 2024-06-06T07:23:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:07 smithi170 ceph-mon[36108]: pgmap v693: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.9 MiB/s wr, 346 op/s 2024-06-06T07:23:07.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81589 8.85 MB/sec execute 372 sec latency 6629.616 ms 2024-06-06T07:23:08.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81713 8.84 MB/sec execute 373 sec latency 7629.651 ms 2024-06-06T07:23:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:09 smithi073 ceph-mon[32303]: pgmap v694: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 337 op/s 2024-06-06T07:23:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:09 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:23:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:09 smithi136 ceph-mon[35901]: pgmap v694: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 337 op/s 2024-06-06T07:23:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:09 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:23:09.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:09 smithi170 ceph-mon[36108]: pgmap v694: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 337 op/s 2024-06-06T07:23:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:09 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:23:09.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 81849 8.84 MB/sec execute 374 sec latency 8629.684 ms 2024-06-06T07:23:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:10 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:23:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:10 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:23:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:10 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:23:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:10 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:23:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:10 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:23:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:10 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:23:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:10 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:23:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:10 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:23:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:10 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:23:10.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82025 8.85 MB/sec execute 375 sec latency 9629.716 ms 2024-06-06T07:23:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:11 smithi073 ceph-mon[32303]: pgmap v695: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-06-06T07:23:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:11 smithi136 ceph-mon[35901]: pgmap v695: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-06-06T07:23:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:11 smithi170 ceph-mon[36108]: pgmap v695: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 338 op/s 2024-06-06T07:23:11.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82214 8.85 MB/sec execute 376 sec latency 10629.750 ms 2024-06-06T07:23:12.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82351 8.85 MB/sec execute 377 sec latency 11629.780 ms 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.240+0000 7fcb175dc640 1 -- 172.21.15.73:0/1046578011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 msgr2=0x7fcb101008c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.240+0000 7fcb175dc640 1 --2- 172.21.15.73:0/1046578011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb101008c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800adb0 tx=0x7fcb0802b7a0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 -- 172.21.15.73:0/1046578011 shutdown_connections 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 --2- 172.21.15.73:0/1046578011 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb10105380 0x7fcb1010cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 --2- 172.21.15.73:0/1046578011 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10100e90 0x7fcb10104ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 --2- 172.21.15.73:0/1046578011 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb101008c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 -- 172.21.15.73:0/1046578011 >> 172.21.15.73:0/1046578011 conn(0x7fcb10075c20 msgr2=0x7fcb10076030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 -- 172.21.15.73:0/1046578011 shutdown_connections 2024-06-06T07:23:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.241+0000 7fcb175dc640 1 -- 172.21.15.73:0/1046578011 wait complete. 2024-06-06T07:23:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.242+0000 7fcb175dc640 1 Processor -- start 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.242+0000 7fcb175dc640 1 -- start start 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.242+0000 7fcb175dc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb10195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb175dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb175dc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb1019a800 0x7fcb1019acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb15378640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb10195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb15378640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb10195f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47758/0 (socket says 172.21.15.73:47758) 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57452/0 (socket says 172.21.15.73:57452) 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb15378640 1 -- 172.21.15.73:0/4063540481 learned_addr learned my addr 172.21.15.73:0/4063540481 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010fa80 con 0x7fcb10105380 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010f900 con 0x7fcb101004e0 2024-06-06T07:23:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb1010fc00 con 0x7fcb1019a800 2024-06-06T07:23:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.243+0000 7fcb15b79640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb1019a800 0x7fcb1019acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 -- 172.21.15.73:0/4063540481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb1019a800 msgr2=0x7fcb1019acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb1019a800 0x7fcb1019acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 -- 172.21.15.73:0/4063540481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 msgr2=0x7fcb10195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb10195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb1019b2f0 con 0x7fcb10105380 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb14b77640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcb100771e0 tx=0x7fcb0c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb067fc640 1 -- 172.21.15.73:0/4063540481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c00cc40 con 0x7fcb10105380 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb067fc640 1 -- 172.21.15.73:0/4063540481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c00cde0 con 0x7fcb10105380 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb067fc640 1 -- 172.21.15.73:0/4063540481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c004a00 con 0x7fcb10105380 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb1019b5e0 con 0x7fcb10105380 2024-06-06T07:23:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.244+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb101a2dd0 con 0x7fcb10105380 2024-06-06T07:23:13.247 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.247+0000 7fcb067fc640 1 -- 172.21.15.73:0/4063540481 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcb0c004ba0 con 0x7fcb10105380 2024-06-06T07:23:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.248+0000 7fcb067fc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 0x7fcafc0a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.248+0000 7fcb1a3c6740 1 Processor -- start 2024-06-06T07:23:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.248+0000 7fcb1a3c6740 1 -- start start 2024-06-06T07:23:13.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.248+0000 7fcb15378640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 0x7fcafc0a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb067fc640 1 -- 172.21.15.73:0/4063540481 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7fcb0c0cf820 con 0x7fcb10105380 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89170 0x560ce0c89550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c94350 0x560ce0c96740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ce0bed4f0 con 0x560ce0c94350 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ce0b8b870 con 0x560ce0c89170 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560ce0be94b0 con 0x560ce0c89a90 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb15b79640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c94350 0x560ce0c96740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:39596/0 (socket says 172.21.15.73:39596) 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 -- 172.21.15.73:0/1245718248 learned_addr learned my addr 172.21.15.73:0/1245718248 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 -- 172.21.15.73:0/1245718248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89170 msgr2=0x560ce0c89550 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:23:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89170 0x560ce0c89550 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 -- 172.21.15.73:0/1245718248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c94350 msgr2=0x560ce0c96740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c94350 0x560ce0c96740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 -- 172.21.15.73:0/1245718248 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce0ba79b0 con 0x560ce0c89a90 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb15378640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 0x7fcafc0a9790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb08004af0 tx=0x7fcb0802bfa0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.249+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0a4010 tx=0x7fcb0c09b230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b05931581e0b0d84 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcafb7fe640 1 -- 172.21.15.73:0/1245718248 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c01c070 con 0x560ce0c89a90 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcafb7fe640 1 -- 172.21.15.73:0/1245718248 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c09b8d0 con 0x560ce0c89a90 2024-06-06T07:23:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcafb7fe640 1 -- 172.21.15.73:0/1245718248 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c09bb30 con 0x560ce0c89a90 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1245718248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 msgr2=0x560ce0c93e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c0a4010 tx=0x7fcb0c09b230 comp rx=0 tx=0).stop 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1245718248 shutdown_connections 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c94350 0x560ce0c96740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c89a90 0x560ce0c93e10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1245718248 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89170 0x560ce0c89550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.250+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1245718248 >> 172.21.15.73:0/1245718248 conn(0x560ce0c85680 msgr2=0x560ce0c85a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.251+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1245718248 shutdown_connections 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.251+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1245718248 wait complete. 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.251+0000 7fcb1a3c6740 1 -- start start 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.251+0000 7fcb1a3c6740 1 Processor -- start 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c91d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560ce0c96f80 con 0x560ce0c89170 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb15378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560ce0b76ae0 con 0x560ce0c89a90 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560ce0c97100 con 0x560ce0c94350 2024-06-06T07:23:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47778/0 (socket says 172.21.15.73:47778) 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb15378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57478/0 (socket says 172.21.15.73:57478) 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb14b77640 1 -- 172.21.15.73:0/3458823870 learned_addr learned my addr 172.21.15.73:0/3458823870 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb1a3c6740 10 client.0 _ll_register_callbacks cb 0x560ce0baf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb15b79640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c91d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb14b77640 1 -- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 msgr2=0x560ce0c91d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.252+0000 7fcb14b77640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c91d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb14b77640 1 -- 172.21.15.73:0/3458823870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 msgr2=0x560ce0c912d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb14b77640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb14b77640 1 -- 172.21.15.73:0/3458823870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce0901210 con 0x560ce0c89a90 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb15b79640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c91d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb15378640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb14b77640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c086440 tx=0x7fcb0c005140 comp rx=0 tx=0).ready entity=mon.2 client_cookie=587bdc437585975f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcaf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c080060 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7fcb0c0ba170 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb0c09b8d0 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.253+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c01c070 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 10 client.26837 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x560ce09cd620 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c09b8d0 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fcb0c0a4160 con 0x560ce0c89a90 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 10 client.26837 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 10 client.26837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 20 client.26837 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0ca4500 0x560ce0ca6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 4 client.26837 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:23:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.254+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ce0c00f10 con 0x560ce0ca4500 2024-06-06T07:23:13.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.255+0000 7fcb15378640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0ca4500 0x560ce0ca6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.255+0000 7fcaf8ff9640 10 client.26837 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.255+0000 7fcb15378640 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0ca4500 0x560ce0ca6940 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b69417e50f0bdfaa server_cookie=e1b87f07285906b1 in_seq=0 out_seq=0 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/3458823870 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ce0c00f10 con 0x560ce0ca4500 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcaf8ff9640 10 client.26837 handle_command_reply: tid=0 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 client.26837 shutdown 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 msgr2=0x560ce0c91810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c086440 tx=0x7fcb0c005140 comp rx=0 tx=0).stop 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 shutdown_connections 2024-06-06T07:23:13.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0ca4500 0x560ce0ca6940 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c91d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c91810 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3458823870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c912d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcaf8ff9640 0 client.26837 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 >> 172.21.15.73:0/3458823870 conn(0x560ce0c85680 msgr2=0x560ce0c8b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 shutdown_connections 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3458823870 wait complete. 2024-06-06T07:23:13.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.256+0000 7fcb1a3c6740 20 client.26837 trim_cache size 0 max 16384 2024-06-06T07:23:13.285 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 Processor -- start 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 -- start start 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c93210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0cac210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101accf0 con 0x560ce0c89170 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101ae830 con 0x560ce0c89a90 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101ad3c0 con 0x560ce0c94350 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.284+0000 7fcb15378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c93210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57494/0 (socket says 172.21.15.73:57494) 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 -- 172.21.15.73:0/3569932038 learned_addr learned my addr 172.21.15.73:0/3569932038 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15b79640 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0cac210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 -- 172.21.15.73:0/3569932038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 msgr2=0x560ce0cac210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0cac210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 -- 172.21.15.73:0/3569932038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 msgr2=0x560ce0c93210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c93210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 -- 172.21.15.73:0/3569932038 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ce0c868b0 con 0x560ce0c89170 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb14b77640 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c93210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcb15378640 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ab70 tx=0x7fcb08042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37f443e82b98178 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcafaffd640 1 -- 172.21.15.73:0/3569932038 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb08030710 con 0x560ce0c89170 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.285+0000 7fcafaffd640 1 -- 172.21.15.73:0/3569932038 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb08058070 con 0x560ce0c89170 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.286+0000 7fcafaffd640 1 -- 172.21.15.73:0/3569932038 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0804a420 con 0x560ce0c89170 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.286+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3569932038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 msgr2=0x560ce0c8bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.286+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fcb0800ab70 tx=0x7fcb08042000 comp rx=0 tx=0).stop 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3569932038 shutdown_connections 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0cac210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c93210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/3569932038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c8bb30 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3569932038 >> 172.21.15.73:0/3569932038 conn(0x560ce0c85680 msgr2=0x560ce0c92b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3569932038 shutdown_connections 2024-06-06T07:23:13.291 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/3569932038 wait complete. 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 -- start start 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.287+0000 7fcb1a3c6740 1 Processor -- start 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c9c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c8d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101ae160 con 0x560ce0c89170 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101ada90 con 0x560ce0c89a90 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcb101aacf0 con 0x560ce0c94350 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47792/0 (socket says 172.21.15.73:47792) 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 -- 172.21.15.73:0/1021564544 learned_addr learned my addr 172.21.15.73:0/1021564544 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb15378640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c9c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb15b79640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c8d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb1a3c6740 10 client.0 _ll_register_callbacks cb 0x7fcb00009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 -- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 msgr2=0x560ce0c8d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c8d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 -- 172.21.15.73:0/1021564544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 msgr2=0x560ce0c9c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c9c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.288+0000 7fcb14b77640 1 -- 172.21.15.73:0/1021564544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ce0c8dba0 con 0x560ce0c89a90 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcb14b77640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c09b5d0 tx=0x7fcb0c0cd090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10ef3c7ed11cccbc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcaf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0b8070 con 0x560ce0c89a90 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7fcb0c097620 con 0x560ce0c89a90 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb08058070 con 0x560ce0c89a90 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fcb0c080040 con 0x560ce0c89a90 2024-06-06T07:23:13.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcb1a3c6740 10 client.26843 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x560ce0d0e7d0 con 0x560ce0c89a90 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.289+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcb0c0cf5e0 con 0x560ce0c89a90 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fcb0c0a40b0 con 0x560ce0c89a90 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 10 client.26843 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 10 client.26843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 20 client.26843 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0d0ed00 0x560ce0d11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 4 client.26843 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fcb08058070 con 0x560ce0d0ed00 2024-06-06T07:23:13.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.290+0000 7fcb15378640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0d0ed00 0x560ce0d11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.291+0000 7fcaf8ff9640 10 client.26843 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:13.295 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.291+0000 7fcb15378640 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0d0ed00 0x560ce0d11120 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a781e369e2f3db4 server_cookie=b73ddd722036221a in_seq=0 out_seq=0 2024-06-06T07:23:13.379 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcaf8ff9640 1 -- 172.21.15.73:0/1021564544 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fcb08058070 con 0x560ce0d0ed00 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcaf8ff9640 10 client.26843 handle_command_reply: tid=0 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 client.26843 shutdown 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 msgr2=0x560ce0c9c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fcb0c09b5d0 tx=0x7fcb0c0cd090 comp rx=0 tx=0).stop 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 shutdown_connections 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560ce0d0ed00 0x560ce0d11120 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560ce0c94350 0x560ce0c8d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560ce0c89a90 0x560ce0c9c890 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 --2- 172.21.15.73:0/1021564544 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560ce0c89170 0x560ce0c9c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 >> 172.21.15.73:0/1021564544 conn(0x560ce0c85680 msgr2=0x560ce0c92b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcaf8ff9640 0 client.26843 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.378+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 shutdown_connections 2024-06-06T07:23:13.380 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb1a3c6740 1 -- 172.21.15.73:0/1021564544 wait complete. 2024-06-06T07:23:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb1a3c6740 20 client.26843 trim_cache size 0 max 16384 2024-06-06T07:23:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:23:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:23:13.381 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 msgr2=0x7fcafc0a9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 0x7fcafc0a9790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fcb08004af0 tx=0x7fcb0802bfa0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 msgr2=0x7fcb10196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.379+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fcb100771e0 tx=0x7fcb0c00ee50 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 shutdown_connections 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcafc0a72d0 0x7fcafc0a9790 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcb1019a800 0x7fcb1019acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb10105380 0x7fcb10196480 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 --2- 172.21.15.73:0/4063540481 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcb101004e0 0x7fcb10195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:13.382 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 >> 172.21.15.73:0/4063540481 conn(0x7fcb10075c20 msgr2=0x7fcb100ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 shutdown_connections 2024-06-06T07:23:13.383 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:13.380+0000 7fcb175dc640 1 -- 172.21.15.73:0/4063540481 wait complete. 2024-06-06T07:23:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:13 smithi073 ceph-mon[32303]: pgmap v696: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 339 op/s 2024-06-06T07:23:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:13 smithi073 ceph-mon[32303]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T07:23:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:13 smithi136 ceph-mon[35901]: pgmap v696: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 339 op/s 2024-06-06T07:23:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:13 smithi136 ceph-mon[35901]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T07:23:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:13 smithi170 ceph-mon[36108]: pgmap v696: 129 pgs: 129 active+clean; 1.2 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.4 MiB/s wr, 339 op/s 2024-06-06T07:23:13.601 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:13 smithi170 ceph-mon[36108]: osdmap e140: 12 total, 12 up, 12 in 2024-06-06T07:23:13.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82490 8.84 MB/sec execute 378 sec latency 12629.812 ms 2024-06-06T07:23:14.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82654 8.84 MB/sec execute 379 sec latency 13629.842 ms 2024-06-06T07:23:15.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:15 smithi073 ceph-mon[32303]: pgmap v698: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-06-06T07:23:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:15 smithi136 ceph-mon[35901]: pgmap v698: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-06-06T07:23:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:15 smithi170 ceph-mon[36108]: pgmap v698: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.6 MiB/s wr, 305 op/s 2024-06-06T07:23:15.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82852 8.86 MB/sec execute 380 sec latency 14371.477 ms 2024-06-06T07:23:16.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 82996 8.85 MB/sec execute 381 sec latency 94.889 ms 2024-06-06T07:23:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:17 smithi073 ceph-mon[32303]: pgmap v699: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 302 op/s 2024-06-06T07:23:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:17 smithi136 ceph-mon[35901]: pgmap v699: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 302 op/s 2024-06-06T07:23:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:17 smithi170 ceph-mon[36108]: pgmap v699: 129 pgs: 129 active+clean; 1.2 GiB data, 7.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.5 MiB/s wr, 302 op/s 2024-06-06T07:23:17.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83168 8.86 MB/sec execute 382 sec latency 62.608 ms 2024-06-06T07:23:18.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83330 8.86 MB/sec execute 383 sec latency 113.997 ms 2024-06-06T07:23:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:19 smithi073 ceph-mon[32303]: pgmap v700: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-06-06T07:23:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:19 smithi136 ceph-mon[35901]: pgmap v700: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-06-06T07:23:19.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:19 smithi170 ceph-mon[36108]: pgmap v700: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.4 MiB/s wr, 327 op/s 2024-06-06T07:23:19.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83469 8.85 MB/sec execute 384 sec latency 66.206 ms 2024-06-06T07:23:20.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83624 8.83 MB/sec execute 385 sec latency 83.083 ms 2024-06-06T07:23:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:21 smithi073 ceph-mon[32303]: pgmap v701: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:21 smithi136 ceph-mon[35901]: pgmap v701: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:21.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:21 smithi170 ceph-mon[36108]: pgmap v701: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:21.821 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83784 8.84 MB/sec execute 386 sec latency 93.872 ms 2024-06-06T07:23:22.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 83950 8.83 MB/sec execute 387 sec latency 121.011 ms 2024-06-06T07:23:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:23 smithi073 ceph-mon[32303]: pgmap v702: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:23 smithi136 ceph-mon[35901]: pgmap v702: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:23 smithi170 ceph-mon[36108]: pgmap v702: 129 pgs: 129 active+clean; 1.2 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.4 MiB/s wr, 326 op/s 2024-06-06T07:23:23.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84109 8.84 MB/sec execute 388 sec latency 76.624 ms 2024-06-06T07:23:24.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84256 8.84 MB/sec execute 389 sec latency 119.137 ms 2024-06-06T07:23:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:25 smithi073 ceph-mon[32303]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.2 MiB/s wr, 428 op/s 2024-06-06T07:23:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:25 smithi136 ceph-mon[35901]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.2 MiB/s wr, 428 op/s 2024-06-06T07:23:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:25 smithi170 ceph-mon[36108]: pgmap v703: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 8.2 MiB/s wr, 428 op/s 2024-06-06T07:23:25.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84446 8.85 MB/sec execute 390 sec latency 67.955 ms 2024-06-06T07:23:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 -- 172.21.15.73:0/2772259095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106b40 msgr2=0x7f7d60106f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 --2- 172.21.15.73:0/2772259095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106b40 0x7f7d60106f20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009900 tx=0x7f7d5002a410 comp rx=0 tx=0).stop 2024-06-06T07:23:26.508 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 -- 172.21.15.73:0/2772259095 shutdown_connections 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 --2- 172.21.15.73:0/2772259095 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 0x7f7d6010d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 --2- 172.21.15.73:0/2772259095 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60101410 0x7f7d60101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 --2- 172.21.15.73:0/2772259095 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60106b40 0x7f7d60106f20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 -- 172.21.15.73:0/2772259095 >> 172.21.15.73:0/2772259095 conn(0x7f7d600769a0 msgr2=0x7f7d600fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 -- 172.21.15.73:0/2772259095 shutdown_connections 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.505+0000 7f7d68389640 1 -- 172.21.15.73:0/2772259095 wait complete. 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 Processor -- start 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 -- start start 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 0x7f7d6010e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60106b40 0x7f7d6010e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.509 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60111dd0 con 0x7f7d60101410 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60111c50 con 0x7f7d60106b40 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d68389640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d60111f50 con 0x7f7d60101dd0 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d66125640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d66125640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52464/0 (socket says 172.21.15.73:52464) 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d66125640 1 -- 172.21.15.73:0/2425653669 learned_addr learned my addr 172.21.15.73:0/2425653669 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d65924640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 0x7f7d6010e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.506+0000 7f7d66926640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60106b40 0x7f7d6010e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 -- 172.21.15.73:0/2425653669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 msgr2=0x7f7d6010e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 0x7f7d6010e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 -- 172.21.15.73:0/2425653669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60106b40 msgr2=0x7f7d6010e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.510 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60106b40 0x7f7d6010e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 -- 172.21.15.73:0/2425653669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601aabd0 con 0x7f7d60101410 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d66125640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006010 tx=0x7f7d50004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d577fe640 1 -- 172.21.15.73:0/2425653669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5003f030 con 0x7f7d60101410 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d577fe640 1 -- 172.21.15.73:0/2425653669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d500035f0 con 0x7f7d60101410 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d601aada0 con 0x7f7d60101410 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d577fe640 1 -- 172.21.15.73:0/2425653669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50035070 con 0x7f7d60101410 2024-06-06T07:23:26.511 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.507+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d601ab280 con 0x7f7d60101410 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 Processor -- start 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 -- start start 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba0e8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba0f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619ba04c4f0 con 0x5619ba0f3350 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.510+0000 7f7d6b174740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619b9fea870 con 0x5619ba0e8a90 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d6b174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619ba0484b0 con 0x5619ba0e8170 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d577fe640 1 -- 172.21.15.73:0/2425653669 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d50032280 con 0x7f7d60101410 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.513 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52466/0 (socket says 172.21.15.73:52466) 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 -- 172.21.15.73:0/2462639101 learned_addr learned my addr 172.21.15.73:0/2462639101 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d66926640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba0e8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d66125640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba0f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 -- 172.21.15.73:0/2462639101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 msgr2=0x5619ba0e8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba0e8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 -- 172.21.15.73:0/2462639101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 msgr2=0x5619ba0f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba0f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 -- 172.21.15.73:0/2462639101 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba0069b0 con 0x5619ba0f3350 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d66926640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba0e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.511+0000 7f7d65924640 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099d0 tx=0x7f7d5c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335a3a2983d8c478 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d577fe640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 0x7f7d4c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d54bfe640 1 -- 172.21.15.73:0/2462639101 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c017070 con 0x5619ba0f3350 2024-06-06T07:23:26.514 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d54bfe640 1 -- 172.21.15.73:0/2462639101 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d5c013470 con 0x5619ba0f3350 2024-06-06T07:23:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d54bfe640 1 -- 172.21.15.73:0/2462639101 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c0125f0 con 0x5619ba0f3350 2024-06-06T07:23:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d577fe640 1 -- 172.21.15.73:0/2425653669 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f7d500efd70 con 0x7f7d60101410 2024-06-06T07:23:26.515 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.512+0000 7f7d66926640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 0x7f7d4c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- 172.21.15.73:0/2462639101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 msgr2=0x5619ba0f5740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0099d0 tx=0x7f7d5c009e90 comp rx=0 tx=0).stop 2024-06-06T07:23:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d66926640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 0x7f7d4c0a52e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800a5f0 tx=0x7f7d5800d040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- 172.21.15.73:0/2462639101 shutdown_connections 2024-06-06T07:23:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f5740 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba0f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 --2- 172.21.15.73:0/2462639101 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba0e8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- 172.21.15.73:0/2462639101 >> 172.21.15.73:0/2462639101 conn(0x5619ba0e4680 msgr2=0x5619ba0e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- 172.21.15.73:0/2462639101 shutdown_connections 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- 172.21.15.73:0/2462639101 wait complete. 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 -- start start 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.513+0000 7f7d6b174740 1 Processor -- start 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8a90 0x5619ba0f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d66125640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d65924640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8a90 0x5619ba0f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d66926640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d66125640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54720/0 (socket says 172.21.15.73:54720) 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d66125640 1 -- 172.21.15.73:0/631205816 learned_addr learned my addr 172.21.15.73:0/631205816 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619ba0f5f80 con 0x5619ba0f3350 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5619b9fd5ae0 con 0x5619ba0e8170 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5619ba0f6100 con 0x5619ba0e8a90 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.514+0000 7f7d6b174740 10 client.0 _ll_register_callbacks cb 0x5619ba00e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 -- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8a90 msgr2=0x5619ba0f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8a90 0x5619ba0f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 -- 172.21.15.73:0/631205816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 msgr2=0x5619ba0f0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b9d60210 con 0x5619ba0e8170 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d66125640 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b8280 tx=0x7f7d500b8180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48d87aeb557b1a3f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:23:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049040 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c013470 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f7d500037a0 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7d500d9050 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d6b174740 10 client.26852 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.515+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5619b9e2c620 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d5c013470 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f7d5004c060 con 0x5619ba0e8170 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 10 client.26852 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 10 client.26852 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 20 client.26852 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba105900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 4 client.26852 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ba05ff10 con 0x5619ba103460 2024-06-06T07:23:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.516+0000 7f7d65924640 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba105900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.517+0000 7f7d49ffb640 10 client.26852 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:26.522 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.517+0000 7f7d65924640 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba105900 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51beb7fb39483ba6 server_cookie=be0b58fe3dd4c532 in_seq=0 out_seq=0 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d49ffb640 1 -- 172.21.15.73:0/631205816 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619ba05ff10 con 0x5619ba103460 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d49ffb640 10 client.26852 handle_command_reply: tid=0 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 client.26852 shutdown 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 msgr2=0x5619ba0f0270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b8280 tx=0x7f7d500b8180 comp rx=0 tx=0).stop 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 shutdown_connections 2024-06-06T07:23:26.523 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba105900 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0f0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d49ffb640 0 client.26852 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8a90 0x5619ba0f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 --2- 172.21.15.73:0/631205816 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0f0270 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.518+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 >> 172.21.15.73:0/631205816 conn(0x5619ba0e4680 msgr2=0x5619ba0ea3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.519+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 shutdown_connections 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.519+0000 7f7d6b174740 1 -- 172.21.15.73:0/631205816 wait complete. 2024-06-06T07:23:26.524 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.519+0000 7f7d6b174740 20 client.26852 trim_cache size 0 max 16384 2024-06-06T07:23:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.539+0000 7f7d6b174740 1 Processor -- start 2024-06-06T07:23:26.540 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 -- start start 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba10b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0ed510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b5510 con 0x5619ba0f3350 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b7050 con 0x5619ba0e8a90 2024-06-06T07:23:26.541 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d6b174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b5be0 con 0x5619ba0e8170 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66926640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0ed510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48344/0 (socket says 172.21.15.73:48344) 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 -- 172.21.15.73:0/4086317607 learned_addr learned my addr 172.21.15.73:0/4086317607 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d65924640 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba10b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 -- 172.21.15.73:0/4086317607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 msgr2=0x5619ba10b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba10b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 -- 172.21.15.73:0/4086317607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 msgr2=0x5619ba0ed510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0ed510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.540+0000 7f7d66125640 1 -- 172.21.15.73:0/4086317607 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ba0dff70 con 0x5619ba0e8170 2024-06-06T07:23:26.542 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.541+0000 7f7d66125640 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b8230 tx=0x7f7d500095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=289117b7f914b019 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.541+0000 7f7d4bfff640 1 -- 172.21.15.73:0/4086317607 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049040 con 0x5619ba0e8170 2024-06-06T07:23:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.541+0000 7f7d4bfff640 1 -- 172.21.15.73:0/4086317607 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d500bbe20 con 0x5619ba0e8170 2024-06-06T07:23:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.541+0000 7f7d4bfff640 1 -- 172.21.15.73:0/4086317607 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d500b8620 con 0x5619ba0e8170 2024-06-06T07:23:26.544 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- 172.21.15.73:0/4086317607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 msgr2=0x5619ba10b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7d500b8230 tx=0x7f7d500095a0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- 172.21.15.73:0/4086317607 shutdown_connections 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0f3350 0x5619ba0ed510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8a90 0x5619ba10b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 --2- 172.21.15.73:0/4086317607 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0e8170 0x5619ba10b210 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- 172.21.15.73:0/4086317607 >> 172.21.15.73:0/4086317607 conn(0x5619ba0e4680 msgr2=0x5619ba0e4a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- 172.21.15.73:0/4086317607 shutdown_connections 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- 172.21.15.73:0/4086317607 wait complete. 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.542+0000 7f7d6b174740 1 -- start start 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 Processor -- start 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0e8a90 0x5619ba0ed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0f3350 0x5619ba0ede90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b6980 con 0x5619ba0e8a90 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b62b0 con 0x5619ba0e8170 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d601b3510 con 0x5619ba0f3350 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d6b174740 10 client.0 _ll_register_callbacks cb 0x7f7d50005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d66125640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d65924640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0e8a90 0x5619ba0ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d66125640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:54738/0 (socket says 172.21.15.73:54738) 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d66125640 1 -- 172.21.15.73:0/3734978529 learned_addr learned my addr 172.21.15.73:0/3734978529 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.543+0000 7f7d66926640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0f3350 0x5619ba0ede90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 -- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0f3350 msgr2=0x5619ba0ede90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0f3350 0x5619ba0ede90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 -- 172.21.15.73:0/3734978529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0e8a90 msgr2=0x5619ba0ed950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0e8a90 0x5619ba0ed950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 -- 172.21.15.73:0/3734978529 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ba16d2e0 con 0x5619ba0e8170 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d66125640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ba0d0 tx=0x7f7d50003d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f71e5f63f99ac9a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:23:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50049050 con 0x5619ba0e8170 2024-06-06T07:23:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f7d500c4600 con 0x5619ba0e8170 2024-06-06T07:23:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.544+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d500bbe20 con 0x5619ba0e8170 2024-06-06T07:23:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f7d500b8600 con 0x5619ba0e8170 2024-06-06T07:23:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 10 client.26858 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5619ba16d2e0 con 0x5619ba0e8170 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d500bbe20 con 0x5619ba0e8170 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f7d500c4ce0 con 0x5619ba0e8170 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 10 client.26858 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 10 client.26858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 20 client.26858 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba170960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 4 client.26858 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.545+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619ba059c20 con 0x5619ba103460 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.546+0000 7f7d65924640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba170960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.546+0000 7f7d49ffb640 10 client.26858 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.546+0000 7f7d65924640 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba170960 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed9a3b5e9b027f7 server_cookie=97ced8b73ab2f2ce in_seq=0 out_seq=0 2024-06-06T07:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d49ffb640 1 -- 172.21.15.73:0/3734978529 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5619ba059c20 con 0x5619ba103460 2024-06-06T07:23:26.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d49ffb640 10 client.26858 handle_command_reply: tid=0 2024-06-06T07:23:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 client.26858 shutdown 2024-06-06T07:23:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 msgr2=0x5619ba0ed410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7d500ba0d0 tx=0x7f7d50003d20 comp rx=0 tx=0).stop 2024-06-06T07:23:26.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 shutdown_connections 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5619ba103460 0x5619ba170960 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5619ba0f3350 0x5619ba0ede90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619ba0e8a90 0x5619ba0ed950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d49ffb640 0 client.26858 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 --2- 172.21.15.73:0/3734978529 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5619ba0e8170 0x5619ba0ed410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.654+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 >> 172.21.15.73:0/3734978529 conn(0x5619ba0e4680 msgr2=0x5619ba0e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.655+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 shutdown_connections 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.655+0000 7f7d6b174740 1 -- 172.21.15.73:0/3734978529 wait complete. 2024-06-06T07:23:26.656 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.655+0000 7f7d6b174740 20 client.26858 trim_cache size 0 max 16384 2024-06-06T07:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 msgr2=0x7f7d4c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.657 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 0x7f7d4c0a52e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7d5800a5f0 tx=0x7f7d5800d040 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 msgr2=0x7f7d6010df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006010 tx=0x7f7d50004270 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 shutdown_connections 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d60106b40 0x7f7d6010e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d4c0a2e20 0x7f7d4c0a52e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d60101dd0 0x7f7d6010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 --2- 172.21.15.73:0/2425653669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d60101410 0x7f7d6010df70 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 >> 172.21.15.73:0/2425653669 conn(0x7f7d600769a0 msgr2=0x7f7d601036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:26.658 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 shutdown_connections 2024-06-06T07:23:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:26.656+0000 7f7d68389640 1 -- 172.21.15.73:0/2425653669 wait complete. 2024-06-06T07:23:26.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84622 8.85 MB/sec execute 391 sec latency 95.269 ms 2024-06-06T07:23:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:27 smithi073 ceph-mon[32303]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-06-06T07:23:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:27 smithi136 ceph-mon[35901]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-06-06T07:23:27.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:27 smithi170 ceph-mon[36108]: pgmap v704: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.2 MiB/s wr, 278 op/s 2024-06-06T07:23:27.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84760 8.84 MB/sec execute 392 sec latency 79.749 ms 2024-06-06T07:23:28.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 84911 8.83 MB/sec execute 393 sec latency 63.181 ms 2024-06-06T07:23:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:29 smithi073 ceph-mon[32303]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2024-06-06T07:23:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:29 smithi136 ceph-mon[35901]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2024-06-06T07:23:29.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:29 smithi170 ceph-mon[36108]: pgmap v705: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.5 MiB/s wr, 430 op/s 2024-06-06T07:23:29.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85068 8.83 MB/sec execute 394 sec latency 117.924 ms 2024-06-06T07:23:30.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85205 8.82 MB/sec execute 395 sec latency 168.881 ms 2024-06-06T07:23:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:31 smithi073 ceph-mon[32303]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-06-06T07:23:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:31 smithi136 ceph-mon[35901]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-06-06T07:23:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:31 smithi170 ceph-mon[36108]: pgmap v706: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.0 MiB/s wr, 293 op/s 2024-06-06T07:23:31.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85380 8.82 MB/sec execute 396 sec latency 73.848 ms 2024-06-06T07:23:32.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85558 8.82 MB/sec execute 397 sec latency 81.277 ms 2024-06-06T07:23:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:33 smithi073 ceph-mon[32303]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-06-06T07:23:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:33 smithi136 ceph-mon[35901]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-06-06T07:23:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:33 smithi170 ceph-mon[36108]: pgmap v707: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.8 MiB/s wr, 290 op/s 2024-06-06T07:23:33.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85725 8.82 MB/sec execute 398 sec latency 111.063 ms 2024-06-06T07:23:34.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 85898 8.82 MB/sec execute 399 sec latency 71.255 ms 2024-06-06T07:23:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:35 smithi073 ceph-mon[32303]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:23:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:35 smithi136 ceph-mon[35901]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:23:35.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:35 smithi170 ceph-mon[36108]: pgmap v708: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.3 MiB/s wr, 419 op/s 2024-06-06T07:23:35.822 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86090 8.84 MB/sec execute 400 sec latency 108.621 ms 2024-06-06T07:23:36.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86254 8.85 MB/sec execute 401 sec latency 99.588 ms 2024-06-06T07:23:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:37 smithi073 ceph-mon[32303]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 286 op/s 2024-06-06T07:23:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:37 smithi136 ceph-mon[35901]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 286 op/s 2024-06-06T07:23:37.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:37 smithi170 ceph-mon[36108]: pgmap v709: 129 pgs: 129 active+clean; 1.3 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 286 op/s 2024-06-06T07:23:37.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86406 8.85 MB/sec execute 402 sec latency 135.908 ms 2024-06-06T07:23:38.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86574 8.85 MB/sec execute 403 sec latency 89.921 ms 2024-06-06T07:23:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:39 smithi073 ceph-mon[32303]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 414 op/s 2024-06-06T07:23:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:39 smithi136 ceph-mon[35901]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 414 op/s 2024-06-06T07:23:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:39 smithi170 ceph-mon[36108]: pgmap v710: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 414 op/s 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 -- 172.21.15.73:0/569010601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094b20 msgr2=0x7f9bcc0a0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 --2- 172.21.15.73:0/569010601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094b20 0x7f9bcc0a0df0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4050cb0 tx=0x7f9bd4097730 comp rx=0 tx=0).stop 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 -- 172.21.15.73:0/569010601 shutdown_connections 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 --2- 172.21.15.73:0/569010601 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094b20 0x7f9bcc0a0df0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 --2- 172.21.15.73:0/569010601 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094180 0x7f9bcc0945e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 --2- 172.21.15.73:0/569010601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 0x7f9bcc0992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 -- 172.21.15.73:0/569010601 >> 172.21.15.73:0/569010601 conn(0x7f9bcc009a70 msgr2=0x7f9bcc009e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 -- 172.21.15.73:0/569010601 shutdown_connections 2024-06-06T07:23:39.811 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.810+0000 7f9bd359e640 1 -- 172.21.15.73:0/569010601 wait complete. 2024-06-06T07:23:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.811+0000 7f9bd359e640 1 Processor -- start 2024-06-06T07:23:39.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.811+0000 7f9bd359e640 1 -- start start 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.812+0000 7f9bd359e640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.812+0000 7f9bd359e640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 0x7f9bcc12eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.812+0000 7f9bd359e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 0x7f9bcc132e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.812+0000 7f9bd359e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0a3990 con 0x7f9bcc098ef0 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.812+0000 7f9bd359e640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0a3810 con 0x7f9bcc094180 2024-06-06T07:23:39.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.813+0000 7f9bd359e640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9bcc0a3b10 con 0x7f9bcc094b20 2024-06-06T07:23:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.814+0000 7f9bd2d9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 0x7f9bcc132e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.814+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.814+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:33008/0 (socket says 172.21.15.73:33008) 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.814+0000 7f9bd259c640 1 -- 172.21.15.73:0/2554079518 learned_addr learned my addr 172.21.15.73:0/2554079518 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 0x7f9bcc12eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 -- 172.21.15.73:0/2554079518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 msgr2=0x7f9bcc12eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 0x7f9bcc12eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 -- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 msgr2=0x7f9bcc132e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 0x7f9bcc132e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 -- 172.21.15.73:0/2554079518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bcc1335b0 con 0x7f9bcc094180 2024-06-06T07:23:39.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 0x7f9bcc12eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d950 tx=0x7f9bc800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.815+0000 7f9bc37fe640 1 -- 172.21.15.73:0/2554079518 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8017030 con 0x7f9bcc094180 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.816+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bcc1338a0 con 0x7f9bcc094180 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.816+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bcc13b130 con 0x7f9bcc094180 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.816+0000 7f9bc37fe640 1 -- 172.21.15.73:0/2554079518 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc8004020 con 0x7f9bcc094180 2024-06-06T07:23:39.817 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.816+0000 7f9bc37fe640 1 -- 172.21.15.73:0/2554079518 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc8005640 con 0x7f9bcc094180 2024-06-06T07:23:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.820+0000 7f9bdbbf4740 1 Processor -- start 2024-06-06T07:23:39.821 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.821+0000 7f9bc37fe640 1 -- 172.21.15.73:0/2554079518 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f9bc8009a30 con 0x7f9bcc094180 2024-06-06T07:23:39.822 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.822+0000 7f9bc37fe640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 0x7f9bbc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.822+0000 7f9bc37fe640 1 -- 172.21.15.73:0/2554079518 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f9bc80cf330 con 0x7f9bcc094180 2024-06-06T07:23:39.823 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.822+0000 7f9bdbbf4740 1 -- start start 2024-06-06T07:23:39.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86711 8.84 MB/sec execute 404 sec latency 163.039 ms 2024-06-06T07:23:39.824 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 0x7f9bbc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945163490 0x56294516d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x562945170140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5629450be4f0 con 0x562945162b70 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56294505c870 con 0x562945163490 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5629450ba4b0 con 0x56294516dd50 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37964/0 (socket says 172.21.15.73:37964) 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 -- 172.21.15.73:0/1698733666 learned_addr learned my addr 172.21.15.73:0/1698733666 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945163490 0x56294516d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x562945170140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 -- 172.21.15.73:0/1698733666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 msgr2=0x562945170140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.825 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x562945170140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 -- 172.21.15.73:0/1698733666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945163490 msgr2=0x56294516d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945163490 0x56294516d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.823+0000 7f9bd259c640 1 -- 172.21.15.73:0/1698733666 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629450789b0 con 0x562945162b70 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.824+0000 7f9bd259c640 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3b20 tx=0x7f9bc809bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16ce9eeed30e9722 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.824+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 0x7f9bbc0a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000b60 tx=0x7f9bd40697e0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.824+0000 7f9bc0ff9640 1 -- 172.21.15.73:0/1698733666 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc809f790 con 0x562945162b70 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.824+0000 7f9bc0ff9640 1 -- 172.21.15.73:0/1698733666 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc80a4050 con 0x562945162b70 2024-06-06T07:23:39.826 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.824+0000 7f9bc0ff9640 1 -- 172.21.15.73:0/1698733666 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80cc890 con 0x562945162b70 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/1698733666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 msgr2=0x562945162f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3b20 tx=0x7f9bc809bae0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/1698733666 shutdown_connections 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x562945170140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945163490 0x56294516d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/1698733666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x562945162f50 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/1698733666 >> 172.21.15.73:0/1698733666 conn(0x56294515f080 msgr2=0x56294515f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:39.829 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/1698733666 shutdown_connections 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/1698733666 wait complete. 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 -- start start 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.825+0000 7f9bdbbf4740 1 Processor -- start 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x56294516acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x56294516b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562945170970 con 0x562945163490 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x562945047ae0 con 0x562945162b70 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x562945170af0 con 0x56294516dd50 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bdbbf4740 10 client.0 _ll_register_callbacks cb 0x5629450809b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.830 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37966/0 (socket says 172.21.15.73:37966) 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 -- 172.21.15.73:0/3623664196 learned_addr learned my addr 172.21.15.73:0/3623664196 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x56294516acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x56294516b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 -- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 msgr2=0x56294516b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x56294516b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 -- 172.21.15.73:0/3623664196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 msgr2=0x56294516acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x56294516acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.826+0000 7f9bd1d9b640 1 -- 172.21.15.73:0/3623664196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562944dd2210 con 0x562945163490 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4013740 tx=0x7f9bc4009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f4b7d49cd0a330 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc4005020 con 0x562945163490 2024-06-06T07:23:39.831 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f9bc40051c0 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80a4050 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f9bc4005400 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bdbbf4740 10 client.17661 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.827+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x562944e9e620 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc80a4050 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f9bc401a030 con 0x562945163490 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 10 client.17661 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 10 client.17661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 20 client.17661 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x562945180450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 4 client.17661 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629450d1f10 con 0x56294517e020 2024-06-06T07:23:39.832 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.828+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x562945180450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.829+0000 7f9bb9ffb640 10 client.17661 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.829+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x562945180450 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b55f8985d10019f8 server_cookie=f789e1679c36ee77 in_seq=0 out_seq=0 2024-06-06T07:23:39.833 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3623664196 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5629450d1f10 con 0x56294517e020 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bb9ffb640 10 client.17661 handle_command_reply: tid=0 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 client.17661 shutdown 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 msgr2=0x56294516b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f9bc4013740 tx=0x7f9bc4009f90 comp rx=0 tx=0).stop 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 shutdown_connections 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x562945180450 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56294516dd50 0x56294516b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945163490 0x56294516b230 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3623664196 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x56294516acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bb9ffb640 0 client.17661 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 >> 172.21.15.73:0/3623664196 conn(0x56294515f080 msgr2=0x562945164e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:39.834 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 shutdown_connections 2024-06-06T07:23:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3623664196 wait complete. 2024-06-06T07:23:39.835 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.830+0000 7f9bdbbf4740 20 client.17661 trim_cache size 0 max 16384 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 Processor -- start 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 -- start start 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x562945185db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 0x5629451862f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb40100c0 con 0x562945162b70 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4011c00 con 0x56294516dd50 2024-06-06T07:23:39.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4010790 con 0x562945163490 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37976/0 (socket says 172.21.15.73:37976) 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 -- 172.21.15.73:0/2171334048 learned_addr learned my addr 172.21.15.73:0/2171334048 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 0x5629451862f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x562945185db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 -- 172.21.15.73:0/2171334048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 msgr2=0x562945185db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x562945185db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 -- 172.21.15.73:0/2171334048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 msgr2=0x5629451862f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 0x5629451862f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.851+0000 7f9bd259c640 1 -- 172.21.15.73:0/2171334048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562945150ce0 con 0x562945162b70 2024-06-06T07:23:39.853 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.852+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 0x5629451862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.852+0000 7f9bd259c640 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8099690 tx=0x7f9bc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b59462db6b9481 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.852+0000 7f9bbbfff640 1 -- 172.21.15.73:0/2171334048 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b9070 con 0x562945162b70 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.852+0000 7f9bbbfff640 1 -- 172.21.15.73:0/2171334048 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc80a4050 con 0x562945162b70 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.852+0000 7f9bbbfff640 1 -- 172.21.15.73:0/2171334048 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b4260 con 0x562945162b70 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/2171334048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 msgr2=0x56294516cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8099690 tx=0x7f9bc8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/2171334048 shutdown_connections 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56294516dd50 0x5629451862f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x562945185db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/2171334048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562945162b70 0x56294516cc30 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/2171334048 >> 172.21.15.73:0/2171334048 conn(0x56294515f080 msgr2=0x56294516c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/2171334048 shutdown_connections 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/2171334048 wait complete. 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 -- start start 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.853+0000 7f9bdbbf4740 1 Processor -- start 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x5629451720b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 0x5629451725f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:33038/0 (socket says 172.21.15.73:33038) 2024-06-06T07:23:39.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4011530 con 0x56294516dd50 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 -- 172.21.15.73:0/3431691566 learned_addr learned my addr 172.21.15.73:0/3431691566 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb4010e60 con 0x562945162b70 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x5629451720b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 0x5629451725f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9bb400e0c0 con 0x562945163490 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bdbbf4740 10 client.0 _ll_register_callbacks cb 0x7f9bc4009860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 -- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 msgr2=0x5629451720b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x5629451720b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 -- 172.21.15.73:0/3431691566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 msgr2=0x5629451725f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 0x5629451725f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd259c640 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629451e7e00 con 0x562945162b70 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x5629451720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:39.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.854+0000 7f9bd2d9d640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 0x5629451725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bd259c640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3b20 tx=0x7f9bc80996e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee024fc0fb7e9710 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80b9040 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f9bc80041d0 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9bc80a4050 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f9bc80cc890 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bdbbf4740 10 client.26876 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5629451e90b0 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.855+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9bc80cca30 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f9bc80a6080 con 0x562945162b70 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 10 client.26876 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 10 client.26876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:39.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 20 client.26876 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x5629451eb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 4 client.26876 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9bc80a4050 con 0x56294517e020 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.856+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x5629451eb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.857+0000 7f9bb9ffb640 10 client.26876 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:39.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:39.857+0000 7f9bd1d9b640 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x5629451eb2f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb00246120d271f server_cookie=f74aefd5d467a5a8 in_seq=0 out_seq=0 2024-06-06T07:23:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bb9ffb640 1 -- 172.21.15.73:0/3431691566 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9bc80a4050 con 0x56294517e020 2024-06-06T07:23:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bb9ffb640 10 client.26876 handle_command_reply: tid=0 2024-06-06T07:23:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 client.26876 shutdown 2024-06-06T07:23:40.010 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 msgr2=0x562945171b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f9bc80a3b20 tx=0x7f9bc80996e0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 shutdown_connections 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56294517e020 0x5629451eb2f0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bb9ffb640 0 client.26876 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56294516dd50 0x5629451725f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x562945163490 0x5629451720b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 --2- 172.21.15.73:0/3431691566 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x562945162b70 0x562945171b70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 >> 172.21.15.73:0/3431691566 conn(0x56294515f080 msgr2=0x562945164db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 shutdown_connections 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 1 -- 172.21.15.73:0/3431691566 wait complete. 2024-06-06T07:23:40.011 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.009+0000 7f9bdbbf4740 20 client.26876 trim_cache size 0 max 16384 2024-06-06T07:23:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:23:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:23:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:23:40.012 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 msgr2=0x7f9bbc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 0x7f9bbc0a5460 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9bd4000b60 tx=0x7f9bd40697e0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 msgr2=0x7f9bcc12e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d950 tx=0x7f9bc800de10 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 shutdown_connections 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9bcc098ef0 0x7f9bcc132e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f9bbc0a2fa0 0x7f9bbc0a5460 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9bcc094b20 0x7f9bcc12eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 --2- 172.21.15.73:0/2554079518 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f9bcc094180 0x7f9bcc12e5c0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 >> 172.21.15.73:0/2554079518 conn(0x7f9bcc009a70 msgr2=0x7f9bcc092080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 shutdown_connections 2024-06-06T07:23:40.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:40.010+0000 7f9bd359e640 1 -- 172.21.15.73:0/2554079518 wait complete. 2024-06-06T07:23:40.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86846 8.83 MB/sec execute 405 sec latency 108.863 ms 2024-06-06T07:23:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:41 smithi073 ceph-mon[32303]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 265 op/s 2024-06-06T07:23:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:41 smithi136 ceph-mon[35901]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 265 op/s 2024-06-06T07:23:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:41 smithi170 ceph-mon[36108]: pgmap v711: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.3 MiB/s wr, 265 op/s 2024-06-06T07:23:41.823 INFO:tasks.workunit.client.0.smithi073.stdout: 10 86997 8.83 MB/sec execute 406 sec latency 247.919 ms 2024-06-06T07:23:42.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87113 8.82 MB/sec execute 407 sec latency 281.939 ms 2024-06-06T07:23:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:43 smithi073 ceph-mon[32303]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-06-06T07:23:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:43 smithi136 ceph-mon[35901]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-06-06T07:23:43.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:43 smithi170 ceph-mon[36108]: pgmap v712: 129 pgs: 129 active+clean; 1.3 GiB data, 7.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.2 MiB/s wr, 259 op/s 2024-06-06T07:23:43.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87250 8.81 MB/sec execute 408 sec latency 133.148 ms 2024-06-06T07:23:44.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87453 8.83 MB/sec execute 409 sec latency 82.544 ms 2024-06-06T07:23:45.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:45 smithi073 ceph-mon[32303]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.5 MiB/s wr, 393 op/s 2024-06-06T07:23:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:45 smithi136 ceph-mon[35901]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.5 MiB/s wr, 393 op/s 2024-06-06T07:23:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:45 smithi170 ceph-mon[36108]: pgmap v713: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.5 MiB/s wr, 393 op/s 2024-06-06T07:23:45.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87631 8.83 MB/sec execute 410 sec latency 78.272 ms 2024-06-06T07:23:46.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87775 8.82 MB/sec execute 411 sec latency 82.512 ms 2024-06-06T07:23:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:47 smithi073 ceph-mon[32303]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-06-06T07:23:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:47 smithi136 ceph-mon[35901]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-06-06T07:23:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:47 smithi170 ceph-mon[36108]: pgmap v714: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.0 MiB/s wr, 264 op/s 2024-06-06T07:23:47.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 87903 8.82 MB/sec execute 412 sec latency 92.931 ms 2024-06-06T07:23:48.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88088 8.83 MB/sec execute 413 sec latency 72.379 ms 2024-06-06T07:23:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:49 smithi073 ceph-mon[32303]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 384 op/s 2024-06-06T07:23:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:49 smithi136 ceph-mon[35901]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 384 op/s 2024-06-06T07:23:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:49 smithi170 ceph-mon[36108]: pgmap v715: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.3 MiB/s wr, 384 op/s 2024-06-06T07:23:49.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88250 8.83 MB/sec execute 414 sec latency 98.415 ms 2024-06-06T07:23:50.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88399 8.82 MB/sec execute 415 sec latency 148.714 ms 2024-06-06T07:23:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:51 smithi073 ceph-mon[32303]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-06-06T07:23:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:51 smithi136 ceph-mon[35901]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-06-06T07:23:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:51 smithi170 ceph-mon[36108]: pgmap v716: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 276 op/s 2024-06-06T07:23:51.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88535 8.81 MB/sec execute 416 sec latency 73.480 ms 2024-06-06T07:23:52.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88697 8.82 MB/sec execute 417 sec latency 93.614 ms 2024-06-06T07:23:53.141 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.140+0000 7f11adc5d640 1 -- 172.21.15.73:0/1843014479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81008d0 msgr2=0x7f11a8100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.140+0000 7f11adc5d640 1 --2- 172.21.15.73:0/1843014479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81008d0 0x7f11a8100d50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f119c009930 tx=0x7f119c02a420 comp rx=0 tx=0).stop 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 -- 172.21.15.73:0/1843014479 shutdown_connections 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 --2- 172.21.15.73:0/1843014479 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a8101290 0x7f11a810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 --2- 172.21.15.73:0/1843014479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a81008d0 0x7f11a8100d50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 --2- 172.21.15.73:0/1843014479 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8106890 0x7f11a8106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.142 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 -- 172.21.15.73:0/1843014479 >> 172.21.15.73:0/1843014479 conn(0x7f11a8077a20 msgr2=0x7f11a8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 -- 172.21.15.73:0/1843014479 shutdown_connections 2024-06-06T07:23:53.143 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.141+0000 7f11adc5d640 1 -- 172.21.15.73:0/1843014479 wait complete. 2024-06-06T07:23:53.144 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 Processor -- start 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 -- start start 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8101290 0x7f11a8198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a8106890 0x7f11a819c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8110ba0 con 0x7f11a8106890 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8110a20 con 0x7f11a8101290 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11adc5d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f11a8110d20 con 0x7f11a81008d0 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a8106890 0x7f11a819c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48242/0 (socket says 172.21.15.73:48242) 2024-06-06T07:23:53.145 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11a77fe640 1 -- 172.21.15.73:0/3514210516 learned_addr learned my addr 172.21.15.73:0/3514210516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.142+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8101290 0x7f11a8198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 -- 172.21.15.73:0/3514210516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8101290 msgr2=0x7f11a8198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8101290 0x7f11a8198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 -- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a8106890 msgr2=0x7f11a819c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a8106890 0x7f11a819c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 -- 172.21.15.73:0/3514210516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a819d110 con 0x7f11a81008d0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a77fe640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f119800e9f0 tx=0x7f119800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a4ff9640 1 -- 172.21.15.73:0/3514210516 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119800cd80 con 0x7f11a81008d0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a819d400 con 0x7f11a81008d0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81a4c90 con 0x7f11a81008d0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a4ff9640 1 -- 172.21.15.73:0/3514210516 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980108e0 con 0x7f11a81008d0 2024-06-06T07:23:53.146 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.143+0000 7f11a4ff9640 1 -- 172.21.15.73:0/3514210516 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198018920 con 0x7f11a81008d0 2024-06-06T07:23:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.147+0000 7f11a4ff9640 1 -- 172.21.15.73:0/3514210516 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f1198010430 con 0x7f11a81008d0 2024-06-06T07:23:53.148 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.147+0000 7f11b096f740 1 Processor -- start 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 -- start start 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 0x5601b3c95550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 0x5601b3ca2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5601b3bf94f0 con 0x5601b3ca0350 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601b3b97870 con 0x5601b3c95a90 2024-06-06T07:23:53.149 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11b096f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5601b3bf54b0 con 0x5601b3c95170 2024-06-06T07:23:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 0x5601b3c95550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 0x5601b3ca2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 0x5601b3c95550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48258/0 (socket says 172.21.15.73:48258) 2024-06-06T07:23:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a7fff640 1 -- 172.21.15.73:0/4039652801 learned_addr learned my addr 172.21.15.73:0/4039652801 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:53.152 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 -- 172.21.15.73:0/4039652801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 msgr2=0x5601b3c95550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 0x5601b3c95550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 -- 172.21.15.73:0/4039652801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 msgr2=0x5601b3c9fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.148+0000 7f11a6ffd640 1 -- 172.21.15.73:0/4039652801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b3bb39b0 con 0x5601b3ca0350 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 0x5601b3ca2740 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f119c02a940 tx=0x7f119c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7593fe0b37c912ad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f118dffb640 1 -- 172.21.15.73:0/4039652801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035040 con 0x5601b3ca0350 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f11a4ff9640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 0x7f11900a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f118dffb640 1 -- 172.21.15.73:0/4039652801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c0054f0 con 0x5601b3ca0350 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f118dffb640 1 -- 172.21.15.73:0/4039652801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c0035f0 con 0x5601b3ca0350 2024-06-06T07:23:53.153 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f11a7fff640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 0x7f11900a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.149+0000 7f11a4ff9640 1 -- 172.21.15.73:0/3514210516 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f11980cfa50 con 0x7f11a81008d0 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- 172.21.15.73:0/4039652801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 msgr2=0x5601b3ca2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11a7fff640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 0x7f11900a52e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f11a00116b0 tx=0x7f11a0009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 0x5601b3ca2740 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f119c02a940 tx=0x7f119c0040c0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- 172.21.15.73:0/4039652801 shutdown_connections 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3ca0350 0x5601b3ca2740 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 --2- 172.21.15.73:0/4039652801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95170 0x5601b3c95550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- 172.21.15.73:0/4039652801 >> 172.21.15.73:0/4039652801 conn(0x5601b3c91680 msgr2=0x5601b3c91a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- 172.21.15.73:0/4039652801 shutdown_connections 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- 172.21.15.73:0/4039652801 wait complete. 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.150+0000 7f11b096f740 1 -- start start 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 Processor -- start 2024-06-06T07:23:53.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36976/0 (socket says 172.21.15.73:36976) 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3c9dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 -- 172.21.15.73:0/2123555265 learned_addr learned my addr 172.21.15.73:0/2123555265 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5601b3ca7330 con 0x5601b3c95170 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5601b3b82ae0 con 0x5601b3c95a90 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5601b3ca74b0 con 0x5601b3ca0350 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11b096f740 10 client.0 _ll_register_callbacks cb 0x5601b3bbb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a7fff640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3c9dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 -- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 msgr2=0x5601b3c9dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3c9dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 -- 172.21.15.73:0/2123555265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 msgr2=0x5601b3c9d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a77fe640 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b390d210 con 0x5601b3c95170 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.151+0000 7f11a7fff640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3c9dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f11a77fe640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f119800c6a0 tx=0x7f119809b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5635fb008b0d2eac server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.156 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:23:53.158 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11980cd1c0 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f119809b200 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119c0054f0 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f119809feb0 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f11b096f740 10 client.17679 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5601b39d9620 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.152+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f119c0054f0 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.153+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f119809aa70 con 0x5601b3c95170 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.153+0000 7f11b096f740 10 client.17679 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.153+0000 7f11b096f740 10 client.17679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.153+0000 7f11b096f740 20 client.17679 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.153+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3cb4840 0x5601b3cb6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.154+0000 7f11b096f740 4 client.17679 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.154+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601b3c0cf10 con 0x5601b3cb4840 2024-06-06T07:23:53.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.154+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3cb4840 0x5601b3cb6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.154+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3cb4840 0x5601b3cb6cb0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfd78c4401184a8 server_cookie=b8705a02ce8c583f in_seq=0 out_seq=0 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.154+0000 7f1186ffd640 10 client.17679 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.155+0000 7f1186ffd640 1 -- 172.21.15.73:0/2123555265 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5601b3c0cf10 con 0x5601b3cb4840 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.155+0000 7f1186ffd640 10 client.17679 handle_command_reply: tid=0 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.155+0000 7f11b096f740 1 client.17679 shutdown 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.156+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 msgr2=0x5601b3c9d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.156+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f119800c6a0 tx=0x7f119809b7e0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 shutdown_connections 2024-06-06T07:23:53.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3cb4840 0x5601b3cb6cb0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3c9dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95a90 0x5601b3c9d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 --2- 172.21.15.73:0/2123555265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c9d210 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 >> 172.21.15.73:0/2123555265 conn(0x5601b3c91680 msgr2=0x5601b3c973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f1186ffd640 0 client.17679 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 shutdown_connections 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 1 -- 172.21.15.73:0/2123555265 wait complete. 2024-06-06T07:23:53.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.157+0000 7f11b096f740 20 client.17679 trim_cache size 0 max 16384 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 Processor -- start 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 -- start start 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 0x5601b3cbc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3ca0350 0x5601b3cbcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1194010080 con 0x5601b3c95170 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1194011bc0 con 0x5601b3ca0350 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.176+0000 7f11b096f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f1194010750 con 0x5601b3c95a90 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 0x5601b3cbc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36982/0 (socket says 172.21.15.73:36982) 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 0x5601b3cbc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48276/0 (socket says 172.21.15.73:48276) 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 -- 172.21.15.73:0/4210318759 learned_addr learned my addr 172.21.15.73:0/4210318759 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a7fff640 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3ca0350 0x5601b3cbcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 -- 172.21.15.73:0/4210318759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 msgr2=0x5601b3cbc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 0x5601b3cbc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 -- 172.21.15.73:0/4210318759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3ca0350 msgr2=0x5601b3cbcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3ca0350 0x5601b3cbcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 -- 172.21.15.73:0/4210318759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601b3c92250 con 0x5601b3c95170 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f11a77fe640 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1198099e60 tx=0x7f119809ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdba7be9e554768 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f118d7fa640 1 -- 172.21.15.73:0/4210318759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198010720 con 0x5601b3c95170 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.177+0000 7f118d7fa640 1 -- 172.21.15.73:0/4210318759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980a4770 con 0x5601b3c95170 2024-06-06T07:23:53.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f118d7fa640 1 -- 172.21.15.73:0/4210318759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198018ac0 con 0x5601b3c95170 2024-06-06T07:23:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 -- 172.21.15.73:0/4210318759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 msgr2=0x5601b3c97230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1198099e60 tx=0x7f119809ba90 comp rx=0 tx=0).stop 2024-06-06T07:23:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 -- 172.21.15.73:0/4210318759 shutdown_connections 2024-06-06T07:23:53.183 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3ca0350 0x5601b3cbcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3c95a90 0x5601b3cbc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 --2- 172.21.15.73:0/4210318759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95170 0x5601b3c97230 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 -- 172.21.15.73:0/4210318759 >> 172.21.15.73:0/4210318759 conn(0x5601b3c91680 msgr2=0x5601b3c96b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.178+0000 7f11b096f740 1 -- 172.21.15.73:0/4210318759 shutdown_connections 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 -- 172.21.15.73:0/4210318759 wait complete. 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 -- start start 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 Processor -- start 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 0x5601b3cacb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3ca8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f11940114f0 con 0x5601b3c95a90 2024-06-06T07:23:53.184 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f1194010e20 con 0x5601b3c95170 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f119400e080 con 0x5601b3ca0350 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11b096f740 10 client.0 _ll_register_callbacks cb 0x7f119800cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3ca8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36990/0 (socket says 172.21.15.73:36990) 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3ca8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:48288/0 (socket says 172.21.15.73:48288) 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a6ffd640 1 -- 172.21.15.73:0/796122855 learned_addr learned my addr 172.21.15.73:0/796122855 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.179+0000 7f11a77fe640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 0x5601b3cacb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 -- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 msgr2=0x5601b3ca8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3ca8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.185 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 -- 172.21.15.73:0/796122855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 msgr2=0x5601b3cacb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 0x5601b3cacb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 -- 172.21.15.73:0/796122855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601b3ca8980 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a77fe640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 0x5601b3cacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11a6ffd640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f119c009930 tx=0x7f119c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a189106a5507af86 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f1186ffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119c035040 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(140..140 src has 1..140) ==== 9554+0+0 (secure 0 0 0) 0x7f119c003a00 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11980a4770 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7f119c003c00 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11b096f740 10 client.17685 fetch_fsmap learned FSMap version 80 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.180+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x5601b3d1f300 con 0x5601b3c95a90 2024-06-06T07:23:53.186 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f11980a4770 con 0x5601b3c95a90 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7f119c03c440 con 0x5601b3c95a90 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 10 client.17685 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 10 client.17685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 20 client.17685 populate_metadata read hostname 'smithi073' 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3d1f580 0x5601b3d21970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 4 client.17685 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601b3c06c20 con 0x5601b3d1f580 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11a77fe640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3d1f580 0x5601b3d21970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.181+0000 7f11a77fe640 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3d1f580 0x5601b3d21970 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edbe7028ac74eb35 server_cookie=9a16f120c8080024 in_seq=0 out_seq=0 2024-06-06T07:23:53.187 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.182+0000 7f1186ffd640 10 client.17685 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.309+0000 7f1186ffd640 1 -- 172.21.15.73:0/796122855 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5601b3c06c20 con 0x5601b3d1f580 2024-06-06T07:23:53.310 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.309+0000 7f1186ffd640 10 client.17685 handle_command_reply: tid=0 2024-06-06T07:23:53.311 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.310+0000 7f11b096f740 1 client.17685 shutdown 2024-06-06T07:23:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 msgr2=0x5601b3ca7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f119c009930 tx=0x7f119c005a20 comp rx=0 tx=0).stop 2024-06-06T07:23:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 shutdown_connections 2024-06-06T07:23:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5601b3d1f580 0x5601b3d21970 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5601b3ca0350 0x5601b3ca8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5601b3c95a90 0x5601b3ca7f00 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 --2- 172.21.15.73:0/796122855 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5601b3c95170 0x5601b3cacb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f1186ffd640 0 client.17685 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 >> 172.21.15.73:0/796122855 conn(0x5601b3c91680 msgr2=0x5601b3c973b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 shutdown_connections 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 1 -- 172.21.15.73:0/796122855 wait complete. 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.313+0000 7f11b096f740 20 client.17685 trim_cache size 0 max 16384 2024-06-06T07:23:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:23:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:23:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:23:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.314+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 msgr2=0x7f11900a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.314+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 0x7f11900a52e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f11a00116b0 tx=0x7f11a0009500 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.314+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 msgr2=0x7f11a8198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.314+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f119800e9f0 tx=0x7f119800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 shutdown_connections 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11a8106890 0x7f11a819c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f11a8101290 0x7f11a8198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11900a2e20 0x7f11900a52e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 --2- 172.21.15.73:0/3514210516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f11a81008d0 0x7f11a8198120 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:23:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.315+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 >> 172.21.15.73:0/3514210516 conn(0x7f11a8077a20 msgr2=0x7f11a80ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:23:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.317+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 shutdown_connections 2024-06-06T07:23:53.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:23:53.317+0000 7f11adc5d640 1 -- 172.21.15.73:0/3514210516 wait complete. 2024-06-06T07:23:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:53 smithi073 ceph-mon[32303]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-06-06T07:23:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:53 smithi136 ceph-mon[35901]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-06-06T07:23:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:53 smithi170 ceph-mon[36108]: pgmap v717: 129 pgs: 129 active+clean; 1.3 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.9 MiB/s wr, 273 op/s 2024-06-06T07:23:53.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88853 8.81 MB/sec execute 418 sec latency 100.807 ms 2024-06-06T07:23:54.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 88996 8.81 MB/sec execute 419 sec latency 95.795 ms 2024-06-06T07:23:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:55 smithi073 ceph-mon[32303]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 411 op/s 2024-06-06T07:23:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:55 smithi136 ceph-mon[35901]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 411 op/s 2024-06-06T07:23:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:55 smithi170 ceph-mon[36108]: pgmap v718: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.2 MiB/s wr, 411 op/s 2024-06-06T07:23:55.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89162 8.81 MB/sec execute 420 sec latency 124.479 ms 2024-06-06T07:23:56.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89369 8.83 MB/sec execute 421 sec latency 92.715 ms 2024-06-06T07:23:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:57 smithi073 ceph-mon[32303]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 277 op/s 2024-06-06T07:23:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:57 smithi136 ceph-mon[35901]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 277 op/s 2024-06-06T07:23:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:57 smithi170 ceph-mon[36108]: pgmap v719: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 4.9 MiB/s wr, 277 op/s 2024-06-06T07:23:57.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89513 8.82 MB/sec execute 422 sec latency 80.799 ms 2024-06-06T07:23:58.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89640 8.81 MB/sec execute 423 sec latency 87.497 ms 2024-06-06T07:23:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:23:59 smithi073 ceph-mon[32303]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 394 op/s 2024-06-06T07:23:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:23:59 smithi136 ceph-mon[35901]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 394 op/s 2024-06-06T07:23:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:23:59 smithi170 ceph-mon[36108]: pgmap v720: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.9 MiB/s wr, 394 op/s 2024-06-06T07:23:59.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89807 8.82 MB/sec execute 424 sec latency 81.015 ms 2024-06-06T07:24:00.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 89957 8.81 MB/sec execute 425 sec latency 521.478 ms 2024-06-06T07:24:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:01 smithi073 ceph-mon[32303]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 300 op/s 2024-06-06T07:24:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:01 smithi136 ceph-mon[35901]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 300 op/s 2024-06-06T07:24:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:01 smithi170 ceph-mon[36108]: pgmap v721: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 300 op/s 2024-06-06T07:24:01.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90111 8.81 MB/sec execute 426 sec latency 1521.511 ms 2024-06-06T07:24:02.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90238 8.80 MB/sec execute 427 sec latency 2521.542 ms 2024-06-06T07:24:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:03 smithi073 ceph-mon[32303]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-06-06T07:24:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:03 smithi136 ceph-mon[35901]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-06-06T07:24:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:03 smithi170 ceph-mon[36108]: pgmap v722: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 5.0 MiB/s wr, 281 op/s 2024-06-06T07:24:03.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90415 8.81 MB/sec execute 428 sec latency 3521.583 ms 2024-06-06T07:24:04.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90556 8.80 MB/sec execute 429 sec latency 4521.616 ms 2024-06-06T07:24:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:05 smithi073 ceph-mon[32303]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.5 MiB/s wr, 417 op/s 2024-06-06T07:24:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:05 smithi073 ceph-mon[32303]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T07:24:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:05 smithi136 ceph-mon[35901]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.5 MiB/s wr, 417 op/s 2024-06-06T07:24:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:05 smithi136 ceph-mon[35901]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T07:24:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:05 smithi170 ceph-mon[36108]: pgmap v723: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 7.5 MiB/s wr, 417 op/s 2024-06-06T07:24:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:05 smithi170 ceph-mon[36108]: osdmap e141: 12 total, 12 up, 12 in 2024-06-06T07:24:05.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90720 8.81 MB/sec execute 430 sec latency 5521.645 ms 2024-06-06T07:24:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 -- 172.21.15.73:0/2401554540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44102c30 msgr2=0x7fbd44103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/2401554540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44102c30 0x7fbd44103010 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009900 tx=0x7fbd4002a510 comp rx=0 tx=0).stop 2024-06-06T07:24:06.448 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 -- 172.21.15.73:0/2401554540 shutdown_connections 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/2401554540 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 0x7fbd4410e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/2401554540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44103550 0x7fbd4410ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/2401554540 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44102c30 0x7fbd44103010 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.444+0000 7fbd4b8da640 1 -- 172.21.15.73:0/2401554540 >> 172.21.15.73:0/2401554540 conn(0x7fbd44077a20 msgr2=0x7fbd44077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.445+0000 7fbd4b8da640 1 -- 172.21.15.73:0/2401554540 shutdown_connections 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.445+0000 7fbd4b8da640 1 -- 172.21.15.73:0/2401554540 wait complete. 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.445+0000 7fbd4b8da640 1 Processor -- start 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 -- start start 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 0x7fbd4419a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.449 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 0x7fbd4419ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44101ab0 con 0x7fbd44102c30 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44101930 con 0x7fbd4410bf70 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd4b8da640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44101c30 con 0x7fbd44103550 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd49676640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 0x7fbd4419a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36122/0 (socket says 172.21.15.73:36122) 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 -- 172.21.15.73:0/1356275475 learned_addr learned my addr 172.21.15.73:0/1356275475 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd49676640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 0x7fbd4419a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54218/0 (socket says 172.21.15.73:54218) 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd49e77640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 0x7fbd4419ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 -- 172.21.15.73:0/1356275475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 msgr2=0x7fbd4419ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.450 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 0x7fbd4419ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 -- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 msgr2=0x7fbd4419a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 0x7fbd4419a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.446+0000 7fbd48e75640 1 -- 172.21.15.73:0/1356275475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd4419f300 con 0x7fbd44103550 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd48e75640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ea60 tx=0x7fbd3400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd3a7fc640 1 -- 172.21.15.73:0/1356275475 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3400cde0 con 0x7fbd44103550 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd3a7fc640 1 -- 172.21.15.73:0/1356275475 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd340035f0 con 0x7fbd44103550 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd3a7fc640 1 -- 172.21.15.73:0/1356275475 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd34010690 con 0x7fbd44103550 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd4419f5f0 con 0x7fbd44103550 2024-06-06T07:24:06.451 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.447+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd441a6e80 con 0x7fbd44103550 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.450+0000 7fbd3a7fc640 1 -- 172.21.15.73:0/1356275475 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fbd340037a0 con 0x7fbd44103550 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 Processor -- start 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 -- start start 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc32e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd3a7fc640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 0x7fbd300a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc35740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ecdb8c4f0 con 0x563ecdc28a90 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ecdb2a870 con 0x563ecdc33350 2024-06-06T07:24:06.453 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ecdb884b0 con 0x563ecdc28170 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd48e75640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 0x7fbd300a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36128/0 (socket says 172.21.15.73:36128) 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 -- 172.21.15.73:0/4158992821 learned_addr learned my addr 172.21.15.73:0/4158992821 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc35740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49676640 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc32e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 -- 172.21.15.73:0/4158992821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 msgr2=0x563ecdc35740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc35740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.451+0000 7fbd49e77640 1 -- 172.21.15.73:0/4158992821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 msgr2=0x563ecdc32e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd49e77640 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc32e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd49e77640 1 -- 172.21.15.73:0/4158992821 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecdb469b0 con 0x563ecdc28170 2024-06-06T07:24:06.454 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd3a7fc640 1 -- 172.21.15.73:0/1356275475 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 9642+0+0 (secure 0 0 0) 0x7fbd340cfb20 con 0x7fbd44103550 2024-06-06T07:24:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd48e75640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 0x7fbd300a52e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd340985b0 tx=0x7fbd34097f70 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd49e77640 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c012a20 tx=0x7fbd3c009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=548a98355bf4bd48 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd2f7fe640 1 -- 172.21.15.73:0/4158992821 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c012110 con 0x563ecdc28170 2024-06-06T07:24:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd2f7fe640 1 -- 172.21.15.73:0/4158992821 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd3c003e20 con 0x563ecdc28170 2024-06-06T07:24:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.452+0000 7fbd2f7fe640 1 -- 172.21.15.73:0/4158992821 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c004030 con 0x563ecdc28170 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/4158992821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 msgr2=0x563ecdc28550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c012a20 tx=0x7fbd3c009a20 comp rx=0 tx=0).stop 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/4158992821 shutdown_connections 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc35740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc32e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/4158992821 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc28550 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/4158992821 >> 172.21.15.73:0/4158992821 conn(0x563ecdc24680 msgr2=0x563ecdc24a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/4158992821 shutdown_connections 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/4158992821 wait complete. 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.453+0000 7fbd4e5eb740 1 -- start start 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 Processor -- start 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28a90 0x563ecdc30890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc33350 0x563ecdc30dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563ecdc35f80 con 0x563ecdc28170 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x563ecdb15ae0 con 0x563ecdc28a90 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x563ecdc36100 con 0x563ecdc33350 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd4e5eb740 10 client.0 _ll_register_callbacks cb 0x563ecdb4e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54242/0 (socket says 172.21.15.73:54242) 2024-06-06T07:24:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd49e77640 1 -- 172.21.15.73:0/2543598443 learned_addr learned my addr 172.21.15.73:0/2543598443 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.454+0000 7fbd49676640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28a90 0x563ecdc30890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 -- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc33350 msgr2=0x563ecdc30dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc33350 0x563ecdc30dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 -- 172.21.15.73:0/2543598443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28a90 msgr2=0x563ecdc30890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28a90 0x563ecdc30890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 -- 172.21.15.73:0/2543598443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecd8a0210 con 0x563ecdc28170 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005c10 tx=0x7fbd3c010fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f723d49846294d39 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0109d0 con 0x563ecdc28170 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c003e20 con 0x563ecdc28170 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9642+0+0 (secure 0 0 0) 0x7fbd3c004000 con 0x563ecdc28170 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbd3c010b70 con 0x563ecdc28170 2024-06-06T07:24:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd4e5eb740 10 client.17697 fetch_fsmap learned FSMap version 80 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.455+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x563ecd96c620 con 0x563ecdc28170 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd3c003e20 con 0x563ecdc28170 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fbd3c026070 con 0x563ecdc28170 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 10 client.17697 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 10 client.17697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 20 client.17697 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdc433c0 0x563ecdc457b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 4 client.17697 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ecdb9ff10 con 0x563ecdc433c0 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.456+0000 7fbd49676640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdc433c0 0x563ecdc457b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.457+0000 7fbd49676640 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdc433c0 0x563ecdc457b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9175d9505a476385 server_cookie=ccfcbfa717253e07 in_seq=0 out_seq=0 2024-06-06T07:24:06.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.457+0000 7fbd2cff9640 10 client.17697 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:06.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2543598443 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ecdb9ff10 con 0x563ecdc433c0 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd2cff9640 10 client.17697 handle_command_reply: tid=0 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 client.17697 shutdown 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 msgr2=0x563ecdc30350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005c10 tx=0x7fbd3c010fb0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 shutdown_connections 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdc433c0 0x563ecdc457b0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc33350 0x563ecdc30dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28a90 0x563ecdc30890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd2cff9640 0 client.17697 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2543598443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28170 0x563ecdc30350 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 >> 172.21.15.73:0/2543598443 conn(0x563ecdc24680 msgr2=0x563ecdc2a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 shutdown_connections 2024-06-06T07:24:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2543598443 wait complete. 2024-06-06T07:24:06.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.458+0000 7fbd4e5eb740 20 client.17697 trim_cache size 0 max 16384 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.481+0000 7fbd4e5eb740 1 Processor -- start 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.481+0000 7fbd4e5eb740 1 -- start start 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.481+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28170 0x563ecdc2fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.481+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 0x563ecdc4aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 0x563ecdc4b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441b1280 con 0x563ecdc33350 2024-06-06T07:24:06.483 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441b2dc0 con 0x563ecdc28170 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd4e5eb740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441b1950 con 0x563ecdc28a90 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 0x563ecdc4aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28170 0x563ecdc2fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49676640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 0x563ecdc4aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36152/0 (socket says 172.21.15.73:36152) 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 0x563ecdc4b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49676640 1 -- 172.21.15.73:0/3749809935 learned_addr learned my addr 172.21.15.73:0/3749809935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 -- 172.21.15.73:0/3749809935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 msgr2=0x563ecdc4aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 0x563ecdc4aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 -- 172.21.15.73:0/3749809935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28170 msgr2=0x563ecdc2fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28170 0x563ecdc2fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 -- 172.21.15.73:0/3749809935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ecdc288c0 con 0x563ecdc33350 2024-06-06T07:24:06.484 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd49e77640 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 0x563ecdc4b3e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005cc0 tx=0x7fbd3c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90454377258dc4ed server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.482+0000 7fbd2effd640 1 -- 172.21.15.73:0/3749809935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0094c0 con 0x563ecdc33350 2024-06-06T07:24:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd2effd640 1 -- 172.21.15.73:0/3749809935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd3c016040 con 0x563ecdc33350 2024-06-06T07:24:06.485 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd2effd640 1 -- 172.21.15.73:0/3749809935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c005570 con 0x563ecdc33350 2024-06-06T07:24:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/3749809935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 msgr2=0x563ecdc4b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 0x563ecdc4b3e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c005cc0 tx=0x7fbd3c000f40 comp rx=0 tx=0).stop 2024-06-06T07:24:06.488 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/3749809935 shutdown_connections 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc33350 0x563ecdc4b3e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28a90 0x563ecdc4aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.483+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/3749809935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc28170 0x563ecdc2fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.484+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/3749809935 >> 172.21.15.73:0/3749809935 conn(0x563ecdc24680 msgr2=0x563ecdc32410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.484+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/3749809935 shutdown_connections 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.484+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/3749809935 wait complete. 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.484+0000 7fbd4e5eb740 1 -- start start 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.484+0000 7fbd4e5eb740 1 Processor -- start 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc36310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc36850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.489 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49e77640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc3bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:36160/0 (socket says 172.21.15.73:36160) 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc36310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49e77640 1 -- 172.21.15.73:0/2182114946 learned_addr learned my addr 172.21.15.73:0/2182114946 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441b26f0 con 0x563ecdc28a90 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441b2020 con 0x563ecdc33350 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fbd441af280 con 0x563ecdc28170 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc36850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd4e5eb740 10 client.0 _ll_register_callbacks cb 0x7fbd3c0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 -- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 msgr2=0x563ecdc3bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc3bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 -- 172.21.15.73:0/2182114946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 msgr2=0x563ecdc36850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc36850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.490 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.485+0000 7fbd49676640 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ecdc37050 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd49676640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc36310 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd40031440 tx=0x7fbd40032ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9003b4b8068ec474 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd2cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40035040 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(141..141 src has 1..141) ==== 9642+0+0 (secure 0 0 0) 0x7fbd40005040 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd3c016040 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7fbd40055050 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd4e5eb740 10 client.17703 fetch_fsmap learned FSMap version 80 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.486+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=4+}) -- 0x563ecdcace30 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbd3c016040 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 80) ==== 8378+0+0 (secure 0 0 0) 0x7fbd400035f0 con 0x563ecdc28a90 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 10 client.17703 fetch_fsmap finished waiting for FSMap version 80 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:06.491 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 20 client.17703 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdcad600 0x563ecdcafa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 4 client.17703 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563ecdb99c20 con 0x563ecdcad600 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.487+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdcad600 0x563ecdcafa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.488+0000 7fbd2cff9640 10 client.17703 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:06.492 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.488+0000 7fbd49e77640 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdcad600 0x563ecdcafa20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c08e94d09f62541 server_cookie=71ddbeb3e2b0f913 in_seq=0 out_seq=0 2024-06-06T07:24:06.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd2cff9640 1 -- 172.21.15.73:0/2182114946 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563ecdb99c20 con 0x563ecdcad600 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd2cff9640 10 client.17703 handle_command_reply: tid=0 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 client.17703 shutdown 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 msgr2=0x563ecdc36310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc36310 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fbd40031440 tx=0x7fbd40032ef0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 shutdown_connections 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x563ecdcad600 0x563ecdcafa20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x563ecdc33350 0x563ecdc36850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.573+0000 7fbd2cff9640 0 client.17703 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563ecdc28a90 0x563ecdc36310 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 1 --2- 172.21.15.73:0/2182114946 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x563ecdc28170 0x563ecdc3bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 >> 172.21.15.73:0/2182114946 conn(0x563ecdc24680 msgr2=0x563ecdc29a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 shutdown_connections 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 1 -- 172.21.15.73:0/2182114946 wait complete. 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.574+0000 7fbd4e5eb740 20 client.17703 trim_cache size 0 max 16384 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:24:06.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:24:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 msgr2=0x7fbd300a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 0x7fbd300a52e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbd340985b0 tx=0x7fbd34097f70 comp rx=0 tx=0).stop 2024-06-06T07:24:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 msgr2=0x7fbd4419a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:06.577 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fbd3400ea60 tx=0x7fbd3400ef20 comp rx=0 tx=0).stop 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 shutdown_connections 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fbd4410bf70 0x7fbd4419ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fbd300a2e20 0x7fbd300a52e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fbd44103550 0x7fbd4419a850 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 --2- 172.21.15.73:0/1356275475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd44102c30 0x7fbd4419a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.575+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 >> 172.21.15.73:0/1356275475 conn(0x7fbd44077a20 msgr2=0x7fbd441039d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:06.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.578+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 shutdown_connections 2024-06-06T07:24:06.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:06.580+0000 7fbd4b8da640 1 -- 172.21.15.73:0/1356275475 wait complete. 2024-06-06T07:24:06.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 90851 8.80 MB/sec execute 431 sec latency 6521.678 ms 2024-06-06T07:24:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:07 smithi073 ceph-mon[32303]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 335 op/s 2024-06-06T07:24:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:07 smithi136 ceph-mon[35901]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 335 op/s 2024-06-06T07:24:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:07 smithi170 ceph-mon[36108]: pgmap v725: 129 pgs: 129 active+clean; 1.3 GiB data, 7.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 6.2 MiB/s wr, 335 op/s 2024-06-06T07:24:07.826 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91023 8.81 MB/sec execute 432 sec latency 7521.716 ms 2024-06-06T07:24:08.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91178 8.80 MB/sec execute 433 sec latency 8521.745 ms 2024-06-06T07:24:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:09 smithi073 ceph-mon[32303]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 348 op/s 2024-06-06T07:24:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:09 smithi136 ceph-mon[35901]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 348 op/s 2024-06-06T07:24:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:09 smithi170 ceph-mon[36108]: pgmap v726: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.4 MiB/s wr, 348 op/s 2024-06-06T07:24:09.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91314 8.80 MB/sec execute 434 sec latency 9521.776 ms 2024-06-06T07:24:10.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:10 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:24:10.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:10 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:24:10.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:10 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:24:10.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91465 8.80 MB/sec execute 435 sec latency 10258.103 ms 2024-06-06T07:24:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:11 smithi073 ceph-mon[32303]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:11 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:11 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:24:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:11 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:24:11.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:11 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:24:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:11 smithi136 ceph-mon[35901]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:11 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:11 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:24:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:11 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:24:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:11 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:24:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:11 smithi170 ceph-mon[36108]: pgmap v727: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:11 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:11 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:24:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:11 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:24:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:11 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:24:11.828 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91611 8.79 MB/sec execute 436 sec latency 112.563 ms 2024-06-06T07:24:12.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91780 8.80 MB/sec execute 437 sec latency 128.907 ms 2024-06-06T07:24:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:13 smithi073 ceph-mon[32303]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:13 smithi073 ceph-mon[32303]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T07:24:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:13 smithi136 ceph-mon[35901]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:13 smithi136 ceph-mon[35901]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T07:24:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:13 smithi170 ceph-mon[36108]: pgmap v728: 129 pgs: 129 active+clean; 1.3 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.0 MiB/s wr, 335 op/s 2024-06-06T07:24:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:13 smithi170 ceph-mon[36108]: osdmap e142: 12 total, 12 up, 12 in 2024-06-06T07:24:13.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 91936 8.79 MB/sec execute 438 sec latency 86.203 ms 2024-06-06T07:24:14.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92070 8.79 MB/sec execute 439 sec latency 80.332 ms 2024-06-06T07:24:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:15 smithi073 ceph-mon[32303]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-06-06T07:24:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:15 smithi136 ceph-mon[35901]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-06-06T07:24:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:15 smithi170 ceph-mon[36108]: pgmap v730: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 6.1 MiB/s wr, 335 op/s 2024-06-06T07:24:15.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92208 8.79 MB/sec execute 440 sec latency 116.724 ms 2024-06-06T07:24:16.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92381 8.81 MB/sec execute 441 sec latency 100.911 ms 2024-06-06T07:24:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:17 smithi073 ceph-mon[32303]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-06-06T07:24:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:17 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:17 smithi136 ceph-mon[35901]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-06-06T07:24:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:17 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:17 smithi170 ceph-mon[36108]: pgmap v731: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.0 MiB/s wr, 330 op/s 2024-06-06T07:24:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:17 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:17.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92532 8.80 MB/sec execute 442 sec latency 80.947 ms 2024-06-06T07:24:18.827 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92656 8.79 MB/sec execute 443 sec latency 86.413 ms 2024-06-06T07:24:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:19 smithi073 ceph-mon[32303]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-06-06T07:24:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:19 smithi136 ceph-mon[35901]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-06-06T07:24:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:19 smithi170 ceph-mon[36108]: pgmap v732: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.7 MiB/s wr, 299 op/s 2024-06-06T07:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 -- 172.21.15.73:0/2563058191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748103550 msgr2=0x7f974810ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 --2- 172.21.15.73:0/2563058191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748103550 0x7f974810ba30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f9738009900 tx=0x7f973802a510 comp rx=0 tx=0).stop 2024-06-06T07:24:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 -- 172.21.15.73:0/2563058191 shutdown_connections 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 --2- 172.21.15.73:0/2563058191 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 0x7f974810e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 --2- 172.21.15.73:0/2563058191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748103550 0x7f974810ba30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 --2- 172.21.15.73:0/2563058191 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748102c30 0x7f9748103010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.709+0000 7f974e88c640 1 -- 172.21.15.73:0/2563058191 >> 172.21.15.73:0/2563058191 conn(0x7f9748077a20 msgr2=0x7f9748077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.710+0000 7f974e88c640 1 -- 172.21.15.73:0/2563058191 shutdown_connections 2024-06-06T07:24:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.710+0000 7f974e88c640 1 -- 172.21.15.73:0/2563058191 wait complete. 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.710+0000 7f974e88c640 1 Processor -- start 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.710+0000 7f974e88c640 1 -- start start 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748103550 0x7f9748100dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 0x7f9748101300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9748111c20 con 0x7f9748102c30 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9748111aa0 con 0x7f974810bf70 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974e88c640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9748111da0 con 0x7f9748103550 2024-06-06T07:24:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37670/0 (socket says 172.21.15.73:37670) 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 -- 172.21.15.73:0/1415451130 learned_addr learned my addr 172.21.15.73:0/1415451130 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f974ce29640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 0x7f9748101300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f97477fe640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748103550 0x7f9748100dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 -- 172.21.15.73:0/1415451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748103550 msgr2=0x7f9748100dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748103550 0x7f9748100dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 -- 172.21.15.73:0/1415451130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 msgr2=0x7f9748101300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 0x7f9748101300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 -- 172.21.15.73:0/1415451130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97481a6600 con 0x7f9748102c30 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.711+0000 7f9747fff640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e9f0 tx=0x7f973c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.712+0000 7f97457fa640 1 -- 172.21.15.73:0/1415451130 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00cd80 con 0x7f9748102c30 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.712+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97481a68f0 con 0x7f9748102c30 2024-06-06T07:24:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.712+0000 7f97457fa640 1 -- 172.21.15.73:0/1415451130 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f973c0108e0 con 0x7f9748102c30 2024-06-06T07:24:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.712+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97481a6e60 con 0x7f9748102c30 2024-06-06T07:24:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.712+0000 7f97457fa640 1 -- 172.21.15.73:0/1415451130 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c018920 con 0x7f9748102c30 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f97457fa640 1 -- 172.21.15.73:0/1415451130 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f973c010430 con 0x7f9748102c30 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 Processor -- start 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 -- start start 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49aa90 0x56215f4a4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f4a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56215f3fe4f0 con 0x56215f49a170 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56215f39c870 con 0x56215f49aa90 2024-06-06T07:24:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.715+0000 7f9751598740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56215f3fa4b0 con 0x56215f4a5350 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37674/0 (socket says 172.21.15.73:37674) 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 -- 172.21.15.73:0/1615777508 learned_addr learned my addr 172.21.15.73:0/1615777508 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f97477fe640 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f4a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f97457fa640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 0x7f97300a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f9747fff640 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49aa90 0x56215f4a4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 -- 172.21.15.73:0/1615777508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 msgr2=0x56215f4a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f4a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 -- 172.21.15.73:0/1615777508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49aa90 msgr2=0x56215f4a4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49aa90 0x56215f4a4e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f974ce29640 1 -- 172.21.15.73:0/1615777508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215f3b89b0 con 0x56215f49a170 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.716+0000 7f97457fa640 1 -- 172.21.15.73:0/1415451130 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f973c0cfb10 con 0x7f9748102c30 2024-06-06T07:24:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f974ce29640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 0x7f97300a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f974ce29640 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9740009f50 tx=0x7f97400118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f458e94aa1185bcf server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f972e7fc640 1 -- 172.21.15.73:0/1615777508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740004250 con 0x56215f49a170 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f972e7fc640 1 -- 172.21.15.73:0/1615777508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974000f930 con 0x56215f49a170 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f974ce29640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 0x7f97300a5260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9740020000 tx=0x7f9740021000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.717+0000 7f972e7fc640 1 -- 172.21.15.73:0/1615777508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400257e0 con 0x56215f49a170 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- 172.21.15.73:0/1615777508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 msgr2=0x56215f49a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f9740009f50 tx=0x7f97400118d0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.722 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- 172.21.15.73:0/1615777508 shutdown_connections 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f4a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49aa90 0x56215f4a4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 --2- 172.21.15.73:0/1615777508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49a170 0x56215f49a550 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- 172.21.15.73:0/1615777508 >> 172.21.15.73:0/1615777508 conn(0x56215f496680 msgr2=0x56215f496a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- 172.21.15.73:0/1615777508 shutdown_connections 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- 172.21.15.73:0/1615777508 wait complete. 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.718+0000 7f9751598740 1 -- start start 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 Processor -- start 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f4a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56215f4a7f80 con 0x56215f4a5350 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f97477fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f97477fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:33106/0 (socket says 172.21.15.73:33106) 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f97477fe640 1 -- 172.21.15.73:0/2038720834 learned_addr learned my addr 172.21.15.73:0/2038720834 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a22b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:40828/0 (socket says 172.21.15.73:40828) 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56215f387ae0 con 0x56215f49a170 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56215f4a8100 con 0x56215f49aa90 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f974ce29640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f4a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.719+0000 7f9751598740 10 client.0 _ll_register_callbacks cb 0x56215f3c09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 -- 172.21.15.73:0/2038720834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 msgr2=0x56215f4a22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 -- 172.21.15.73:0/2038720834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 msgr2=0x56215f4a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f4a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215f112210 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f974ce29640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f4a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97477fe640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9738009900 tx=0x7f97380314a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1512a1c8a0628ef server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97277fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738035040 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974000f930 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f9738031810 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f973804f050 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f9751598740 10 client.27109 fetch_fsmap learned FSMap version 82 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.720+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=4+}) -- 0x56215f1de620 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9738005030 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 82) ==== 8370+0+0 (secure 0 0 0) 0x7f9738004870 con 0x56215f49aa90 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 10 client.27109 fetch_fsmap finished waiting for FSMap version 82 2024-06-06T07:24:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 10 client.27109 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 20 client.27109 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f4b57d0 0x56215f4b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 4 client.27109 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:24:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f974000f930 con 0x56215f4b57d0 2024-06-06T07:24:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.721+0000 7f9747fff640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f4b57d0 0x56215f4b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.722+0000 7f97277fe640 10 client.27109 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.722+0000 7f9747fff640 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f4b57d0 0x56215f4b7c30 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b903e91f7b6f4c93 server_cookie=5b987a8300c5d76b in_seq=0 out_seq=0 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.722+0000 7f97277fe640 1 -- 172.21.15.73:0/2038720834 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f974000f930 con 0x56215f4b57d0 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.722+0000 7f97277fe640 10 client.27109 handle_command_reply: tid=0 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.722+0000 7f9751598740 1 client.27109 shutdown 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 msgr2=0x56215f4a27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9738009900 tx=0x7f97380314a0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 shutdown_connections 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f4b57d0 0x56215f4b7c30 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f4a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4a27f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 --2- 172.21.15.73:0/2038720834 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f97277fe640 0 client.27109 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 >> 172.21.15.73:0/2038720834 conn(0x56215f496680 msgr2=0x56215f49c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 shutdown_connections 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 1 -- 172.21.15.73:0/2038720834 wait complete. 2024-06-06T07:24:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.723+0000 7f9751598740 20 client.27109 trim_cache size 0 max 16384 2024-06-06T07:24:19.758 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.757+0000 7f9751598740 1 Processor -- start 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 -- start start 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4bd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4bda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f494f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4bd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4bd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:40830/0 (socket says 172.21.15.73:40830) 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9747fff640 1 -- 172.21.15.73:0/4266001884 learned_addr learned my addr 172.21.15.73:0/4266001884 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9734010090 con 0x56215f4a5350 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9734011bd0 con 0x56215f49a170 2024-06-06T07:24:19.759 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.758+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f9734010760 con 0x56215f49aa90 2024-06-06T07:24:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.759+0000 7f974ce29640 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f494f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.759+0000 7f974ce29640 1 -- 172.21.15.73:0/4266001884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 msgr2=0x56215f4bda40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:24:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.759+0000 7f974ce29640 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4bda40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.759+0000 7f974ce29640 1 -- 172.21.15.73:0/4266001884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 msgr2=0x56215f4bd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.759+0000 7f974ce29640 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4bd500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.760+0000 7f974ce29640 1 -- 172.21.15.73:0/4266001884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56215f491f70 con 0x56215f4a5350 2024-06-06T07:24:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.762+0000 7f974ce29640 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f494f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f974001d040 tx=0x7f9740005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4984ef57c6e847cd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.762+0000 7f972dffb640 1 -- 172.21.15.73:0/4266001884 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740029040 con 0x56215f4a5350 2024-06-06T07:24:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.762+0000 7f972dffb640 1 -- 172.21.15.73:0/4266001884 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97400041d0 con 0x56215f4a5350 2024-06-06T07:24:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.763+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 msgr2=0x56215f494f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.763+0000 7f9751598740 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f494f50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f974001d040 tx=0x7f9740005bd0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 shutdown_connections 2024-06-06T07:24:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f4a5350 0x56215f494f50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f49aa90 0x56215f4bda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 --2- 172.21.15.73:0/4266001884 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4bd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 >> 172.21.15.73:0/4266001884 conn(0x56215f496680 msgr2=0x56215f49d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 shutdown_connections 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.764+0000 7f9751598740 1 -- 172.21.15.73:0/4266001884 wait complete. 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.765+0000 7f9751598740 1 -- start start 2024-06-06T07:24:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.765+0000 7f9751598740 1 Processor -- start 2024-06-06T07:24:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.765+0000 7f9751598740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9751598740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 0x56215f4a9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9747fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:40834/0 (socket says 172.21.15.73:40834) 2024-06-06T07:24:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9747fff640 1 -- 172.21.15.73:0/1525484199 learned_addr learned my addr 172.21.15.73:0/1525484199 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f526cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9734011500 con 0x56215f49aa90 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f9734010e30 con 0x56215f49a170 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f974ce29640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f526cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.766+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f973400e090 con 0x56215f4a5350 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9751598740 10 client.0 _ll_register_callbacks cb 0x7f9738009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f97477fe640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 0x56215f4a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 -- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 msgr2=0x56215f526cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f526cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 -- 172.21.15.73:0/1525484199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 msgr2=0x56215f4a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 0x56215f4a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56215f5273c0 con 0x56215f49a170 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f97477fe640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 0x56215f4a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9747fff640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f973c098640 tx=0x7f973c0cfe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db342b258f2c1aad server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f97277fe640 10 client.26927 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97400041d0 con 0x56215f49a170 2024-06-06T07:24:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.767+0000 7f97277fe640 1 -- 172.21.15.73:0/1525484199 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0a4830 con 0x56215f49a170 2024-06-06T07:24:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.768+0000 7f97277fe640 1 -- 172.21.15.73:0/1525484199 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f973c0982e0 con 0x56215f49a170 2024-06-06T07:24:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.768+0000 7f97277fe640 1 -- 172.21.15.73:0/1525484199 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f973c0cd070 con 0x56215f49a170 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.768+0000 7f9751598740 10 client.26927 fetch_fsmap learned FSMap version 82 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.768+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({fsmap=82}) -- 0x56215f528600 con 0x56215f49a170 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f97277fe640 1 -- 172.21.15.73:0/1525484199 <== mon.2 v2:172.21.15.136:3300/0 4 ==== fsmap(e 82) ==== 8370+0+0 (secure 0 0 0) 0x7f973c0ba2d0 con 0x56215f49a170 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 10 client.26927 fetch_fsmap finished waiting for FSMap version 82 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 10 client.26927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 20 client.26927 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f528d20 0x56215f52b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 4 client.26927 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f97400041d0 con 0x56215f528d20 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.769+0000 7f97477fe640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f528d20 0x56215f52b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.770+0000 7f97277fe640 10 client.26927 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:19.771 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.770+0000 7f97477fe640 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f528d20 0x56215f52b140 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8236f4adc319d3c3 server_cookie=50da024dee33ee14 in_seq=0 out_seq=0 2024-06-06T07:24:19.833 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92791 8.79 MB/sec execute 444 sec latency 110.541 ms 2024-06-06T07:24:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.914+0000 7f97277fe640 1 -- 172.21.15.73:0/1525484199 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f97400041d0 con 0x56215f528d20 2024-06-06T07:24:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.914+0000 7f97277fe640 10 client.26927 handle_command_reply: tid=0 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.914+0000 7f9751598740 1 client.26927 shutdown 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 msgr2=0x56215f4a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f973c098640 tx=0x7f973c0cfe40 comp rx=0 tx=0).stop 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 shutdown_connections 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56215f528d20 0x56215f52b140 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56215f4a5350 0x56215f526cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56215f49aa90 0x56215f4a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 --2- 172.21.15.73:0/1525484199 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56215f49a170 0x56215f4a8b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 >> 172.21.15.73:0/1525484199 conn(0x56215f496680 msgr2=0x56215f4a2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f97277fe640 0 client.26927 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 shutdown_connections 2024-06-06T07:24:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 1 -- 172.21.15.73:0/1525484199 wait complete. 2024-06-06T07:24:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.915+0000 7f9751598740 20 client.26927 trim_cache size 0 max 16384 2024-06-06T07:24:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:24:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:24:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:24:19.918 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 msgr2=0x7f97300a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 0x7f97300a5260 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9740020000 tx=0x7f9740021000 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 msgr2=0x7f9748100880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e9f0 tx=0x7f973c00eeb0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 shutdown_connections 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f974810bf70 0x7f9748101300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f97300a2da0 0x7f97300a5260 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f9748103550 0x7f9748100dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 --2- 172.21.15.73:0/1415451130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9748102c30 0x7f9748100880 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.916+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 >> 172.21.15.73:0/1415451130 conn(0x7f9748077a20 msgr2=0x7f97480fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.917+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 shutdown_connections 2024-06-06T07:24:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:19.917+0000 7f974e88c640 1 -- 172.21.15.73:0/1415451130 wait complete. 2024-06-06T07:24:20.833 INFO:tasks.workunit.client.0.smithi073.stdout: 10 92938 8.78 MB/sec execute 445 sec latency 145.109 ms 2024-06-06T07:24:21.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:21 smithi073 ceph-mon[32303]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:21 smithi136 ceph-mon[35901]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:21.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:21 smithi170 ceph-mon[36108]: pgmap v733: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:21.833 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93064 8.78 MB/sec execute 446 sec latency 129.091 ms 2024-06-06T07:24:22.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:22 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:22 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:22 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:22.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93195 8.77 MB/sec execute 447 sec latency 75.567 ms 2024-06-06T07:24:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:23 smithi136 ceph-mon[35901]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:23 smithi170 ceph-mon[36108]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:23 smithi073 ceph-mon[32303]: pgmap v734: 129 pgs: 129 active+clean; 1.4 GiB data, 8.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 311 op/s 2024-06-06T07:24:23.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93353 8.77 MB/sec execute 448 sec latency 79.680 ms 2024-06-06T07:24:24.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93507 8.76 MB/sec execute 449 sec latency 105.913 ms 2024-06-06T07:24:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:25 smithi136 ceph-mon[35901]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:24:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:25 smithi170 ceph-mon[36108]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:24:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:25 smithi073 ceph-mon[32303]: pgmap v735: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:24:25.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93682 8.75 MB/sec execute 450 sec latency 85.044 ms 2024-06-06T07:24:26.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93830 8.75 MB/sec execute 451 sec latency 95.264 ms 2024-06-06T07:24:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:27 smithi136 ceph-mon[35901]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-06-06T07:24:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:27 smithi170 ceph-mon[36108]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-06-06T07:24:27.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:27 smithi073 ceph-mon[32303]: pgmap v736: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 255 op/s 2024-06-06T07:24:27.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 93967 8.75 MB/sec execute 452 sec latency 101.208 ms 2024-06-06T07:24:28.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94134 8.76 MB/sec execute 453 sec latency 75.770 ms 2024-06-06T07:24:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:29 smithi136 ceph-mon[35901]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 382 op/s 2024-06-06T07:24:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:29 smithi170 ceph-mon[36108]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 382 op/s 2024-06-06T07:24:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:29 smithi073 ceph-mon[32303]: pgmap v737: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.0 MiB/s wr, 382 op/s 2024-06-06T07:24:29.834 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94300 8.76 MB/sec execute 454 sec latency 98.834 ms 2024-06-06T07:24:30.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94484 8.77 MB/sec execute 455 sec latency 301.103 ms 2024-06-06T07:24:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:31 smithi136 ceph-mon[35901]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2024-06-06T07:24:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:31 smithi170 ceph-mon[36108]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2024-06-06T07:24:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:31 smithi073 ceph-mon[32303]: pgmap v738: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.0 MiB/s wr, 283 op/s 2024-06-06T07:24:31.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94675 8.76 MB/sec execute 456 sec latency 60.153 ms 2024-06-06T07:24:32.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:32 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:32.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:32 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:32.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:32 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:24:32.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94815 8.75 MB/sec execute 457 sec latency 63.394 ms 2024-06-06T07:24:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 -- 172.21.15.73:0/3171863416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74106890 msgr2=0x7f7f74106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/3171863416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74106890 0x7f7f74106c70 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7f64009930 tx=0x7f7f6402a420 comp rx=0 tx=0).stop 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 -- 172.21.15.73:0/3171863416 shutdown_connections 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/3171863416 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f74101290 0x7f7f7410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/3171863416 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f741008d0 0x7f7f74100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/3171863416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74106890 0x7f7f74106c70 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.048+0000 7f7f7a4df640 1 -- 172.21.15.73:0/3171863416 >> 172.21.15.73:0/3171863416 conn(0x7f7f74077a20 msgr2=0x7f7f74077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.049+0000 7f7f7a4df640 1 -- 172.21.15.73:0/3171863416 shutdown_connections 2024-06-06T07:24:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.049+0000 7f7f7a4df640 1 -- 172.21.15.73:0/3171863416 wait complete. 2024-06-06T07:24:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.049+0000 7f7f7a4df640 1 Processor -- start 2024-06-06T07:24:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 -- start start 2024-06-06T07:24:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f741008d0 0x7f7f7419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 0x7f7f741a3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54400/0 (socket says 172.21.15.73:54400) 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f737fe640 1 -- 172.21.15.73:0/725967254 learned_addr learned my addr 172.21.15.73:0/725967254 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7410fa90 con 0x7f7f74101290 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7410f910 con 0x7f7f74106890 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7410fc10 con 0x7f7f741008d0 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f73fff640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f741008d0 0x7f7f7419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.050+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 0x7f7f741a3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 -- 172.21.15.73:0/725967254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f741008d0 msgr2=0x7f7f7419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f741008d0 0x7f7f7419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 -- 172.21.15.73:0/725967254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 msgr2=0x7f7f741a3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 0x7f7f741a3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f741a3750 con 0x7f7f74101290 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 0x7f7f741a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f737fe640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800ea60 tx=0x7f7f6800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f717fa640 1 -- 172.21.15.73:0/725967254 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6800cde0 con 0x7f7f74101290 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f717fa640 1 -- 172.21.15.73:0/725967254 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f680035f0 con 0x7f7f74101290 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f741a3a40 con 0x7f7f74101290 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f717fa640 1 -- 172.21.15.73:0/725967254 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68010690 con 0x7f7f74101290 2024-06-06T07:24:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.051+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f741ab2d0 con 0x7f7f74101290 2024-06-06T07:24:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.055+0000 7f7f717fa640 1 -- 172.21.15.73:0/725967254 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7f680037a0 con 0x7f7f74101290 2024-06-06T07:24:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 Processor -- start 2024-06-06T07:24:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f717fa640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 0x7f7f5c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 -- start start 2024-06-06T07:24:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 0x7f7f5c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.058 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca0d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56214c9644f0 con 0x56214ca00a90 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56214c902870 con 0x56214ca00170 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56214c9604b0 con 0x56214ca0b350 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.056+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56206/0 (socket says 172.21.15.73:56206) 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54404/0 (socket says 172.21.15.73:54404) 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 -- 172.21.15.73:0/664789839 learned_addr learned my addr 172.21.15.73:0/664789839 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca0d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f717fa640 1 -- 172.21.15.73:0/725967254 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f7f680cfa20 con 0x7f7f74101290 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 -- 172.21.15.73:0/664789839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 msgr2=0x56214ca0d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca0d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 -- 172.21.15.73:0/664789839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 msgr2=0x56214ca0ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 0x7f7f5c0a52e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009fd0 tx=0x7f7f6c011f60 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 -- 172.21.15.73:0/664789839 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214c91e9b0 con 0x56214ca00170 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f737fe640 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f73fff640 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006010 tx=0x7f7f64005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dff8b6b68367ec21 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f5a7fc640 1 -- 172.21.15.73:0/664789839 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64035040 con 0x56214ca00170 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f5a7fc640 1 -- 172.21.15.73:0/664789839 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f64004480 con 0x56214ca00170 2024-06-06T07:24:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.057+0000 7f7f5a7fc640 1 -- 172.21.15.73:0/664789839 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640035f0 con 0x56214ca00170 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/664789839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 msgr2=0x56214ca00550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7f64006010 tx=0x7f7f64005be0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/664789839 shutdown_connections 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca0d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00a90 0x56214ca0ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/664789839 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca00550 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/664789839 >> 172.21.15.73:0/664789839 conn(0x56214c9fc680 msgr2=0x56214c9fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.058+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/664789839 shutdown_connections 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/664789839 wait complete. 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 -- start start 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 Processor -- start 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca087e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca08d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56214ca0df20 con 0x56214ca00170 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56214c8edae0 con 0x56214ca00a90 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56214ca0e0a0 con 0x56214ca0b350 2024-06-06T07:24:33.064 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f7d1e0740 10 client.0 _ll_register_callbacks cb 0x56214c9269b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.059+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54410/0 (socket says 172.21.15.73:54410) 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f78a7c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca08d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 -- 172.21.15.73:0/791560035 learned_addr learned my addr 172.21.15.73:0/791560035 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f737fe640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca087e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 -- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 msgr2=0x56214ca08d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca08d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 -- 172.21.15.73:0/791560035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 msgr2=0x56214ca087e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca087e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 -- 172.21.15.73:0/791560035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214c678210 con 0x56214ca00170 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f73fff640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004650 tx=0x7f7f64004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf5d414d2962ac73 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f537fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:24:33.065 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64052060 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f7f64004280 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f64004480 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.060+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f7f64035070 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 10 client.17730 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56214c744620 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f64004480 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f7f64031920 con 0x56214ca00170 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 10 client.17730 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 20 client.17730 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:33.066 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca1b4b0 0x56214ca1d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 4 client.17730 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:24:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56214c977f10 con 0x56214ca1b4b0 2024-06-06T07:24:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.061+0000 7f7f737fe640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca1b4b0 0x56214ca1d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.062+0000 7f7f737fe640 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca1b4b0 0x56214ca1d920 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caffb9ba89f98020 server_cookie=82bf953554bd91fa in_seq=0 out_seq=0 2024-06-06T07:24:33.067 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.062+0000 7f7f537fe640 10 client.17730 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f537fe640 1 -- 172.21.15.73:0/791560035 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56214c977f10 con 0x56214ca1b4b0 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f537fe640 10 client.17730 handle_command_reply: tid=0 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 client.17730 shutdown 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 msgr2=0x56214ca082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004650 tx=0x7f7f64004980 comp rx=0 tx=0).stop 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 shutdown_connections 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca1b4b0 0x56214ca1d920 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca0b350 0x56214ca08d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.068 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/791560035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca00170 0x56214ca082a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 >> 172.21.15.73:0/791560035 conn(0x56214c9fc680 msgr2=0x56214ca02400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f537fe640 0 client.17730 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 shutdown_connections 2024-06-06T07:24:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.063+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/791560035 wait complete. 2024-06-06T07:24:33.069 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.064+0000 7f7f7d1e0740 20 client.17730 trim_cache size 0 max 16384 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 Processor -- start 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 -- start start 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00170 0x56214c9f7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca23740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b54a0 con 0x56214ca0b350 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b6fe0 con 0x56214ca00a90 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.092+0000 7f7f7d1e0740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b5b70 con 0x56214ca00170 2024-06-06T07:24:33.094 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56224/0 (socket says 172.21.15.73:56224) 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00170 0x56214c9f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 -- 172.21.15.73:0/987880266 learned_addr learned my addr 172.21.15.73:0/987880266 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 -- 172.21.15.73:0/987880266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00170 msgr2=0x56214c9f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00170 0x56214c9f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 -- 172.21.15.73:0/987880266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 msgr2=0x56214ca23740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca23740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 -- 172.21.15.73:0/987880266 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56214ca17180 con 0x56214ca00a90 2024-06-06T07:24:33.095 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.093+0000 7f7f737fe640 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7f68098500 tx=0x7f7f6809ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5e95a84ded2f12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.094+0000 7f7f59ffb640 1 -- 172.21.15.73:0/987880266 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6809ce60 con 0x56214ca00a90 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.094+0000 7f7f59ffb640 1 -- 172.21.15.73:0/987880266 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f68078d40 con 0x56214ca00a90 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.094+0000 7f7f59ffb640 1 -- 172.21.15.73:0/987880266 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f68004000 con 0x56214ca00a90 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/987880266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 msgr2=0x56214ca23200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7f68098500 tx=0x7f7f6809ba20 comp rx=0 tx=0).stop 2024-06-06T07:24:33.096 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/987880266 shutdown_connections 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca23740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00a90 0x56214ca23200 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/987880266 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00170 0x56214c9f7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.095+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/987880266 >> 172.21.15.73:0/987880266 conn(0x56214c9fc680 msgr2=0x56214ca06b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.096+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/987880266 shutdown_connections 2024-06-06T07:24:33.097 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.096+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/987880266 wait complete. 2024-06-06T07:24:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.096+0000 7f7f7d1e0740 1 -- start start 2024-06-06T07:24:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.096+0000 7f7f7d1e0740 1 Processor -- start 2024-06-06T07:24:33.098 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f7d1e0740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00a90 0x56214ca132f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f73fff640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:56232/0 (socket says 172.21.15.73:56232) 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f73fff640 1 -- 172.21.15.73:0/745102304 learned_addr learned my addr 172.21.15.73:0/745102304 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.097+0000 7f7f78a7c640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b6910 con 0x56214ca0b350 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b6240 con 0x56214ca00170 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7f741b34a0 con 0x56214ca00a90 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f7d1e0740 10 client.0 _ll_register_callbacks cb 0x7f7f68004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 -- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00a90 msgr2=0x56214ca132f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.099 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00a90 0x56214ca132f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 -- 172.21.15.73:0/745102304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 msgr2=0x56214ca139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56214ca04590 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.098+0000 7f7f73fff640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004720 tx=0x7f7f640316d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e18c5f1207595e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f537fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64052040 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7f7f64031a50 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f68078d40 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f7f64005850 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f7d1e0740 10 client.26945 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.099+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x56214ca854d0 con 0x56214ca00170 2024-06-06T07:24:33.101 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f64009520 con 0x56214ca00170 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f7f6404e8e0 con 0x56214ca00170 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 10 client.26945 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 10 client.26945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 20 client.26945 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca85ba0 0x56214ca87fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 4 client.26945 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.100+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f7f68078d40 con 0x56214ca85ba0 2024-06-06T07:24:33.102 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.101+0000 7f7f737fe640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca85ba0 0x56214ca87fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.102+0000 7f7f737fe640 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca85ba0 0x56214ca87fc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49eeb8c2078e0083 server_cookie=4ad64143c555bb5f in_seq=0 out_seq=0 2024-06-06T07:24:33.103 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.102+0000 7f7f537fe640 10 client.26945 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:33.208 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.207+0000 7f7f537fe640 1 -- 172.21.15.73:0/745102304 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f7f68078d40 con 0x56214ca85ba0 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.207+0000 7f7f537fe640 10 client.26945 handle_command_reply: tid=0 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.207+0000 7f7f7d1e0740 1 client.26945 shutdown 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.207+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 msgr2=0x56214ca05f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.207+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7f64004720 tx=0x7f7f640316d0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 shutdown_connections 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56214ca85ba0 0x56214ca87fc0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56214ca0b350 0x56214ca139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56214ca00a90 0x56214ca132f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 --2- 172.21.15.73:0/745102304 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56214ca00170 0x56214ca05f80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 >> 172.21.15.73:0/745102304 conn(0x56214c9fc680 msgr2=0x56214ca08240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f537fe640 0 client.26945 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 shutdown_connections 2024-06-06T07:24:33.209 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 1 -- 172.21.15.73:0/745102304 wait complete. 2024-06-06T07:24:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.208+0000 7f7f7d1e0740 20 client.26945 trim_cache size 0 max 16384 2024-06-06T07:24:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:24:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:24:33.210 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:24:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 msgr2=0x7f7f5c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 0x7f7f5c0a52e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c009fd0 tx=0x7f7f6c011f60 comp rx=0 tx=0).stop 2024-06-06T07:24:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 msgr2=0x7f7f7419eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7f6800ea60 tx=0x7f7f6800ef20 comp rx=0 tx=0).stop 2024-06-06T07:24:33.213 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 shutdown_connections 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7f74106890 0x7f7f741a3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f74101290 0x7f7f7419eca0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7f5c0a2e20 0x7f7f5c0a52e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 --2- 172.21.15.73:0/725967254 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7f741008d0 0x7f7f7419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.212+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 >> 172.21.15.73:0/725967254 conn(0x7f7f74077a20 msgr2=0x7f7f740fd790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.213+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 shutdown_connections 2024-06-06T07:24:33.214 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:33.213+0000 7f7f7a4df640 1 -- 172.21.15.73:0/725967254 wait complete. 2024-06-06T07:24:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:33 smithi170 ceph-mon[36108]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-06-06T07:24:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:33 smithi073 ceph-mon[32303]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-06-06T07:24:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:33 smithi136 ceph-mon[35901]: pgmap v739: 129 pgs: 129 active+clean; 1.4 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 4.6 MiB/s wr, 257 op/s 2024-06-06T07:24:33.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 94954 8.75 MB/sec execute 458 sec latency 81.600 ms 2024-06-06T07:24:34.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95128 8.76 MB/sec execute 459 sec latency 106.048 ms 2024-06-06T07:24:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:35 smithi170 ceph-mon[36108]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2024-06-06T07:24:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:35 smithi073 ceph-mon[32303]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2024-06-06T07:24:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:35 smithi136 ceph-mon[35901]: pgmap v740: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2024-06-06T07:24:35.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95295 8.77 MB/sec execute 460 sec latency 161.975 ms 2024-06-06T07:24:36.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95490 8.78 MB/sec execute 461 sec latency 78.996 ms 2024-06-06T07:24:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:37 smithi170 ceph-mon[36108]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-06T07:24:37.602 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:37 smithi136 ceph-mon[35901]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-06T07:24:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:37 smithi073 ceph-mon[32303]: pgmap v741: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-06T07:24:37.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95680 8.79 MB/sec execute 462 sec latency 84.957 ms 2024-06-06T07:24:38.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95817 8.78 MB/sec execute 463 sec latency 69.193 ms 2024-06-06T07:24:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:39 smithi170 ceph-mon[36108]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-06-06T07:24:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:39 smithi073 ceph-mon[32303]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-06-06T07:24:39.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:39 smithi136 ceph-mon[35901]: pgmap v742: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 7.6 MiB/s wr, 389 op/s 2024-06-06T07:24:39.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 95951 8.77 MB/sec execute 464 sec latency 77.005 ms 2024-06-06T07:24:40.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96117 8.78 MB/sec execute 465 sec latency 78.746 ms 2024-06-06T07:24:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:41 smithi170 ceph-mon[36108]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 265 op/s 2024-06-06T07:24:41.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:41 smithi073 ceph-mon[32303]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 265 op/s 2024-06-06T07:24:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:41 smithi136 ceph-mon[35901]: pgmap v743: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.6 MiB/s wr, 265 op/s 2024-06-06T07:24:41.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96263 8.78 MB/sec execute 466 sec latency 164.959 ms 2024-06-06T07:24:42.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96410 8.77 MB/sec execute 467 sec latency 104.678 ms 2024-06-06T07:24:43.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:43 smithi170 ceph-mon[36108]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 262 op/s 2024-06-06T07:24:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:43 smithi073 ceph-mon[32303]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 262 op/s 2024-06-06T07:24:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:43 smithi136 ceph-mon[35901]: pgmap v744: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.5 MiB/s wr, 262 op/s 2024-06-06T07:24:43.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96582 8.77 MB/sec execute 468 sec latency 180.353 ms 2024-06-06T07:24:44.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96747 8.77 MB/sec execute 469 sec latency 91.490 ms 2024-06-06T07:24:45.576 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:45 smithi073 ceph-mon[32303]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-06-06T07:24:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:45 smithi170 ceph-mon[36108]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-06-06T07:24:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:45 smithi136 ceph-mon[35901]: pgmap v745: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 8.4 MiB/s wr, 398 op/s 2024-06-06T07:24:45.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 96894 8.76 MB/sec execute 470 sec latency 69.877 ms 2024-06-06T07:24:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 -- 172.21.15.73:0/1251281062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee081047e0 msgr2=0x7fee08108820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 --2- 172.21.15.73:0/1251281062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee081047e0 0x7fee08108820 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fedf8009900 tx=0x7fedf802a410 comp rx=0 tx=0).stop 2024-06-06T07:24:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 -- 172.21.15.73:0/1251281062 shutdown_connections 2024-06-06T07:24:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 --2- 172.21.15.73:0/1251281062 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08108f00 0x7fee08110ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 --2- 172.21.15.73:0/1251281062 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee081047e0 0x7fee08108820 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 --2- 172.21.15.73:0/1251281062 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee08103e30 0x7fee08104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 -- 172.21.15.73:0/1251281062 >> 172.21.15.73:0/1251281062 conn(0x7fee080ff6b0 msgr2=0x7fee08101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 -- 172.21.15.73:0/1251281062 shutdown_connections 2024-06-06T07:24:46.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.350+0000 7fee0fd90640 1 -- 172.21.15.73:0/1251281062 wait complete. 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 Processor -- start 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 -- start start 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 0x7fee0819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 0x7fee0819f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee08113eb0 con 0x7fee08108f00 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee08113d30 con 0x7fee08103e30 2024-06-06T07:24:46.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0fd90640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee08114030 con 0x7fee0819ec30 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0db2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.351+0000 7fee0db2c640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:50582/0 (socket says 172.21.15.73:50582) 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 0x7fee0819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0db2c640 1 -- 172.21.15.73:0/972810372 learned_addr learned my addr 172.21.15.73:0/972810372 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0e32d640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 0x7fee0819f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 -- 172.21.15.73:0/972810372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 msgr2=0x7fee0819f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 0x7fee0819f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 -- 172.21.15.73:0/972810372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 msgr2=0x7fee0819a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 -- 172.21.15.73:0/972810372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee0819f720 con 0x7fee08108f00 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0e32d640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 0x7fee0819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0db2c640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:24:46.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0d32b640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 0x7fee0819a8b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fedf802a930 tx=0x7fedf8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fedfeffd640 1 -- 172.21.15.73:0/972810372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf803f070 con 0x7fee08108f00 2024-06-06T07:24:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee0819f9b0 con 0x7fee08108f00 2024-06-06T07:24:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fedfeffd640 1 -- 172.21.15.73:0/972810372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf8031b60 con 0x7fee08108f00 2024-06-06T07:24:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fedfeffd640 1 -- 172.21.15.73:0/972810372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8035030 con 0x7fee08108f00 2024-06-06T07:24:46.357 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.352+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee081a7170 con 0x7fee08108f00 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.355+0000 7fedfeffd640 1 -- 172.21.15.73:0/972810372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fedf8042070 con 0x7fee08108f00 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.355+0000 7fee12b80740 1 Processor -- start 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 -- start start 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55794c6534f0 con 0x55794c6fa350 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55794c5f1870 con 0x55794c6efa90 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee12b80740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55794c64f4b0 con 0x55794c6ef170 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57888/0 (socket says 172.21.15.73:57888) 2024-06-06T07:24:46.359 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 -- 172.21.15.73:0/1278182025 learned_addr learned my addr 172.21.15.73:0/1278182025 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0d32b640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0db2c640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fedfeffd640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 0x7fedf40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 -- 172.21.15.73:0/1278182025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 msgr2=0x55794c6f9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 -- 172.21.15.73:0/1278182025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 msgr2=0x55794c6fc740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fc740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.356+0000 7fee0e32d640 1 -- 172.21.15.73:0/1278182025 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c60d9b0 con 0x55794c6ef170 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fee0d32b640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fedfeffd640 1 -- 172.21.15.73:0/972810372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fedf80efdf0 con 0x7fee08108f00 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fee0e32d640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 0x7fedf40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fd0 tx=0x7fee00010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b6b0315afb064a server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.360 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fedf3fff640 1 -- 172.21.15.73:0/1278182025 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00022070 con 0x55794c6ef170 2024-06-06T07:24:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fedf3fff640 1 -- 172.21.15.73:0/1278182025 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee0001b380 con 0x55794c6ef170 2024-06-06T07:24:46.361 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.357+0000 7fedf3fff640 1 -- 172.21.15.73:0/1278182025 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0001ce60 con 0x55794c6ef170 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- 172.21.15.73:0/1278182025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 msgr2=0x55794c6ef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee0e32d640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 0x7fedf40a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee0001f000 tx=0x7fee00020000 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fee00009fd0 tx=0x7fee00010860 comp rx=0 tx=0).stop 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- 172.21.15.73:0/1278182025 shutdown_connections 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.363 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 --2- 172.21.15.73:0/1278182025 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6ef550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- 172.21.15.73:0/1278182025 >> 172.21.15.73:0/1278182025 conn(0x55794c6eb680 msgr2=0x55794c6eba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- 172.21.15.73:0/1278182025 shutdown_connections 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- 172.21.15.73:0/1278182025 wait complete. 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.358+0000 7fee12b80740 1 -- start start 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 Processor -- start 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6f7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55794c6fcf80 con 0x55794c6fa350 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee0db2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6f7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55794c5dcae0 con 0x55794c6efa90 2024-06-06T07:24:46.364 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee0db2c640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6f7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57890/0 (socket says 172.21.15.73:57890) 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee0db2c640 1 -- 172.21.15.73:0/2722948693 learned_addr learned my addr 172.21.15.73:0/2722948693 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55794c6fd100 con 0x55794c6ef170 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee0e32d640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.359+0000 7fee12b80740 10 client.0 _ll_register_callbacks cb 0x55794c6159b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 -- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 msgr2=0x55794c6f7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6f7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 -- 172.21.15.73:0/2722948693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 msgr2=0x55794c6f7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6f7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 -- 172.21.15.73:0/2722948693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c367210 con 0x55794c6efa90 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f78b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ed130 tx=0x7fedf8005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b59f3bba7b9a7ae6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fedf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:24:46.365 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8049050 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fedf80b8570 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee0001b380 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fedf8031d10 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee12b80740 10 client.26957 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.360+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55794c433620 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf80bc600 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fedf803bbe0 con 0x55794c6efa90 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 10 client.26957 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 10 client.26957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 20 client.26957 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c70a530 0x55794c70c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 4 client.26957 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fee0001b380 con 0x55794c70a530 2024-06-06T07:24:46.366 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.361+0000 7fee0db2c640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c70a530 0x55794c70c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.362+0000 7fedf17fa640 10 client.26957 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:46.367 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.362+0000 7fee0db2c640 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c70a530 0x55794c70c9f0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c74ee1f4fe3f85 server_cookie=d4b29a37608e6535 in_seq=0 out_seq=0 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.362+0000 7fedf17fa640 1 -- 172.21.15.73:0/2722948693 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fee0001b380 con 0x55794c70a530 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.362+0000 7fedf17fa640 10 client.26957 handle_command_reply: tid=0 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.362+0000 7fee12b80740 1 client.26957 shutdown 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 msgr2=0x55794c6f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f78b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fedf80ed130 tx=0x7fedf8005c00 comp rx=0 tx=0).stop 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 shutdown_connections 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c70a530 0x55794c70c9f0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6f7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6efa90 0x55794c6f78b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.368 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 --2- 172.21.15.73:0/2722948693 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6f7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fedf17fa640 0 client.26957 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 >> 172.21.15.73:0/2722948693 conn(0x55794c6eb680 msgr2=0x55794c6f1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 shutdown_connections 2024-06-06T07:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 1 -- 172.21.15.73:0/2722948693 wait complete. 2024-06-06T07:24:46.369 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.363+0000 7fee12b80740 20 client.26957 trim_cache size 0 max 16384 2024-06-06T07:24:46.390 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.387+0000 7fee12b80740 1 Processor -- start 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 -- start start 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6eaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6fa350 0x55794c712300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b10b0 con 0x55794c6efa90 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b2bf0 con 0x55794c6fa350 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee12b80740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b1780 con 0x55794c6ef170 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0d32b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0d32b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54276/0 (socket says 172.21.15.73:54276) 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0d32b640 1 -- 172.21.15.73:0/2112262716 learned_addr learned my addr 172.21.15.73:0/2112262716 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0e32d640 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6fa350 0x55794c712300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.391 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0d32b640 1 -- 172.21.15.73:0/2112262716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 msgr2=0x55794c6eaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.388+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6eaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fee0d32b640 1 -- 172.21.15.73:0/2112262716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6fa350 msgr2=0x55794c712300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6fa350 0x55794c712300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fee0d32b640 1 -- 172.21.15.73:0/2112262716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55794c6fce60 con 0x55794c6efa90 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fee0d32b640 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fedf80bbc00 tx=0x7fedf8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bdeec34e181a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fedf37fe640 1 -- 172.21.15.73:0/2112262716 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf8049040 con 0x55794c6efa90 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fedf37fe640 1 -- 172.21.15.73:0/2112262716 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf80bc600 con 0x55794c6efa90 2024-06-06T07:24:46.392 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.389+0000 7fedf37fe640 1 -- 172.21.15.73:0/2112262716 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedf80bbf00 con 0x55794c6efa90 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- 172.21.15.73:0/2112262716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 msgr2=0x55794c6f92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fedf80bbc00 tx=0x7fedf8005fd0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- 172.21.15.73:0/2112262716 shutdown_connections 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6fa350 0x55794c712300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6efa90 0x55794c6f92b0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.395 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 --2- 172.21.15.73:0/2112262716 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6ef170 0x55794c6eaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- 172.21.15.73:0/2112262716 >> 172.21.15.73:0/2112262716 conn(0x55794c6eb680 msgr2=0x55794c6f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- 172.21.15.73:0/2112262716 shutdown_connections 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- 172.21.15.73:0/2112262716 wait complete. 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 -- start start 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.390+0000 7fee12b80740 1 Processor -- start 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6ef170 0x55794c702c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6efa90 0x55794c6fdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b2520 con 0x55794c6fa350 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fee081b1e50 con 0x55794c6ef170 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fee081af0b0 con 0x55794c6efa90 2024-06-06T07:24:46.396 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee12b80740 10 client.0 _ll_register_callbacks cb 0x7fedf8005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54284/0 (socket says 172.21.15.73:54284) 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 -- 172.21.15.73:0/1706172999 learned_addr learned my addr 172.21.15.73:0/1706172999 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0d32b640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6efa90 0x55794c6fdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 -- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6efa90 msgr2=0x55794c6fdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6efa90 0x55794c6fdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 -- 172.21.15.73:0/1706172999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6ef170 msgr2=0x55794c702c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6ef170 0x55794c702c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 -- 172.21.15.73:0/1706172999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55794c77cae0 con 0x55794c6fa350 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.391+0000 7fee0e32d640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee00005c10 tx=0x7fee0001c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942dbb593c1957e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fedf17fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00022040 con 0x55794c6fa350 2024-06-06T07:24:46.397 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fee00004c90 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fedf80bc600 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fee0003b050 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fee12b80740 10 client.17751 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55794c77dc70 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.392+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fedf80bc600 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fee0001b8d0 con 0x55794c6fa350 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 10 client.17751 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 10 client.17751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 20 client.17751 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c77e060 0x55794c7804a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 4 client.17751 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55794c666f10 con 0x55794c77e060 2024-06-06T07:24:46.398 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee0db2c640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c77e060 0x55794c7804a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fedf17fa640 10 client.17751 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:46.399 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.393+0000 7fee0db2c640 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c77e060 0x55794c7804a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ecd9fc4a0325f32 server_cookie=1cfc0d875bc89f0d in_seq=0 out_seq=0 2024-06-06T07:24:46.497 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.496+0000 7fedf17fa640 1 -- 172.21.15.73:0/1706172999 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55794c666f10 con 0x55794c77e060 2024-06-06T07:24:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.496+0000 7fedf17fa640 10 client.17751 handle_command_reply: tid=0 2024-06-06T07:24:46.498 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.496+0000 7fee12b80740 1 client.17751 shutdown 2024-06-06T07:24:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 msgr2=0x55794c6fe6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fee00005c10 tx=0x7fee0001c000 comp rx=0 tx=0).stop 2024-06-06T07:24:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 shutdown_connections 2024-06-06T07:24:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55794c77e060 0x55794c7804a0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.500 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fedf17fa640 0 client.17751 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55794c6fa350 0x55794c6fe6b0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55794c6efa90 0x55794c6fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 --2- 172.21.15.73:0/1706172999 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55794c6ef170 0x55794c702c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.498+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 >> 172.21.15.73:0/1706172999 conn(0x55794c6eb680 msgr2=0x55794c6f9430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.499+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 shutdown_connections 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.499+0000 7fee12b80740 1 -- 172.21.15.73:0/1706172999 wait complete. 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.499+0000 7fee12b80740 20 client.17751 trim_cache size 0 max 16384 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:24:46.501 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:24:46.502 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:24:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.501+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 msgr2=0x7fedf40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.501+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 0x7fedf40a52e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fee0001f000 tx=0x7fee00020000 comp rx=0 tx=0).stop 2024-06-06T07:24:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.501+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 msgr2=0x7fee0819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.501+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 0x7fee0819a8b0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fedf802a930 tx=0x7fedf8003c10 comp rx=0 tx=0).stop 2024-06-06T07:24:46.504 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 shutdown_connections 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fedf40a2e20 0x7fedf40a52e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fee0819ec30 0x7fee0819f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee08108f00 0x7fee0819a8b0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 --2- 172.21.15.73:0/972810372 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fee08103e30 0x7fee0819a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 >> 172.21.15.73:0/972810372 conn(0x7fee080ff6b0 msgr2=0x7fee081019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.502+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 shutdown_connections 2024-06-06T07:24:46.505 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:46.503+0000 7fee0fd90640 1 -- 172.21.15.73:0/972810372 wait complete. 2024-06-06T07:24:46.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97018 8.75 MB/sec execute 471 sec latency 92.927 ms 2024-06-06T07:24:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:47 smithi170 ceph-mon[36108]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-06-06T07:24:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:47 smithi073 ceph-mon[32303]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-06-06T07:24:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:47 smithi136 ceph-mon[35901]: pgmap v746: 129 pgs: 129 active+clean; 1.4 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 6.1 MiB/s wr, 280 op/s 2024-06-06T07:24:47.835 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97185 8.75 MB/sec execute 472 sec latency 101.446 ms 2024-06-06T07:24:48.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97336 8.75 MB/sec execute 473 sec latency 75.696 ms 2024-06-06T07:24:49.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:48 smithi073 ceph-mon[32303]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 436 op/s 2024-06-06T07:24:49.051 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:48 smithi136 ceph-mon[35901]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 436 op/s 2024-06-06T07:24:49.097 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:48 smithi170 ceph-mon[36108]: pgmap v747: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 436 op/s 2024-06-06T07:24:49.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97503 8.74 MB/sec execute 474 sec latency 75.462 ms 2024-06-06T07:24:50.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97680 8.75 MB/sec execute 475 sec latency 65.963 ms 2024-06-06T07:24:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:51 smithi073 ceph-mon[32303]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 301 op/s 2024-06-06T07:24:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:51 smithi136 ceph-mon[35901]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 301 op/s 2024-06-06T07:24:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:51 smithi170 ceph-mon[36108]: pgmap v748: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.2 MiB/s wr, 301 op/s 2024-06-06T07:24:51.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 97818 8.74 MB/sec execute 476 sec latency 131.616 ms 2024-06-06T07:24:52.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98010 8.75 MB/sec execute 477 sec latency 74.648 ms 2024-06-06T07:24:53.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:53 smithi073 ceph-mon[32303]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-06-06T07:24:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:53 smithi136 ceph-mon[35901]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-06-06T07:24:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:53 smithi170 ceph-mon[36108]: pgmap v749: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.1 MiB/s wr, 298 op/s 2024-06-06T07:24:53.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98214 8.76 MB/sec execute 478 sec latency 101.574 ms 2024-06-06T07:24:54.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98376 8.76 MB/sec execute 479 sec latency 83.844 ms 2024-06-06T07:24:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:55 smithi073 ceph-mon[32303]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 422 op/s 2024-06-06T07:24:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:55 smithi136 ceph-mon[35901]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 422 op/s 2024-06-06T07:24:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:55 smithi170 ceph-mon[36108]: pgmap v750: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.5 MiB/s wr, 422 op/s 2024-06-06T07:24:55.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98534 8.76 MB/sec execute 480 sec latency 76.927 ms 2024-06-06T07:24:56.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98705 8.77 MB/sec execute 481 sec latency 127.677 ms 2024-06-06T07:24:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:57 smithi073 ceph-mon[32303]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-06-06T07:24:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:57 smithi136 ceph-mon[35901]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-06-06T07:24:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:57 smithi170 ceph-mon[36108]: pgmap v751: 129 pgs: 129 active+clean; 1.4 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 4.6 MiB/s wr, 286 op/s 2024-06-06T07:24:57.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 98877 8.77 MB/sec execute 482 sec latency 145.070 ms 2024-06-06T07:24:58.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99038 8.77 MB/sec execute 483 sec latency 80.669 ms 2024-06-06T07:24:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:24:59 smithi073 ceph-mon[32303]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:24:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:24:59 smithi136 ceph-mon[35901]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:24:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:24:59 smithi170 ceph-mon[36108]: pgmap v752: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 7.1 MiB/s wr, 409 op/s 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 -- 172.21.15.73:0/3899931846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 msgr2=0x7fd934106ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 --2- 172.21.15.73:0/3899931846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd934106ca0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009cd0 tx=0x7fd92c02a8e0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 -- 172.21.15.73:0/3899931846 shutdown_connections 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 --2- 172.21.15.73:0/3899931846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9341012c0 0x7fd93410d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 --2- 172.21.15.73:0/3899931846 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd934100900 0x7fd934100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 --2- 172.21.15.73:0/3899931846 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd934106ca0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.643+0000 7fd939ab2640 1 -- 172.21.15.73:0/3899931846 >> 172.21.15.73:0/3899931846 conn(0x7fd9340779f0 msgr2=0x7fd934077e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.644+0000 7fd939ab2640 1 -- 172.21.15.73:0/3899931846 shutdown_connections 2024-06-06T07:24:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.644+0000 7fd939ab2640 1 -- 172.21.15.73:0/3899931846 wait complete. 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.644+0000 7fd939ab2640 1 Processor -- start 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.644+0000 7fd939ab2640 1 -- start start 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd934100900 0x7fd93419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9341012c0 0x7fd93419a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd934110b40 con 0x7fd934100900 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341109c0 con 0x7fd9341012c0 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd939ab2640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd934110cc0 con 0x7fd9341068c0 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd934100900 0x7fd93419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:60360/0 (socket says 172.21.15.73:60360) 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 -- 172.21.15.73:0/4208618507 learned_addr learned my addr 172.21.15.73:0/4208618507 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd932ffd640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9341012c0 0x7fd93419a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 -- 172.21.15.73:0/4208618507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9341012c0 msgr2=0x7fd93419a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9341012c0 0x7fd93419a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.645+0000 7fd933fff640 1 -- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd934100900 msgr2=0x7fd93419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd933fff640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd934100900 0x7fd93419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd933fff640 1 -- 172.21.15.73:0/4208618507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd93419f420 con 0x7fd9341068c0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd933fff640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9b0 tx=0x7fd92400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd930ff9640 1 -- 172.21.15.73:0/4208618507 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92400cd80 con 0x7fd9341068c0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd93419f710 con 0x7fd9341068c0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9341a6fa0 con 0x7fd9341068c0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd930ff9640 1 -- 172.21.15.73:0/4208618507 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9240035f0 con 0x7fd9341068c0 2024-06-06T07:24:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.646+0000 7fd930ff9640 1 -- 172.21.15.73:0/4208618507 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd924010690 con 0x7fd9341068c0 2024-06-06T07:24:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.649+0000 7fd93c7bd740 1 Processor -- start 2024-06-06T07:24:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.650+0000 7fd930ff9640 1 -- 172.21.15.73:0/4208618507 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fd9240037a0 con 0x7fd9341068c0 2024-06-06T07:24:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 -- start start 2024-06-06T07:24:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f54740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec55eab4f0 con 0x55ec55f47170 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec55e49870 con 0x55ec55f47a90 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec55ea74b0 con 0x55ec55f52350 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57542/0 (socket says 172.21.15.73:57542) 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd932ffd640 1 -- 172.21.15.73:0/2193319587 learned_addr learned my addr 172.21.15.73:0/2193319587 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.651+0000 7fd9337fe640 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd930ff9640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 0x7fd9200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 -- 172.21.15.73:0/2193319587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 msgr2=0x55ec55f54740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f54740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 -- 172.21.15.73:0/2193319587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 msgr2=0x55ec55f51e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f51e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 -- 172.21.15.73:0/2193319587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec55e659b0 con 0x55ec55f47170 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd933fff640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 0x7fd9200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009bb0 tx=0x7fd91c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96dccf549a1aadc server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd929ffb640 1 -- 172.21.15.73:0/2193319587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c004030 con 0x55ec55f47170 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.652+0000 7fd929ffb640 1 -- 172.21.15.73:0/2193319587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0041d0 con 0x55ec55f47170 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd930ff9640 1 -- 172.21.15.73:0/4208618507 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fd9240cfb50 con 0x7fd9341068c0 2024-06-06T07:24:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd929ffb640 1 -- 172.21.15.73:0/2193319587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0126b0 con 0x55ec55f47170 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2193319587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 msgr2=0x55ec55f47550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009bb0 tx=0x7fd91c0117b0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd933fff640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 0x7fd9200a52e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9240ccf40 tx=0x7fd92409cf90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2193319587 shutdown_connections 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f54740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f51e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2193319587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47170 0x55ec55f47550 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.653+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2193319587 >> 172.21.15.73:0/2193319587 conn(0x55ec55f43680 msgr2=0x55ec55f43a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2193319587 shutdown_connections 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2193319587 wait complete. 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- start start 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 Processor -- start 2024-06-06T07:24:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47170 0x55ec55f4f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f4fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ec55f54f80 con 0x55ec55f52350 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55ec55e34ae0 con 0x55ec55f47a90 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55ec55f55100 con 0x55ec55f47170 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd93c7bd740 10 client.0 _ll_register_callbacks cb 0x55ec55e6d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.654+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47170 0x55ec55f4f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f4fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:46648/0 (socket says 172.21.15.73:46648) 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 -- 172.21.15.73:0/1360082357 learned_addr learned my addr 172.21.15.73:0/1360082357 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 -- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47170 msgr2=0x55ec55f4f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47170 0x55ec55f4f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 -- 172.21.15.73:0/1360082357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 msgr2=0x55ec55f4fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f4fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 -- 172.21.15.73:0/1360082357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec55bbf210 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd9337fe640 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009cd0 tx=0x7fd92c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de4c27563a281704 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd92c003970 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91c0041d0 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.655+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fd92c003b10 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd92c053050 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 10 client.26972 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55ec55c8b620 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0041d0 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fd92c04e030 con 0x55ec55f47a90 2024-06-06T07:24:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 10 client.26972 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 10 client.26972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 20 client.26972 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55f6ad80 0x55ec55f6d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 4 client.26972 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec55ebef10 con 0x55ec55f6ad80 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.656+0000 7fd932ffd640 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55f6ad80 0x55ec55f6d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.657+0000 7fd932ffd640 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55f6ad80 0x55ec55f6d170 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ee5d68d2b7558 server_cookie=2c24404a4f6123f7 in_seq=0 out_seq=0 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.657+0000 7fd912ffd640 10 client.26972 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.657+0000 7fd912ffd640 1 -- 172.21.15.73:0/1360082357 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec55ebef10 con 0x55ec55f6ad80 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.657+0000 7fd912ffd640 10 client.26972 handle_command_reply: tid=0 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.657+0000 7fd93c7bd740 1 client.26972 shutdown 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 msgr2=0x55ec55f4f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fd92c009cd0 tx=0x7fd92c031850 comp rx=0 tx=0).stop 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 shutdown_connections 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55f6ad80 0x55ec55f6d170 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f4fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47a90 0x55ec55f4f7f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/1360082357 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47170 0x55ec55f4f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 >> 172.21.15.73:0/1360082357 conn(0x55ec55f43680 msgr2=0x55ec55f49400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd912ffd640 0 client.26972 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 shutdown_connections 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 1 -- 172.21.15.73:0/1360082357 wait complete. 2024-06-06T07:24:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.658+0000 7fd93c7bd740 20 client.26972 trim_cache size 0 max 16384 2024-06-06T07:24:59.682 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.680+0000 7fd93c7bd740 1 Processor -- start 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 -- start start 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f67110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 0x55ec55f4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341acc80 con 0x55ec55f52350 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ae7c0 con 0x55ec55f47170 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ad350 con 0x55ec55f47a90 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f67110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57568/0 (socket says 172.21.15.73:57568) 2024-06-06T07:24:59.683 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd932ffd640 1 -- 172.21.15.73:0/2279671063 learned_addr learned my addr 172.21.15.73:0/2279671063 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.681+0000 7fd9337fe640 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 0x55ec55f4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 -- 172.21.15.73:0/2279671063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 msgr2=0x55ec55f4fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 0x55ec55f4fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 -- 172.21.15.73:0/2279671063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 msgr2=0x55ec55f67110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f67110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 -- 172.21.15.73:0/2279671063 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec55f48820 con 0x55ec55f52350 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd9337fe640 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 0x55ec55f4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd932ffd640 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd91c003be0 tx=0x7fd91c003b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10c8008933c7b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd9297fa640 1 -- 172.21.15.73:0/2279671063 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c005240 con 0x55ec55f52350 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd9297fa640 1 -- 172.21.15.73:0/2279671063 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0053e0 con 0x55ec55f52350 2024-06-06T07:24:59.684 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.682+0000 7fd9297fa640 1 -- 172.21.15.73:0/2279671063 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0129d0 con 0x55ec55f52350 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2279671063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 msgr2=0x55ec55f72a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fd91c003be0 tx=0x7fd91c003b50 comp rx=0 tx=0).stop 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2279671063 shutdown_connections 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f52350 0x55ec55f72a80 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f47a90 0x55ec55f4fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/2279671063 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f67110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2279671063 >> 172.21.15.73:0/2279671063 conn(0x55ec55f43680 msgr2=0x55ec55f4a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2279671063 shutdown_connections 2024-06-06T07:24:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- 172.21.15.73:0/2279671063 wait complete. 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.683+0000 7fd93c7bd740 1 -- start start 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 Processor -- start 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47a90 0x55ec55f55b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f56080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ae0f0 con 0x55ec55f47a90 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341ada20 con 0x55ec55f47170 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fd9341aac80 con 0x55ec55f52350 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd93c7bd740 10 client.0 _ll_register_callbacks cb 0x7fd91c005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd932ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:46668/0 (socket says 172.21.15.73:46668) 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd932ffd640 1 -- 172.21.15.73:0/242602899 learned_addr learned my addr 172.21.15.73:0/242602899 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:24:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd932ffd640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f56080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.684+0000 7fd9337fe640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47a90 0x55ec55f55b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 -- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 msgr2=0x55ec55f56080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f56080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 -- 172.21.15.73:0/242602899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47a90 msgr2=0x55ec55f55b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47a90 0x55ec55f55b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 -- 172.21.15.73:0/242602899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec55f565c0 con 0x55ec55f47170 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd932ffd640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd91c003e80 tx=0x7fd91c005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caf237db9c9cd89c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd91c009e80 con 0x55ec55f47170 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(142..142 src has 1..142) ==== 9554+0+0 (secure 0 0 0) 0x7fd91c012450 con 0x55ec55f47170 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd91c0053e0 con 0x55ec55f47170 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fd91c012650 con 0x55ec55f47170 2024-06-06T07:24:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd93c7bd740 10 client.26978 fetch_fsmap learned FSMap version 84 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.685+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55ec55fd5770 con 0x55ec55f47170 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd91c0053e0 con 0x55ec55f47170 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fd91c025040 con 0x55ec55f47170 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 10 client.26978 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 10 client.26978 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 20 client.26978 populate_metadata read hostname 'smithi073' 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55fd5c40 0x55ec55fd8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 4 client.26978 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec55eb8c20 con 0x55ec55fd5c40 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd9337fe640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55fd5c40 0x55ec55fd8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.686+0000 7fd9337fe640 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55fd5c40 0x55ec55fd8030 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d43a99f55873c1 server_cookie=9fca76f3cb67adfb in_seq=0 out_seq=0 2024-06-06T07:24:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.687+0000 7fd912ffd640 10 client.26978 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd912ffd640 1 -- 172.21.15.73:0/242602899 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ec55eb8c20 con 0x55ec55fd5c40 2024-06-06T07:24:59.812 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd912ffd640 10 client.26978 handle_command_reply: tid=0 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd93c7bd740 1 client.26978 shutdown 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 msgr2=0x55ec55f5a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd91c003e80 tx=0x7fd91c005f90 comp rx=0 tx=0).stop 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 shutdown_connections 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.811+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55ec55fd5c40 0x55ec55fd8030 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55ec55f52350 0x55ec55f56080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ec55f47a90 0x55ec55f55b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 --2- 172.21.15.73:0/242602899 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55ec55f47170 0x55ec55f5a7d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 >> 172.21.15.73:0/242602899 conn(0x55ec55f43680 msgr2=0x55ec55f48b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd912ffd640 0 client.26978 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 shutdown_connections 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 1 -- 172.21.15.73:0/242602899 wait complete. 2024-06-06T07:24:59.813 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.812+0000 7fd93c7bd740 20 client.26978 trim_cache size 0 max 16384 2024-06-06T07:24:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:24:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:24:59.814 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 msgr2=0x7fd9200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 0x7fd9200a52e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd9240ccf40 tx=0x7fd92409cf90 comp rx=0 tx=0).stop 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 msgr2=0x7fd93419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd92400e9b0 tx=0x7fd92400ee70 comp rx=0 tx=0).stop 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 shutdown_connections 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fd9200a2e20 0x7fd9200a52e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.815 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fd9341068c0 0x7fd93419ecf0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fd9341012c0 0x7fd93419a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 --2- 172.21.15.73:0/4208618507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd934100900 0x7fd93419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:24:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 >> 172.21.15.73:0/4208618507 conn(0x7fd9340779f0 msgr2=0x7fd93410c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:24:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 shutdown_connections 2024-06-06T07:24:59.816 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:24:59.813+0000 7fd939ab2640 1 -- 172.21.15.73:0/4208618507 wait complete. 2024-06-06T07:24:59.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99196 8.77 MB/sec execute 484 sec latency 102.734 ms 2024-06-06T07:25:00.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99329 8.77 MB/sec execute 485 sec latency 91.405 ms 2024-06-06T07:25:01.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:01 smithi073 ceph-mon[32303]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-06-06T07:25:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:01 smithi136 ceph-mon[35901]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-06-06T07:25:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:01 smithi170 ceph-mon[36108]: pgmap v753: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 255 op/s 2024-06-06T07:25:01.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99488 8.77 MB/sec execute 486 sec latency 68.836 ms 2024-06-06T07:25:02.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99665 8.77 MB/sec execute 487 sec latency 168.384 ms 2024-06-06T07:25:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:03 smithi073 ceph-mon[32303]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 249 op/s 2024-06-06T07:25:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:03 smithi136 ceph-mon[35901]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 249 op/s 2024-06-06T07:25:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:03 smithi170 ceph-mon[36108]: pgmap v754: 129 pgs: 129 active+clean; 1.5 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 5.0 MiB/s wr, 249 op/s 2024-06-06T07:25:03.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99816 8.77 MB/sec execute 488 sec latency 89.696 ms 2024-06-06T07:25:04.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 99915 8.76 MB/sec execute 489 sec latency 170.537 ms 2024-06-06T07:25:05.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:05 smithi073 ceph-mon[32303]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.2 MiB/s wr, 414 op/s 2024-06-06T07:25:05.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:05 smithi073 ceph-mon[32303]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T07:25:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:05 smithi136 ceph-mon[35901]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.2 MiB/s wr, 414 op/s 2024-06-06T07:25:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:05 smithi136 ceph-mon[35901]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T07:25:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:05 smithi170 ceph-mon[36108]: pgmap v755: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 8.2 MiB/s wr, 414 op/s 2024-06-06T07:25:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:05 smithi170 ceph-mon[36108]: osdmap e143: 12 total, 12 up, 12 in 2024-06-06T07:25:05.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100041 8.76 MB/sec execute 490 sec latency 235.643 ms 2024-06-06T07:25:06.836 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100219 8.78 MB/sec execute 491 sec latency 65.973 ms 2024-06-06T07:25:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:07 smithi073 ceph-mon[32303]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 349 op/s 2024-06-06T07:25:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:07 smithi136 ceph-mon[35901]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 349 op/s 2024-06-06T07:25:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:07 smithi170 ceph-mon[36108]: pgmap v757: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 7.0 MiB/s wr, 349 op/s 2024-06-06T07:25:07.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100346 8.77 MB/sec execute 492 sec latency 129.669 ms 2024-06-06T07:25:08.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100475 8.75 MB/sec execute 493 sec latency 165.212 ms 2024-06-06T07:25:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:09 smithi073 ceph-mon[32303]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 342 op/s 2024-06-06T07:25:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:09 smithi136 ceph-mon[35901]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 342 op/s 2024-06-06T07:25:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:09 smithi170 ceph-mon[36108]: pgmap v758: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.7 MiB/s wr, 342 op/s 2024-06-06T07:25:09.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100617 8.74 MB/sec execute 494 sec latency 73.064 ms 2024-06-06T07:25:10.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100756 8.73 MB/sec execute 495 sec latency 82.324 ms 2024-06-06T07:25:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:11 smithi073 ceph-mon[32303]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:11 smithi136 ceph-mon[35901]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:11 smithi170 ceph-mon[36108]: pgmap v759: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:11.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 100907 8.72 MB/sec execute 496 sec latency 111.984 ms 2024-06-06T07:25:12.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:12 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:25:12.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:12 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:25:12.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:12 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:25:12.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101034 8.72 MB/sec execute 497 sec latency 110.084 ms 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 -- 172.21.15.73:0/3404802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 msgr2=0x7fe3f0101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/3404802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0101e00 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009890 tx=0x7fe3e002a380 comp rx=0 tx=0).stop 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 -- 172.21.15.73:0/3404802240 shutdown_connections 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/3404802240 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f0102340 0x7fe3f010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/3404802240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0101e00 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/3404802240 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0107990 0x7fe3f0107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 -- 172.21.15.73:0/3404802240 >> 172.21.15.73:0/3404802240 conn(0x7fe3f00fd450 msgr2=0x7fe3f00ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 -- 172.21.15.73:0/3404802240 shutdown_connections 2024-06-06T07:25:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.946+0000 7fe3f7b42640 1 -- 172.21.15.73:0/3404802240 wait complete. 2024-06-06T07:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 Processor -- start 2024-06-06T07:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 -- start start 2024-06-06T07:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0102340 0x7fe3f01987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f0107990 0x7fe3f019cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0111c90 con 0x7fe3f0101980 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0111b10 con 0x7fe3f0102340 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f7b42640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f0111e10 con 0x7fe3f0107990 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.947+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34090/0 (socket says 172.21.15.73:34090) 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 -- 172.21.15.73:0/1021956751 learned_addr learned my addr 172.21.15.73:0/1021956751 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0102340 0x7fe3f01987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 -- 172.21.15.73:0/1021956751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f0107990 msgr2=0x7fe3f019cae0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f0107990 0x7fe3f019cae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 -- 172.21.15.73:0/1021956751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0102340 msgr2=0x7fe3f01987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0102340 0x7fe3f01987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 -- 172.21.15.73:0/1021956751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f019d210 con 0x7fe3f0101980 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec013f40 tx=0x7fe3ec013f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3e6ffd640 1 -- 172.21.15.73:0/1021956751 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0110d0 con 0x7fe3f0101980 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f019d4a0 con 0x7fe3f0101980 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3e6ffd640 1 -- 172.21.15.73:0/1021956751 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3ec00f850 con 0x7fe3f0101980 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.948+0000 7fe3e6ffd640 1 -- 172.21.15.73:0/1021956751 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec01c720 con 0x7fe3f0101980 2024-06-06T07:25:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.949+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f0073bd0 con 0x7fe3f0101980 2024-06-06T07:25:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.952+0000 7fe3e6ffd640 1 -- 172.21.15.73:0/1021956751 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fe3ec00fa00 con 0x7fe3f0101980 2024-06-06T07:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.953+0000 7fe3e6ffd640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 0x7fe3dc0a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.953+0000 7fe3fa851740 1 Processor -- start 2024-06-06T07:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.953+0000 7fe3e6ffd640 1 -- 172.21.15.73:0/1021956751 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fe3ec0d6f70 con 0x7fe3f0101980 2024-06-06T07:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 0x7fe3dc0a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 -- start start 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea3696550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea36a1350 0x555ea36a3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ea35fa4f0 con 0x555ea3696a90 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555ea3598870 con 0x555ea36a1350 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3fa851740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ea35f64b0 con 0x555ea3696170 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.954+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 0x7fe3dc0a5360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0000c00 tx=0x7fe3e0005c50 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34102/0 (socket says 172.21.15.73:34102) 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 -- 172.21.15.73:0/2679660439 learned_addr learned my addr 172.21.15.73:0/2679660439 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f60df640 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea36a1350 0x555ea36a3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 -- 172.21.15.73:0/2679660439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 msgr2=0x555ea3696550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea3696550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 -- 172.21.15.73:0/2679660439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea36a1350 msgr2=0x555ea36a3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea36a1350 0x555ea36a3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 -- 172.21.15.73:0/2679660439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea35b49b0 con 0x555ea3696a90 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009890 tx=0x7fe3e003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c0e18c532c25326 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3dbfff640 1 -- 172.21.15.73:0/2679660439 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e004e070 con 0x555ea3696a90 2024-06-06T07:25:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.955+0000 7fe3dbfff640 1 -- 172.21.15.73:0/2679660439 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3e0045040 con 0x555ea3696a90 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3dbfff640 1 -- 172.21.15.73:0/2679660439 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e0037040 con 0x555ea3696a90 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 -- 172.21.15.73:0/2679660439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 msgr2=0x555ea36a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009890 tx=0x7fe3e003b000 comp rx=0 tx=0).stop 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 -- 172.21.15.73:0/2679660439 shutdown_connections 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea36a1350 0x555ea36a3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696a90 0x555ea36a0e10 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 --2- 172.21.15.73:0/2679660439 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea3696550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.956+0000 7fe3fa851740 1 -- 172.21.15.73:0/2679660439 >> 172.21.15.73:0/2679660439 conn(0x555ea3692680 msgr2=0x555ea3692a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.957+0000 7fe3fa851740 1 -- 172.21.15.73:0/2679660439 shutdown_connections 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.957+0000 7fe3fa851740 1 -- 172.21.15.73:0/2679660439 wait complete. 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.957+0000 7fe3fa851740 1 -- start start 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.957+0000 7fe3fa851740 1 Processor -- start 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696a90 0x555ea369e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea369ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x555ea36a3f20 con 0x555ea36a1350 2024-06-06T07:25:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x555ea3583ae0 con 0x555ea3696170 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x555ea36a40a0 con 0x555ea3696a90 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3fa851740 10 client.0 _ll_register_callbacks cb 0x555ea35bc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea369ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47022/0 (socket says 172.21.15.73:47022) 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f58de640 1 -- 172.21.15.73:0/1874041418 learned_addr learned my addr 172.21.15.73:0/1874041418 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 -- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696a90 msgr2=0x555ea369e7b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696a90 0x555ea369e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 -- 172.21.15.73:0/1874041418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 msgr2=0x555ea369e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f60df640 1 -- 172.21.15.73:0/1874041418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea330e210 con 0x555ea36a1350 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.958+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea369ecf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005fd0 tx=0x7fe3e8010f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a41cdc0a37adf9 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e800ec80 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fe3e800ee20 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3e0045040 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe3e8023050 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3fa851740 10 client.17781 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x555ea33da620 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.959+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3e0045040 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fe3e800b040 con 0x555ea36a1350 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 10 client.17781 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 10 client.17781 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 20 client.17781 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea36b3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 4 client.17781 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ea360df10 con 0x555ea36b15f0 2024-06-06T07:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea36b3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3d97fa640 10 client.17781 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.960+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea36b3a10 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2669600e869dda64 server_cookie=e36c642a349a6f4 in_seq=0 out_seq=0 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1874041418 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ea360df10 con 0x555ea36b15f0 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3d97fa640 10 client.17781 handle_command_reply: tid=0 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 client.17781 shutdown 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 msgr2=0x555ea369ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea369ecf0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8005fd0 tx=0x7fe3e8010f80 comp rx=0 tx=0).stop 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 shutdown_connections 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea36b3a10 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea369ecf0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3d97fa640 0 client.17781 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696a90 0x555ea369e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1874041418 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696170 0x555ea369e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 >> 172.21.15.73:0/1874041418 conn(0x555ea3692680 msgr2=0x555ea36983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 shutdown_connections 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 1 -- 172.21.15.73:0/1874041418 wait complete. 2024-06-06T07:25:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.961+0000 7fe3fa851740 20 client.17781 trim_cache size 0 max 16384 2024-06-06T07:25:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.990+0000 7fe3fa851740 1 Processor -- start 2024-06-06T07:25:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.990+0000 7fe3fa851740 1 -- start start 2024-06-06T07:25:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.990+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea3698230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea3698770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f50dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea3698230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f50dd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea3698230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:47030/0 (socket says 172.21.15.73:47030) 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f60df640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea3698770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f58de640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:44630/0 (socket says 172.21.15.73:44630) 2024-06-06T07:25:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f60df640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea3698770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34134/0 (socket says 172.21.15.73:34134) 2024-06-06T07:25:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3f50dd640 1 -- 172.21.15.73:0/4125520123 learned_addr learned my addr 172.21.15.73:0/4125520123 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01acdf0 con 0x555ea36a1350 2024-06-06T07:25:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ae930 con 0x555ea3696a90 2024-06-06T07:25:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.991+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ad4c0 con 0x555ea3696170 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 -- 172.21.15.73:0/4125520123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 msgr2=0x555ea3698230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea3698230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 -- 172.21.15.73:0/4125520123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 msgr2=0x555ea3698770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea3698770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 -- 172.21.15.73:0/4125520123 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ea36b3d80 con 0x555ea3696170 2024-06-06T07:25:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3f58de640 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec011b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8e65aafd3021852 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3db7fe640 1 -- 172.21.15.73:0/4125520123 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0abda0 con 0x555ea3696170 2024-06-06T07:25:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.992+0000 7fe3db7fe640 1 -- 172.21.15.73:0/4125520123 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3ec0a14c0 con 0x555ea3696170 2024-06-06T07:25:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.993+0000 7fe3db7fe640 1 -- 172.21.15.73:0/4125520123 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3ec0a70d0 con 0x555ea3696170 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 msgr2=0x555ea369eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec000c00 tx=0x7fe3ec011b40 comp rx=0 tx=0).stop 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 shutdown_connections 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea36a1350 0x555ea3698770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea3698230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 --2- 172.21.15.73:0/4125520123 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea3696170 0x555ea369eb10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.994+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 >> 172.21.15.73:0/4125520123 conn(0x555ea3692680 msgr2=0x555ea3697b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.996+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 shutdown_connections 2024-06-06T07:25:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.996+0000 7fe3fa851740 1 -- 172.21.15.73:0/4125520123 wait complete. 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.996+0000 7fe3fa851740 1 -- start start 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.996+0000 7fe3fa851740 1 Processor -- start 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696170 0x555ea36a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea36a48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01ae260 con 0x555ea3696170 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01adb90 con 0x555ea3696a90 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f01aadf0 con 0x555ea36a1350 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3f60df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3f60df640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:44634/0 (socket says 172.21.15.73:44634) 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3f60df640 1 -- 172.21.15.73:0/1539027516 learned_addr learned my addr 172.21.15.73:0/1539027516 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.997+0000 7fe3fa851740 10 client.0 _ll_register_callbacks cb 0x7fe3e8009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f60df640 1 -- 172.21.15.73:0/1539027516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 msgr2=0x555ea36a48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea36a48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea36a48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f60df640 1 -- 172.21.15.73:0/1539027516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696170 msgr2=0x555ea36a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696170 0x555ea36a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f60df640 1 -- 172.21.15.73:0/1539027516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ea372b5e0 con 0x555ea36a1350 2024-06-06T07:25:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.998+0000 7fe3f50dd640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea36a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:25:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.999+0000 7fe3f60df640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8002230 tx=0x7fe3e80175e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=343b3b2868658c15 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:13.000 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.999+0000 7fe3d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.999+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8003b70 con 0x555ea36a1350 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.999+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fe3e8003d10 con 0x555ea36a1350 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:12.999+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe3ec0a14c0 con 0x555ea36a1350 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.000+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fe3e8034050 con 0x555ea36a1350 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.000+0000 7fe3fa851740 10 client.27166 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:13.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.000+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x555ea372c8d0 con 0x555ea36a1350 2024-06-06T07:25:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.001+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3ec0a14c0 con 0x555ea36a1350 2024-06-06T07:25:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.001+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fe3e802f030 con 0x555ea36a1350 2024-06-06T07:25:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.001+0000 7fe3fa851740 10 client.27166 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.001+0000 7fe3fa851740 10 client.27166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.002+0000 7fe3fa851740 20 client.27166 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.002+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea372f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.002+0000 7fe3fa851740 4 client.27166 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:25:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.002+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ea3607c20 con 0x555ea36b15f0 2024-06-06T07:25:13.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.004+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea372f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:13.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.005+0000 7fe3f58de640 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea372f120 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=640543511af99741 server_cookie=25f031b160dc9a60 in_seq=0 out_seq=0 2024-06-06T07:25:13.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.005+0000 7fe3d97fa640 10 client.27166 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.107+0000 7fe3d97fa640 1 -- 172.21.15.73:0/1539027516 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555ea3607c20 con 0x555ea36b15f0 2024-06-06T07:25:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.107+0000 7fe3d97fa640 10 client.27166 handle_command_reply: tid=0 2024-06-06T07:25:13.108 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.107+0000 7fe3fa851740 1 client.27166 shutdown 2024-06-06T07:25:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 msgr2=0x555ea36a4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:13.109 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fe3e8002230 tx=0x7fe3e80175e0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 shutdown_connections 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x555ea36b15f0 0x555ea372f120 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x555ea36a1350 0x555ea36a4e50 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3d97fa640 0 client.27166 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x555ea3696a90 0x555ea36a48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.108+0000 7fe3fa851740 1 --2- 172.21.15.73:0/1539027516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555ea3696170 0x555ea36a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.109+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 >> 172.21.15.73:0/1539027516 conn(0x555ea3692680 msgr2=0x555ea36983b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.109+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 shutdown_connections 2024-06-06T07:25:13.110 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.109+0000 7fe3fa851740 1 -- 172.21.15.73:0/1539027516 wait complete. 2024-06-06T07:25:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.109+0000 7fe3fa851740 20 client.27166 trim_cache size 0 max 16384 2024-06-06T07:25:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:25:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:25:13.111 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:25:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.110+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 msgr2=0x7fe3dc0a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:13.112 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.110+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 0x7fe3dc0a5360 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0000c00 tx=0x7fe3e0005c50 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.110+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 msgr2=0x7fe3f0198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.110+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec013f40 tx=0x7fe3ec013f70 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 shutdown_connections 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fe3dc0a2ea0 0x7fe3dc0a5360 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fe3f0107990 0x7fe3f019cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fe3f0102340 0x7fe3f01987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 --2- 172.21.15.73:0/1021956751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f0101980 0x7fe3f0198270 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 >> 172.21.15.73:0/1021956751 conn(0x7fe3f00fd450 msgr2=0x7fe3f00fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 shutdown_connections 2024-06-06T07:25:13.113 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:13.111+0000 7fe3f7b42640 1 -- 172.21.15.73:0/1021956751 wait complete. 2024-06-06T07:25:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:13 smithi073 ceph-mon[32303]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:13 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:25:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:13 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:25:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:13 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:25:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:13 smithi073 ceph-mon[32303]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T07:25:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:13 smithi136 ceph-mon[35901]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:13 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:25:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:13 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:25:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:13 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:25:13.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:13 smithi136 ceph-mon[35901]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T07:25:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:13 smithi170 ceph-mon[36108]: pgmap v760: 129 pgs: 129 active+clean; 1.5 GiB data, 8.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 6.9 MiB/s wr, 358 op/s 2024-06-06T07:25:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:13 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:25:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:13 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:25:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:13 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:25:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:13 smithi170 ceph-mon[36108]: osdmap e144: 12 total, 12 up, 12 in 2024-06-06T07:25:13.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101168 8.71 MB/sec execute 498 sec latency 109.955 ms 2024-06-06T07:25:14.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101354 8.72 MB/sec execute 499 sec latency 95.221 ms 2024-06-06T07:25:15.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:15 smithi073 ceph-mon[32303]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-06-06T07:25:15.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:15 smithi136 ceph-mon[35901]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-06-06T07:25:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:15 smithi170 ceph-mon[36108]: pgmap v762: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 312 op/s 2024-06-06T07:25:15.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101573 8.74 MB/sec execute 500 sec latency 568.001 ms 2024-06-06T07:25:16.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101756 8.74 MB/sec execute 501 sec latency 626.184 ms 2024-06-06T07:25:17.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:17 smithi073 ceph-mon[32303]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 310 op/s 2024-06-06T07:25:17.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:17 smithi136 ceph-mon[35901]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 310 op/s 2024-06-06T07:25:17.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:17 smithi170 ceph-mon[36108]: pgmap v763: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 5.3 MiB/s wr, 310 op/s 2024-06-06T07:25:17.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 101900 8.74 MB/sec execute 502 sec latency 134.506 ms 2024-06-06T07:25:18.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102062 8.74 MB/sec execute 503 sec latency 77.788 ms 2024-06-06T07:25:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:19 smithi073 ceph-mon[32303]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-06-06T07:25:19.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:19 smithi136 ceph-mon[35901]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-06-06T07:25:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:19 smithi170 ceph-mon[36108]: pgmap v764: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.9 MiB/s wr, 328 op/s 2024-06-06T07:25:19.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102229 8.75 MB/sec execute 504 sec latency 71.686 ms 2024-06-06T07:25:20.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102362 8.74 MB/sec execute 505 sec latency 343.409 ms 2024-06-06T07:25:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:21 smithi073 ceph-mon[32303]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:21 smithi136 ceph-mon[35901]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:21.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:21 smithi170 ceph-mon[36108]: pgmap v765: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:21.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102509 8.74 MB/sec execute 506 sec latency 111.322 ms 2024-06-06T07:25:22.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102628 8.73 MB/sec execute 507 sec latency 102.366 ms 2024-06-06T07:25:23.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:23 smithi073 ceph-mon[32303]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:23.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:23 smithi136 ceph-mon[35901]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:23 smithi170 ceph-mon[36108]: pgmap v766: 129 pgs: 129 active+clean; 1.5 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.7 MiB/s wr, 314 op/s 2024-06-06T07:25:23.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102802 8.74 MB/sec execute 508 sec latency 90.355 ms 2024-06-06T07:25:24.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 102972 8.74 MB/sec execute 509 sec latency 99.387 ms 2024-06-06T07:25:25.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:25 smithi073 ceph-mon[32303]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 407 op/s 2024-06-06T07:25:25.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:25 smithi136 ceph-mon[35901]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 407 op/s 2024-06-06T07:25:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:25 smithi170 ceph-mon[36108]: pgmap v767: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 407 op/s 2024-06-06T07:25:25.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103120 8.74 MB/sec execute 510 sec latency 89.673 ms 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 -- 172.21.15.73:0/2889966881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 msgr2=0x7fdb14106c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/2889966881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb14106c70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009900 tx=0x7fdb1002a3f0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 -- 172.21.15.73:0/2889966881 shutdown_connections 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/2889966881 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb14101290 0x7fdb1410d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/2889966881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb141008d0 0x7fdb14100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.245 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/2889966881 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb14106c70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.243+0000 7fdb1bdee640 1 -- 172.21.15.73:0/2889966881 >> 172.21.15.73:0/2889966881 conn(0x7fdb14077a20 msgr2=0x7fdb14077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.244+0000 7fdb1bdee640 1 -- 172.21.15.73:0/2889966881 shutdown_connections 2024-06-06T07:25:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.244+0000 7fdb1bdee640 1 -- 172.21.15.73:0/2889966881 wait complete. 2024-06-06T07:25:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.244+0000 7fdb1bdee640 1 Processor -- start 2024-06-06T07:25:26.246 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.244+0000 7fdb1bdee640 1 -- start start 2024-06-06T07:25:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 0x7fdb141986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.248 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14110b10 con 0x7fdb14101290 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14110990 con 0x7fdb14106890 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1bdee640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb14110c90 con 0x7fdb141008d0 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1a38b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:39136/0 (socket says 172.21.15.73:39136) 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 -- 172.21.15.73:0/3591678731 learned_addr learned my addr 172.21.15.73:0/3591678731 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1a38b640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57018/0 (socket says 172.21.15.73:57018) 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19389640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 0x7fdb141986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 -- 172.21.15.73:0/3591678731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 msgr2=0x7fdb1419ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 -- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 msgr2=0x7fdb141986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.249 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 0x7fdb141986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb19b8a640 1 -- 172.21.15.73:0/3591678731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1419d170 con 0x7fdb141008d0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.245+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb19389640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 0x7fdb141986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006010 tx=0x7fdb10004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb0affd640 1 -- 172.21.15.73:0/3591678731 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10048070 con 0x7fdb141008d0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb0affd640 1 -- 172.21.15.73:0/3591678731 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb1003ab20 con 0x7fdb141008d0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb0affd640 1 -- 172.21.15.73:0/3591678731 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb10035050 con 0x7fdb141008d0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1419d400 con 0x7fdb141008d0 2024-06-06T07:25:26.250 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.246+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb140fd790 con 0x7fdb141008d0 2024-06-06T07:25:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.249+0000 7fdb0affd640 1 -- 172.21.15.73:0/3591678731 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fdb1004b070 con 0x7fdb141008d0 2024-06-06T07:25:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 Processor -- start 2024-06-06T07:25:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 -- start start 2024-06-06T07:25:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086adf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.252 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564086a434f0 con 0x564086adf9d0 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5640869e1870 con 0x564086af2b60 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564086a3f4b0 con 0x564086adf0b0 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb0affd640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 0x7fdb000a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb0affd640 1 -- 172.21.15.73:0/3591678731 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fdb100f35c0 con 0x7fdb141008d0 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.250+0000 7fdb19b8a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.253 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 0x7fdb000a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44426/0 (socket says 172.21.15.73:44426) 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 -- 172.21.15.73:0/1571961502 learned_addr learned my addr 172.21.15.73:0/1571961502 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 -- 172.21.15.73:0/1571961502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 msgr2=0x564086adf490 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086adf490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 -- 172.21.15.73:0/1571961502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 msgr2=0x564086af4f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af4f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.254 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.251+0000 7fdb19b8a640 1 -- 172.21.15.73:0/1571961502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640869fd9b0 con 0x564086adf9d0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.252+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 0x7fdb000a52e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0099a0 tx=0x7fdb0c009500 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.252+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdb100c7db0 tx=0x7fdb1002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51811bdaa0be039 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.252+0000 7fdafffff640 1 -- 172.21.15.73:0/1571961502 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100bf670 con 0x564086adf9d0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.252+0000 7fdafffff640 1 -- 172.21.15.73:0/1571961502 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb100c3a20 con 0x564086adf9d0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.252+0000 7fdafffff640 1 -- 172.21.15.73:0/1571961502 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb100bbce0 con 0x564086adf9d0 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.253+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1571961502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 msgr2=0x564086af2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.258 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.253+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdb100c7db0 tx=0x7fdb1002a3f0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.253+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1571961502 shutdown_connections 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.253+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af2620 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1571961502 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086adf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1571961502 >> 172.21.15.73:0/1571961502 conn(0x564086adb680 msgr2=0x564086adba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1571961502 shutdown_connections 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1571961502 wait complete. 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.254+0000 7fdb1ebd8740 1 -- start start 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 Processor -- start 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086aefc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.259 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564086af54f0 con 0x564086adf9d0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5640869ccae0 con 0x564086af2b60 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x564086af5670 con 0x564086adf0b0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44440/0 (socket says 172.21.15.73:44440) 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb19389640 1 -- 172.21.15.73:0/4048521698 learned_addr learned my addr 172.21.15.73:0/4048521698 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb1ebd8740 10 client.0 _ll_register_callbacks cb 0x564086a059b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 -- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 msgr2=0x564086aefc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.255+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086aefc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086aefc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 -- 172.21.15.73:0/4048521698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 msgr2=0x564086af01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.260 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 -- 172.21.15.73:0/4048521698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564086757210 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086aefc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af06e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00fef0 tx=0x7fdb0c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d66d7785ffdb49d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c004b20 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fdb0c004cc0 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb100c3a20 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fdb0c005020 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1ebd8740 10 client.26996 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.256+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x564086823620 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb0c021070 con 0x564086af2b60 2024-06-06T07:25:26.261 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fdb0c0255d0 con 0x564086af2b60 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 10 client.26996 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 10 client.26996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 20 client.26996 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b05030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 4 client.26996 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:25:26.262 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.257+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdb100c3a20 con 0x564086b02c40 2024-06-06T07:25:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.258+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b05030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.259+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b05030 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=576ef7070e7d5d4e server_cookie=6ce64dc65460c6d4 in_seq=0 out_seq=0 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.259+0000 7fdafd7fa640 10 client.26996 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdafd7fa640 1 -- 172.21.15.73:0/4048521698 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fdb100c3a20 con 0x564086b02c40 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdafd7fa640 10 client.26996 handle_command_reply: tid=0 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 client.26996 shutdown 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 msgr2=0x564086af06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af06e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c00fef0 tx=0x7fdb0c004ab0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 shutdown_connections 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b05030 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdafd7fa640 0 client.26996 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.260+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af06e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/4048521698 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086aefc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 >> 172.21.15.73:0/4048521698 conn(0x564086adb680 msgr2=0x564086ae13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 shutdown_connections 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/4048521698 wait complete. 2024-06-06T07:25:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.261+0000 7fdb1ebd8740 20 client.26996 trim_cache size 0 max 16384 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 Processor -- start 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 -- start start 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086adf0b0 0x564086af0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086af2b60 0x564086b0ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141acc70 con 0x564086adf9d0 2024-06-06T07:25:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141ae7b0 con 0x564086adf0b0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141ad340 con 0x564086af2b60 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.290+0000 7fdb1a38b640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086af2b60 0x564086b0ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19b8a640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086adf0b0 0x564086af0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44450/0 (socket says 172.21.15.73:44450) 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 -- 172.21.15.73:0/3254759531 learned_addr learned my addr 172.21.15.73:0/3254759531 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 -- 172.21.15.73:0/3254759531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086af2b60 msgr2=0x564086b0ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086af2b60 0x564086b0ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 -- 172.21.15.73:0/3254759531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086adf0b0 msgr2=0x564086af0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086adf0b0 0x564086af0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 -- 172.21.15.73:0/3254759531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564086af0c40 con 0x564086adf9d0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdb19389640 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400b680 tx=0x7fdb0400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c627f6813ab4a009 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdaff7fe640 1 -- 172.21.15.73:0/3254759531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04003bd0 con 0x564086adf9d0 2024-06-06T07:25:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdaff7fe640 1 -- 172.21.15.73:0/3254759531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb04003d70 con 0x564086adf9d0 2024-06-06T07:25:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.291+0000 7fdaff7fe640 1 -- 172.21.15.73:0/3254759531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb04004e10 con 0x564086adf9d0 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/3254759531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 msgr2=0x564086b0a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fdb0400b680 tx=0x7fdb0400bb40 comp rx=0 tx=0).stop 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/3254759531 shutdown_connections 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086af2b60 0x564086b0ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086b0a8f0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/3254759531 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086adf0b0 0x564086af0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/3254759531 >> 172.21.15.73:0/3254759531 conn(0x564086adb680 msgr2=0x564086aefc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.292+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/3254759531 shutdown_connections 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/3254759531 wait complete. 2024-06-06T07:25:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 -- start start 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 Processor -- start 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086af6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.293+0000 7fdb1ebd8740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141ae0e0 con 0x564086adf9d0 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141ada10 con 0x564086af2b60 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1ebd8740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fdb141aac70 con 0x564086adf0b0 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1ebd8740 10 client.0 _ll_register_callbacks cb 0x7fdb0c00fe00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44452/0 (socket says 172.21.15.73:44452) 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 -- 172.21.15.73:0/1143657488 learned_addr learned my addr 172.21.15.73:0/1143657488 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086af6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 -- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 msgr2=0x564086af6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086af6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 -- 172.21.15.73:0/1143657488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 msgr2=0x564086af73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 -- 172.21.15.73:0/1143657488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564086b6cb60 con 0x564086adf9d0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb1a38b640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdb19389640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdb04004650 tx=0x7fdb04004c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ae0f296e1561f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.294+0000 7fdafd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0402a060 con 0x564086adf9d0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fdb04003f20 con 0x564086adf9d0 2024-06-06T07:25:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb04003d70 con 0x564086adf9d0 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fdb04031050 con 0x564086adf9d0 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 10 client.17802 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x564086b6de70 con 0x564086adf9d0 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb04003d70 con 0x564086adf9d0 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fdb04019030 con 0x564086adf9d0 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 10 client.17802 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 10 client.17802 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 20 client.17802 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b70550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 4 client.17802 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564086a56f10 con 0x564086b02c40 2024-06-06T07:25:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.295+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b70550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.296+0000 7fdafd7fa640 10 client.17802 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.296+0000 7fdb19b8a640 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b70550 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66699db4263c6353 server_cookie=5872fa06e72f3c59 in_seq=0 out_seq=0 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdafd7fa640 1 -- 172.21.15.73:0/1143657488 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x564086a56f10 con 0x564086b02c40 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdafd7fa640 10 client.17802 handle_command_reply: tid=0 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 client.17802 shutdown 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 msgr2=0x564086af6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fdb04004650 tx=0x7fdb04004c40 comp rx=0 tx=0).stop 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 shutdown_connections 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x564086b02c40 0x564086b70550 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x564086af2b60 0x564086af73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564086adf9d0 0x564086af6ea0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdafd7fa640 0 client.17802 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 --2- 172.21.15.73:0/1143657488 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x564086adf0b0 0x564086af6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.432+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 >> 172.21.15.73:0/1143657488 conn(0x564086adb680 msgr2=0x564086aee0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.433+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 shutdown_connections 2024-06-06T07:25:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.433+0000 7fdb1ebd8740 1 -- 172.21.15.73:0/1143657488 wait complete. 2024-06-06T07:25:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.433+0000 7fdb1ebd8740 20 client.17802 trim_cache size 0 max 16384 2024-06-06T07:25:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:25:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:25:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.433+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 msgr2=0x7fdb000a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.433+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 0x7fdb000a52e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c0099a0 tx=0x7fdb0c009500 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 msgr2=0x7fdb14198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdb10006010 tx=0x7fdb10004270 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 shutdown_connections 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fdb14106890 0x7fdb1419ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb14101290 0x7fdb141986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fdb000a2e20 0x7fdb000a52e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 --2- 172.21.15.73:0/3591678731 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fdb141008d0 0x7fdb14198180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 >> 172.21.15.73:0/3591678731 conn(0x7fdb14077a20 msgr2=0x7fdb14069490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 shutdown_connections 2024-06-06T07:25:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:26.434+0000 7fdb1bdee640 1 -- 172.21.15.73:0/3591678731 wait complete. 2024-06-06T07:25:26.837 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103243 8.74 MB/sec execute 511 sec latency 110.727 ms 2024-06-06T07:25:27.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:27 smithi073 ceph-mon[32303]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2024-06-06T07:25:27.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:27 smithi136 ceph-mon[35901]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2024-06-06T07:25:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:27 smithi170 ceph-mon[36108]: pgmap v768: 129 pgs: 129 active+clean; 1.5 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 5.5 MiB/s wr, 261 op/s 2024-06-06T07:25:27.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103367 8.73 MB/sec execute 512 sec latency 77.028 ms 2024-06-06T07:25:28.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103511 8.72 MB/sec execute 513 sec latency 109.105 ms 2024-06-06T07:25:29.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:29 smithi073 ceph-mon[32303]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 392 op/s 2024-06-06T07:25:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:29 smithi136 ceph-mon[35901]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 392 op/s 2024-06-06T07:25:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:29 smithi170 ceph-mon[36108]: pgmap v769: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 7.9 MiB/s wr, 392 op/s 2024-06-06T07:25:29.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103677 8.73 MB/sec execute 514 sec latency 85.321 ms 2024-06-06T07:25:30.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103827 8.72 MB/sec execute 515 sec latency 102.204 ms 2024-06-06T07:25:31.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:31 smithi073 ceph-mon[32303]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 274 op/s 2024-06-06T07:25:31.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:31 smithi136 ceph-mon[35901]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 274 op/s 2024-06-06T07:25:31.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:31 smithi170 ceph-mon[36108]: pgmap v770: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.5 MiB/s wr, 274 op/s 2024-06-06T07:25:31.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 103987 8.73 MB/sec execute 516 sec latency 82.126 ms 2024-06-06T07:25:32.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104121 8.72 MB/sec execute 517 sec latency 79.613 ms 2024-06-06T07:25:33.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:33 smithi073 ceph-mon[32303]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:33.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:33 smithi136 ceph-mon[35901]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:33.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:33 smithi170 ceph-mon[36108]: pgmap v771: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:33.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104267 8.72 MB/sec execute 518 sec latency 59.558 ms 2024-06-06T07:25:34.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104442 8.72 MB/sec execute 519 sec latency 96.310 ms 2024-06-06T07:25:35.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:35 smithi073 ceph-mon[32303]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:35.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:35 smithi136 ceph-mon[35901]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:35.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:35 smithi170 ceph-mon[36108]: pgmap v772: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:35.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104628 8.72 MB/sec execute 520 sec latency 121.030 ms 2024-06-06T07:25:36.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104759 8.71 MB/sec execute 521 sec latency 80.997 ms 2024-06-06T07:25:37.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:37 smithi073 ceph-mon[32303]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-06-06T07:25:37.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:37 smithi136 ceph-mon[35901]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-06-06T07:25:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:37 smithi170 ceph-mon[36108]: pgmap v773: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.1 MiB/s wr, 283 op/s 2024-06-06T07:25:37.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 104907 8.71 MB/sec execute 522 sec latency 70.414 ms 2024-06-06T07:25:38.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105081 8.71 MB/sec execute 523 sec latency 81.895 ms 2024-06-06T07:25:39.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:39 smithi073 ceph-mon[32303]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 395 op/s 2024-06-06T07:25:39.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:39 smithi136 ceph-mon[35901]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 395 op/s 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 -- 172.21.15.73:0/2530559677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8101990 msgr2=0x7fb8b8101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/2530559677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8101990 0x7fb8b8101e10 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009930 tx=0x7fb8a002a420 comp rx=0 tx=0).stop 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 -- 172.21.15.73:0/2530559677 shutdown_connections 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/2530559677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b8102350 0x7fb8b810e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/2530559677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8101990 0x7fb8b8101e10 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/2530559677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b81079a0 0x7fb8b8107d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 -- 172.21.15.73:0/2530559677 >> 172.21.15.73:0/2530559677 conn(0x7fb8b80fd460 msgr2=0x7fb8b80ff880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 -- 172.21.15.73:0/2530559677 shutdown_connections 2024-06-06T07:25:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.567+0000 7fb8be3e7640 1 -- 172.21.15.73:0/2530559677 wait complete. 2024-06-06T07:25:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 Processor -- start 2024-06-06T07:25:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 -- start start 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8102350 0x7fb8b8198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 0x7fb8b819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111c80 con 0x7fb8b8101990 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111b00 con 0x7fb8b81079a0 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8be3e7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b8111e00 con 0x7fb8b8102350 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8b7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46630/0 (socket says 172.21.15.73:46630) 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8b7fff640 1 -- 172.21.15.73:0/1683856878 learned_addr learned my addr 172.21.15.73:0/1683856878 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.568+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 0x7fb8b819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8affff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8102350 0x7fb8b8198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 -- 172.21.15.73:0/1683856878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8102350 msgr2=0x7fb8b8198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8102350 0x7fb8b8198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 -- 172.21.15.73:0/1683856878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 msgr2=0x7fb8b819ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 0x7fb8b819ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 -- 172.21.15.73:0/1683856878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8b819d130 con 0x7fb8b8101990 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 0x7fb8b819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800e9f0 tx=0x7fb8a800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b5ffb640 1 -- 172.21.15.73:0/1683856878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a800cd80 con 0x7fb8b8101990 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b5ffb640 1 -- 172.21.15.73:0/1683856878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8a80108e0 con 0x7fb8b8101990 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8b5ffb640 1 -- 172.21.15.73:0/1683856878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8018920 con 0x7fb8b8101990 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8b819d420 con 0x7fb8b8101990 2024-06-06T07:25:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.569+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8b81a4cb0 con 0x7fb8b8101990 2024-06-06T07:25:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 Processor -- start 2024-06-06T07:25:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 -- start start 2024-06-06T07:25:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8b5ffb640 1 -- 172.21.15.73:0/1683856878 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb8a8010a90 con 0x7fb8b8101990 2024-06-06T07:25:39.574 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed792e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed795740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653ed6ec4f0 con 0x5653ed788170 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ed68a870 con 0x5653ed793350 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5653ed6e84b0 con 0x5653ed788a90 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46638/0 (socket says 172.21.15.73:46638) 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8bc984640 1 -- 172.21.15.73:0/564629228 learned_addr learned my addr 172.21.15.73:0/564629228 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8affff640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed795740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.573+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed792e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 -- 172.21.15.73:0/564629228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 msgr2=0x5653ed792e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed792e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.575 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 -- 172.21.15.73:0/564629228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 msgr2=0x5653ed795740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed795740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 -- 172.21.15.73:0/564629228 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ed6a69b0 con 0x5653ed788170 2024-06-06T07:25:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed792e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:25:39.576 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8bc984640 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00117e0 tx=0x7fb8b0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45646b2e72120fa server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.578 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8ae7fc640 1 -- 172.21.15.73:0/564629228 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0011310 con 0x5653ed788170 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8b5ffb640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 0x7fb8a40a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8ae7fc640 1 -- 172.21.15.73:0/564629228 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8b0003e20 con 0x5653ed788170 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8ae7fc640 1 -- 172.21.15.73:0/564629228 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00044f0 con 0x5653ed788170 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.574+0000 7fb8b5ffb640 1 -- 172.21.15.73:0/1683856878 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb8a80cfb50 con 0x7fb8b8101990 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 0x7fb8a40a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 -- 172.21.15.73:0/564629228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 msgr2=0x5653ed788550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00117e0 tx=0x7fb8b0011ca0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8b7fff640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 0x7fb8a40a52e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5653ed789710 tx=0x7fb8a8000bc0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 -- 172.21.15.73:0/564629228 shutdown_connections 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed795740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.579 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed792e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/564629228 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed788550 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 -- 172.21.15.73:0/564629228 >> 172.21.15.73:0/564629228 conn(0x5653ed784680 msgr2=0x5653ed784a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 -- 172.21.15.73:0/564629228 shutdown_connections 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.575+0000 7fb8c11d4740 1 -- 172.21.15.73:0/564629228 wait complete. 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 -- start start 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 Processor -- start 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed790780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed790cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653ed795f80 con 0x5653ed793350 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.576+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x5653ed675ae0 con 0x5653ed788170 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5653ed796100 con 0x5653ed788a90 2024-06-06T07:25:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8c11d4740 10 client.0 _ll_register_callbacks cb 0x5653ed6ae9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:50658/0 (socket says 172.21.15.73:50658) 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8affff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed790780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 -- 172.21.15.73:0/2121858930 learned_addr learned my addr 172.21.15.73:0/2121858930 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed790cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 -- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 msgr2=0x5653ed790780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed790780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 -- 172.21.15.73:0/2121858930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 msgr2=0x5653ed790cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed790cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8bc984640 1 -- 172.21.15.73:0/2121858930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ed400210 con 0x5653ed788170 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.577+0000 7fb8affff640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed790780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8bc984640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00115a0 tx=0x7fb8b00040d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41dae5360c922dd server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:25:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0004c50 con 0x5653ed788170 2024-06-06T07:25:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b0003e20 con 0x5653ed788170 2024-06-06T07:25:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb8b0004df0 con 0x5653ed788170 2024-06-06T07:25:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb8b000f9a0 con 0x5653ed788170 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8c11d4740 10 client.27011 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.578+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5653ed4cc620 con 0x5653ed788170 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8b0003e20 con 0x5653ed788170 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fb8b0025030 con 0x5653ed788170 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 10 client.27011 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 10 client.27011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 20 client.27011 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed7a33c0 0x5653ed7a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 4 client.27011 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.579+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653ed6fff10 con 0x5653ed7a33c0 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.580+0000 7fb8affff640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed7a33c0 0x5653ed7a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.582+0000 7fb8affff640 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed7a33c0 0x5653ed7a57b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=baa0d8b7c16799d0 server_cookie=d8b824f36026b7ad in_seq=0 out_seq=0 2024-06-06T07:25:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.582+0000 7fb8977fe640 10 client.27011 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:39.584 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.583+0000 7fb8977fe640 1 -- 172.21.15.73:0/2121858930 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5653ed6fff10 con 0x5653ed7a33c0 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.583+0000 7fb8977fe640 10 client.27011 handle_command_reply: tid=0 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.583+0000 7fb8c11d4740 1 client.27011 shutdown 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 msgr2=0x5653ed790240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00115a0 tx=0x7fb8b00040d0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 shutdown_connections 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed7a33c0 0x5653ed7a57b0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed790cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8977fe640 0 client.27011 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed790780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/2121858930 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed790240 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 >> 172.21.15.73:0/2121858930 conn(0x5653ed784680 msgr2=0x5653ed78a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 shutdown_connections 2024-06-06T07:25:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 1 -- 172.21.15.73:0/2121858930 wait complete. 2024-06-06T07:25:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.584+0000 7fb8c11d4740 20 client.27011 trim_cache size 0 max 16384 2024-06-06T07:25:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:39 smithi170 ceph-mon[36108]: pgmap v774: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.1 MiB/s wr, 395 op/s 2024-06-06T07:25:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.609+0000 7fb8c11d4740 1 Processor -- start 2024-06-06T07:25:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.609+0000 7fb8c11d4740 1 -- start start 2024-06-06T07:25:39.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46656/0 (socket says 172.21.15.73:46656) 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 -- 172.21.15.73:0/693492896 learned_addr learned my addr 172.21.15.73:0/693492896 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed7aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed7ab410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81acd90 con 0x5653ed788170 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ae8d0 con 0x5653ed793350 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ad460 con 0x5653ed788a90 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed7ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 -- 172.21.15.73:0/693492896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 msgr2=0x5653ed7aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed7aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 -- 172.21.15.73:0/693492896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 msgr2=0x5653ed7ab410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed7ab410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.610+0000 7fb8bc984640 1 -- 172.21.15.73:0/693492896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653ed7a0290 con 0x5653ed788170 2024-06-06T07:25:39.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.611+0000 7fb8bc984640 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00114e0 tx=0x7fb8b0011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67f02c5bf24e79f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.611+0000 7fb8adffb640 1 -- 172.21.15.73:0/693492896 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b000f790 con 0x5653ed788170 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.611+0000 7fb8adffb640 1 -- 172.21.15.73:0/693492896 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8b0005370 con 0x5653ed788170 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.611+0000 7fb8adffb640 1 -- 172.21.15.73:0/693492896 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8b00056c0 con 0x5653ed788170 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 msgr2=0x5653ed783660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb8b00114e0 tx=0x7fb8b0011ca0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.616 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 shutdown_connections 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed793350 0x5653ed7ab410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed7aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/693492896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed788170 0x5653ed783660 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 >> 172.21.15.73:0/693492896 conn(0x5653ed784680 msgr2=0x5653ed782f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 shutdown_connections 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- 172.21.15.73:0/693492896 wait complete. 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.612+0000 7fb8c11d4740 1 -- start start 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 Processor -- start 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed79b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed79b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed78c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.617 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81ae200 con 0x5653ed793350 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81adb30 con 0x5653ed788170 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb8b81aad90 con 0x5653ed788a90 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8c11d4740 10 client.0 _ll_register_callbacks cb 0x7fb8a8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed79b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8bc984640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed79b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:50682/0 (socket says 172.21.15.73:50682) 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8bc984640 1 -- 172.21.15.73:0/1215461675 learned_addr learned my addr 172.21.15.73:0/1215461675 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8affff640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed79b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.613+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed78c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 -- 172.21.15.73:0/1215461675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 msgr2=0x5653ed79b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed79b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 -- 172.21.15.73:0/1215461675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 msgr2=0x5653ed78c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed78c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.618 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 -- 172.21.15.73:0/1215461675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653ed78ce00 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed78c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8affff640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed79b890 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009850 tx=0x7fb8a0005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=537e98ce6acb336f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8977fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0035040 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb8a00046c0 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.614+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8b0005370 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb8a0053050 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 10 client.27193 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5653ed80d6d0 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8b0005370 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fb8a0003af0 con 0x5653ed788a90 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 10 client.27193 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:39.619 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 10 client.27193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 20 client.27193 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed80ded0 0x5653ed8102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 4 client.27193 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.615+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653ed6f9c20 con 0x5653ed80ded0 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.616+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed80ded0 0x5653ed8102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.616+0000 7fb8bc984640 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed80ded0 0x5653ed8102c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9276df3baa9a5b7a server_cookie=2e52e44a4ec8cfda in_seq=0 out_seq=0 2024-06-06T07:25:39.620 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.616+0000 7fb8977fe640 10 client.27193 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.697+0000 7fb8977fe640 1 -- 172.21.15.73:0/1215461675 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5653ed6f9c20 con 0x5653ed80ded0 2024-06-06T07:25:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8977fe640 10 client.27193 handle_command_reply: tid=0 2024-06-06T07:25:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 client.27193 shutdown 2024-06-06T07:25:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 msgr2=0x5653ed79b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed79b890 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009850 tx=0x7fb8a0005280 comp rx=0 tx=0).stop 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 shutdown_connections 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5653ed80ded0 0x5653ed8102c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653ed793350 0x5653ed78c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x5653ed788a90 0x5653ed79b890 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 --2- 172.21.15.73:0/1215461675 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x5653ed788170 0x5653ed79b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 >> 172.21.15.73:0/1215461675 conn(0x5653ed784680 msgr2=0x5653ed7837e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8977fe640 0 client.27193 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 shutdown_connections 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 1 -- 172.21.15.73:0/1215461675 wait complete. 2024-06-06T07:25:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.698+0000 7fb8c11d4740 20 client.27193 trim_cache size 0 max 16384 2024-06-06T07:25:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:25:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:25:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 msgr2=0x7fb8a40a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 0x7fb8a40a52e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x5653ed789710 tx=0x7fb8a8000bc0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 msgr2=0x7fb8b8198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800e9f0 tx=0x7fb8a800eeb0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 shutdown_connections 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb8b81079a0 0x7fb8b819ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb8a40a2e20 0x7fb8a40a52e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb8b8102350 0x7fb8b8198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 --2- 172.21.15.73:0/1683856878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8b8101990 0x7fb8b8198140 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 >> 172.21.15.73:0/1683856878 conn(0x7fb8b80fd460 msgr2=0x7fb8b80fdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 shutdown_connections 2024-06-06T07:25:39.703 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:39.699+0000 7fb8be3e7640 1 -- 172.21.15.73:0/1683856878 wait complete. 2024-06-06T07:25:39.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105241 8.72 MB/sec execute 524 sec latency 75.317 ms 2024-06-06T07:25:40.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105395 8.71 MB/sec execute 525 sec latency 66.946 ms 2024-06-06T07:25:41.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:41 smithi073 ceph-mon[32303]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 279 op/s 2024-06-06T07:25:41.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:41 smithi136 ceph-mon[35901]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 279 op/s 2024-06-06T07:25:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:41 smithi170 ceph-mon[36108]: pgmap v775: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 279 op/s 2024-06-06T07:25:41.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105594 8.72 MB/sec execute 526 sec latency 69.014 ms 2024-06-06T07:25:42.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105780 8.72 MB/sec execute 527 sec latency 168.447 ms 2024-06-06T07:25:43.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:43 smithi073 ceph-mon[32303]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 263 op/s 2024-06-06T07:25:43.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:43 smithi136 ceph-mon[35901]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 263 op/s 2024-06-06T07:25:43.596 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:43 smithi170 ceph-mon[36108]: pgmap v776: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.9 MiB/s wr, 263 op/s 2024-06-06T07:25:43.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 105936 8.72 MB/sec execute 528 sec latency 77.243 ms 2024-06-06T07:25:44.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106105 8.73 MB/sec execute 529 sec latency 93.367 ms 2024-06-06T07:25:45.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:45 smithi073 ceph-mon[32303]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:45.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:45 smithi136 ceph-mon[35901]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:45.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:45 smithi170 ceph-mon[36108]: pgmap v777: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 7.8 MiB/s wr, 407 op/s 2024-06-06T07:25:45.838 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106266 8.74 MB/sec execute 530 sec latency 109.488 ms 2024-06-06T07:25:46.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106402 8.73 MB/sec execute 531 sec latency 131.795 ms 2024-06-06T07:25:47.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:47 smithi073 ceph-mon[32303]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:47.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:47 smithi136 ceph-mon[35901]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:47.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:47 smithi170 ceph-mon[36108]: pgmap v778: 129 pgs: 129 active+clean; 1.5 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.3 MiB/s wr, 271 op/s 2024-06-06T07:25:47.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106547 8.73 MB/sec execute 532 sec latency 156.932 ms 2024-06-06T07:25:48.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106700 8.72 MB/sec execute 533 sec latency 61.223 ms 2024-06-06T07:25:49.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:49 smithi073 ceph-mon[32303]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-06-06T07:25:49.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:49 smithi136 ceph-mon[35901]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-06-06T07:25:49.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:49 smithi170 ceph-mon[36108]: pgmap v779: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 7.9 MiB/s wr, 402 op/s 2024-06-06T07:25:49.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 106854 8.71 MB/sec execute 534 sec latency 146.926 ms 2024-06-06T07:25:50.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107000 8.71 MB/sec execute 535 sec latency 94.132 ms 2024-06-06T07:25:51.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:51 smithi073 ceph-mon[32303]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 293 op/s 2024-06-06T07:25:51.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:51 smithi136 ceph-mon[35901]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 293 op/s 2024-06-06T07:25:51.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:51 smithi170 ceph-mon[36108]: pgmap v780: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 6.0 MiB/s wr, 293 op/s 2024-06-06T07:25:51.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107142 8.71 MB/sec execute 536 sec latency 92.576 ms 2024-06-06T07:25:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 -- 172.21.15.73:0/519223998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 msgr2=0x7fb9381089a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/519223998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381089a0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb930009ca0 tx=0x7fb93002a970 comp rx=0 tx=0).stop 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 -- 172.21.15.73:0/519223998 shutdown_connections 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/519223998 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb938108ee0 0x7fb938109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/519223998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381089a0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/519223998 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb93810e230 0x7fb93810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.837+0000 7fb93e0fc640 1 -- 172.21.15.73:0/519223998 >> 172.21.15.73:0/519223998 conn(0x7fb93806cf00 msgr2=0x7fb93806d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.838+0000 7fb93e0fc640 1 -- 172.21.15.73:0/519223998 shutdown_connections 2024-06-06T07:25:52.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107300 8.71 MB/sec execute 537 sec latency 85.242 ms 2024-06-06T07:25:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.839+0000 7fb93e0fc640 1 -- 172.21.15.73:0/519223998 wait complete. 2024-06-06T07:25:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.839+0000 7fb93e0fc640 1 Processor -- start 2024-06-06T07:25:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.839+0000 7fb93e0fc640 1 -- start start 2024-06-06T07:25:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb938108ee0 0x7fb9381acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb93811a4e0 con 0x7fb93810eb50 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb93811a360 con 0x7fb938108ee0 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb93e0fc640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb93811a660 con 0x7fb93810e230 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57138/0 (socket says 172.21.15.73:57138) 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb936ffd640 1 -- 172.21.15.73:0/1598007159 learned_addr learned my addr 172.21.15.73:0/1598007159 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb9377fe640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb938108ee0 0x7fb9381acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.840+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37456/0 (socket says 172.21.15.73:37456) 2024-06-06T07:25:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb936ffd640 1 -- 172.21.15.73:0/1598007159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb938108ee0 msgr2=0x7fb9381acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb938108ee0 0x7fb9381acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb936ffd640 1 -- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 msgr2=0x7fb9381171c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb936ffd640 1 -- 172.21.15.73:0/1598007159 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9381179d0 con 0x7fb93810e230 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.841+0000 7fb937fff640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb93002af60 tx=0x7fb930030920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb934ff9640 1 -- 172.21.15.73:0/1598007159 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930034030 con 0x7fb93810e230 2024-06-06T07:25:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb938117c60 con 0x7fb93810e230 2024-06-06T07:25:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb93819f9b0 con 0x7fb93810e230 2024-06-06T07:25:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb934ff9640 1 -- 172.21.15.73:0/1598007159 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb930004040 con 0x7fb93810e230 2024-06-06T07:25:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.842+0000 7fb934ff9640 1 -- 172.21.15.73:0/1598007159 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930039680 con 0x7fb93810e230 2024-06-06T07:25:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.846+0000 7fb940e07740 1 Processor -- start 2024-06-06T07:25:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.846+0000 7fb940e07740 1 -- start start 2024-06-06T07:25:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef96550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3ef96a90 0x55de3efa0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de3eefa4f0 con 0x55de3ef96a90 2024-06-06T07:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de3ee98870 con 0x55de3efa1350 2024-06-06T07:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb940e07740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55de3eef64b0 con 0x55de3ef96170 2024-06-06T07:25:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb934ff9640 1 -- 172.21.15.73:0/1598007159 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fb930037020 con 0x7fb93810e230 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53300/0 (socket says 172.21.15.73:53300) 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb936ffd640 1 -- 172.21.15.73:0/3100765535 learned_addr learned my addr 172.21.15.73:0/3100765535 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.847+0000 7fb937fff640 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef96550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 -- 172.21.15.73:0/3100765535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 msgr2=0x55de3ef96550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef96550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 -- 172.21.15.73:0/3100765535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3ef96a90 msgr2=0x55de3efa0e10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3ef96a90 0x55de3efa0e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 -- 172.21.15.73:0/3100765535 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3eeb49b0 con 0x55de3efa1350 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb934ff9640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 0x7fb9200a5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb936ffd640 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb9300ed9c0 tx=0x7fb9300f6b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c516334354a9751 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb91dffb640 1 -- 172.21.15.73:0/3100765535 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300b8cb0 con 0x55de3efa1350 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb934ff9640 1 -- 172.21.15.73:0/1598007159 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb9300f07a0 con 0x7fb93810e230 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb91dffb640 1 -- 172.21.15.73:0/3100765535 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9300098d0 con 0x55de3efa1350 2024-06-06T07:25:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.848+0000 7fb91dffb640 1 -- 172.21.15.73:0/3100765535 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb930041570 con 0x55de3efa1350 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.849+0000 7fb940e07740 1 -- 172.21.15.73:0/3100765535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 msgr2=0x55de3efa3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.849+0000 7fb940e07740 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb9300ed9c0 tx=0x7fb9300f6b20 comp rx=0 tx=0).stop 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.850+0000 7fb937fff640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 0x7fb9200a5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.850+0000 7fb940e07740 1 -- 172.21.15.73:0/3100765535 shutdown_connections 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.850+0000 7fb940e07740 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3efa1350 0x55de3efa3740 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb940e07740 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3ef96a90 0x55de3efa0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb940e07740 1 --2- 172.21.15.73:0/3100765535 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef96550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb940e07740 1 -- 172.21.15.73:0/3100765535 >> 172.21.15.73:0/3100765535 conn(0x55de3ef92680 msgr2=0x55de3ef92a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb937fff640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 0x7fb9200a5360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb928006fd0 tx=0x7fb928008040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb940e07740 1 -- 172.21.15.73:0/3100765535 shutdown_connections 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.851+0000 7fb940e07740 1 -- 172.21.15.73:0/3100765535 wait complete. 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.852+0000 7fb940e07740 1 -- start start 2024-06-06T07:25:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.852+0000 7fb940e07740 1 Processor -- start 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.852+0000 7fb940e07740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef9e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.852+0000 7fb940e07740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 0x55de3ef9e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb940e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef9e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef9e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57166/0 (socket says 172.21.15.73:57166) 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb9377fe640 1 -- 172.21.15.73:0/1173503337 learned_addr learned my addr 172.21.15.73:0/1173503337 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55de3ee98870 con 0x55de3efa1350 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55de3ee83ae0 con 0x55de3ef96a90 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55de3efa3d40 con 0x55de3ef96170 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 0x55de3ef9e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb940e07740 10 client.0 _ll_register_callbacks cb 0x55de3eebc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb937fff640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 -- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 msgr2=0x55de3ef9e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef9e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 -- 172.21.15.73:0/1173503337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 msgr2=0x55de3ef9ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3ec0e210 con 0x55de3ef96a90 2024-06-06T07:25:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb936ffd640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 0x55de3ef9e750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb930009430 tx=0x7fb930033780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81fdac91e2a4a770 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.853+0000 7fb916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9300d5060 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55de3ef0df10 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb9300baab0 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb930041750 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb940e07740 10 client.27026 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55de3ecda620 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55de3ef0df10 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.854+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fb9300c0ce0 con 0x55de3ef96a90 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 10 client.27026 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 10 client.27026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 20 client.27026 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3efb1010 0x55de3efb3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 4 client.27026 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:25:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de3ef07c20 con 0x55de3efb1010 2024-06-06T07:25:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.855+0000 7fb9377fe640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3efb1010 0x55de3efb3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.856+0000 7fb9377fe640 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3efb1010 0x55de3efb3450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3614a0743014024 server_cookie=606b6e41587accef in_seq=0 out_seq=0 2024-06-06T07:25:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.856+0000 7fb916ffd640 10 client.27026 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.856+0000 7fb916ffd640 1 -- 172.21.15.73:0/1173503337 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55de3ef07c20 con 0x55de3efb1010 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.856+0000 7fb916ffd640 10 client.27026 handle_command_reply: tid=0 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.856+0000 7fb940e07740 1 client.27026 shutdown 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 msgr2=0x55de3ef9e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 0x55de3ef9e750 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb930009430 tx=0x7fb930033780 comp rx=0 tx=0).stop 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 shutdown_connections 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3efb1010 0x55de3efb3450 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96a90 0x55de3ef9e750 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 --2- 172.21.15.73:0/1173503337 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96170 0x55de3ef9e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 >> 172.21.15.73:0/1173503337 conn(0x55de3ef92680 msgr2=0x55de3ef98400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb916ffd640 0 client.27026 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 shutdown_connections 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 1 -- 172.21.15.73:0/1173503337 wait complete. 2024-06-06T07:25:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.857+0000 7fb940e07740 20 client.27026 trim_cache size 0 max 16384 2024-06-06T07:25:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.888+0000 7fb940e07740 1 Processor -- start 2024-06-06T07:25:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.888+0000 7fb940e07740 1 -- start start 2024-06-06T07:25:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efb8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efb9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb924010080 con 0x55de3efa1350 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb924011bc0 con 0x55de3ef96170 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efb9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37486/0 (socket says 172.21.15.73:37486) 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb937fff640 1 -- 172.21.15.73:0/1014379249 learned_addr learned my addr 172.21.15.73:0/1014379249 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb924010750 con 0x55de3ef96a90 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.889+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efb8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 -- 172.21.15.73:0/1014379249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 msgr2=0x55de3efb9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efb9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 -- 172.21.15.73:0/1014379249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 msgr2=0x55de3efb8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efb8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 -- 172.21.15.73:0/1014379249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3ef8dfe0 con 0x55de3efa1350 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb937fff640 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb9280155d0 tx=0x7fb928014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ddc0339f76b32d server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.890+0000 7fb91d7fa640 1 -- 172.21.15.73:0/1014379249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928014520 con 0x55de3efa1350 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb91d7fa640 1 -- 172.21.15.73:0/1014379249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb92801a030 con 0x55de3efa1350 2024-06-06T07:25:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb91d7fa640 1 -- 172.21.15.73:0/1014379249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92802aac0 con 0x55de3efa1350 2024-06-06T07:25:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 msgr2=0x55de3ef9e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb9280155d0 tx=0x7fb928014000 comp rx=0 tx=0).stop 2024-06-06T07:25:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 shutdown_connections 2024-06-06T07:25:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3ef9e1d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efb9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 --2- 172.21.15.73:0/1014379249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efb8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.891+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 >> 172.21.15.73:0/1014379249 conn(0x55de3ef92680 msgr2=0x55de3ef9cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.892+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 shutdown_connections 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.892+0000 7fb940e07740 1 -- 172.21.15.73:0/1014379249 wait complete. 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.892+0000 7fb940e07740 1 -- start start 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.892+0000 7fb940e07740 1 Processor -- start 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efa4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:53336/0 (socket says 172.21.15.73:53336) 2024-06-06T07:25:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efa4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb936ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efa4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:57180/0 (socket says 172.21.15.73:57180) 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb937fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37498/0 (socket says 172.21.15.73:37498) 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 -- 172.21.15.73:0/3773966715 learned_addr learned my addr 172.21.15.73:0/3773966715 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9240114f0 con 0x55de3efa1350 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fb924010e20 con 0x55de3ef96170 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fb92400e060 con 0x55de3ef96a90 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb940e07740 10 client.0 _ll_register_callbacks cb 0x7fb9300f68e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 -- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 msgr2=0x55de3efa4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efa4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.893+0000 7fb9377fe640 1 -- 172.21.15.73:0/3773966715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 msgr2=0x55de3efa4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb9377fe640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb9377fe640 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de3f019d60 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb937fff640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb9377fe640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb92c012700 tx=0x7fb92c012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b780a253de42b1c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb916ffd640 10 client.0 ms_handle_connect on v2:172.21.15.136:3300/0 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c021070 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb92801a030 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mon.2 v2:172.21.15.136:3300/0 2 ==== osd_map(144..144 src has 1..144) ==== 9554+0+0 (secure 0 0 0) 0x7fb92c004020 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fb92c02e050 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb940e07740 10 client.27032 fetch_fsmap learned FSMap version 84 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.894+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55de3f01b100 con 0x55de3ef96170 2024-06-06T07:25:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mon.2 v2:172.21.15.136:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb92801a030 con 0x55de3ef96170 2024-06-06T07:25:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mon.2 v2:172.21.15.136:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fb92c003680 con 0x55de3ef96170 2024-06-06T07:25:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 10 client.27032 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:25:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 10 client.27032 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:25:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 20 client.27032 populate_metadata read hostname 'smithi073' 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3f01b860 0x55de3f01dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 4 client.27032 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55de3ef01f70 con 0x55de3f01b860 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.895+0000 7fb936ffd640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3f01b860 0x55de3f01dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.896+0000 7fb936ffd640 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3f01b860 0x55de3f01dc80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd027387e3d6cc61 server_cookie=54413fb33e76a169 in_seq=0 out_seq=0 2024-06-06T07:25:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.896+0000 7fb916ffd640 10 client.27032 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.984+0000 7fb916ffd640 1 -- 172.21.15.73:0/3773966715 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55de3ef01f70 con 0x55de3f01b860 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.984+0000 7fb916ffd640 10 client.27032 handle_command_reply: tid=0 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 client.27032 shutdown 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 msgr2=0x55de3efa3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb92c012700 tx=0x7fb92c012bc0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 shutdown_connections 2024-06-06T07:25:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55de3f01b860 0x55de3f01dc80 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55de3efa1350 0x55de3efa4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb916ffd640 0 client.27032 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55de3ef96a90 0x55de3efa4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 --2- 172.21.15.73:0/3773966715 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55de3ef96170 0x55de3efa3ce0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 >> 172.21.15.73:0/3773966715 conn(0x55de3ef92680 msgr2=0x55de3ef90f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 shutdown_connections 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 1 -- 172.21.15.73:0/3773966715 wait complete. 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.985+0000 7fb940e07740 20 client.27032 trim_cache size 0 max 16384 2024-06-06T07:25:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:25:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:25:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:25:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 msgr2=0x7fb9200a5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 0x7fb9200a5360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb928006fd0 tx=0x7fb928008040 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 msgr2=0x7fb9381ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb93002af60 tx=0x7fb930030920 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 shutdown_connections 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb93810eb50 0x7fb9381171c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fb9200a2ea0 0x7fb9200a5360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fb93810e230 0x7fb9381ad0e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 --2- 172.21.15.73:0/1598007159 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fb938108ee0 0x7fb9381acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 >> 172.21.15.73:0/1598007159 conn(0x7fb93806cf00 msgr2=0x7fb93810bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 shutdown_connections 2024-06-06T07:25:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:25:52.986+0000 7fb93e0fc640 1 -- 172.21.15.73:0/1598007159 wait complete. 2024-06-06T07:25:53.526 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:53 smithi073 ceph-mon[32303]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 278 op/s 2024-06-06T07:25:53.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:53 smithi136 ceph-mon[35901]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 278 op/s 2024-06-06T07:25:53.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:53 smithi170 ceph-mon[36108]: pgmap v781: 129 pgs: 129 active+clean; 1.5 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.6 MiB/s wr, 278 op/s 2024-06-06T07:25:53.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107461 8.71 MB/sec execute 538 sec latency 91.534 ms 2024-06-06T07:25:54.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107635 8.72 MB/sec execute 539 sec latency 90.015 ms 2024-06-06T07:25:55.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:55 smithi073 ceph-mon[32303]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-06-06T07:25:55.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:55 smithi136 ceph-mon[35901]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-06-06T07:25:55.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:55 smithi170 ceph-mon[36108]: pgmap v782: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 413 op/s 2024-06-06T07:25:55.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107788 8.72 MB/sec execute 540 sec latency 92.911 ms 2024-06-06T07:25:56.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 107939 8.72 MB/sec execute 541 sec latency 79.479 ms 2024-06-06T07:25:57.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:57 smithi073 ceph-mon[32303]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:25:57.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:57 smithi136 ceph-mon[35901]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:25:57.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:57 smithi170 ceph-mon[36108]: pgmap v783: 129 pgs: 129 active+clean; 1.6 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:25:57.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108089 8.72 MB/sec execute 542 sec latency 66.473 ms 2024-06-06T07:25:58.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108235 8.71 MB/sec execute 543 sec latency 99.427 ms 2024-06-06T07:25:59.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:25:59 smithi073 ceph-mon[32303]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:25:59.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:25:59 smithi136 ceph-mon[35901]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:25:59.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:25:59 smithi170 ceph-mon[36108]: pgmap v784: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.4 MiB/s wr, 402 op/s 2024-06-06T07:25:59.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108382 8.71 MB/sec execute 544 sec latency 68.910 ms 2024-06-06T07:26:00.839 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108527 8.71 MB/sec execute 545 sec latency 127.337 ms 2024-06-06T07:26:01.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:01 smithi073 ceph-mon[32303]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-06T07:26:01.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:01 smithi136 ceph-mon[35901]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-06T07:26:01.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:01 smithi170 ceph-mon[36108]: pgmap v785: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-06T07:26:01.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108685 8.71 MB/sec execute 546 sec latency 1127.769 ms 2024-06-06T07:26:02.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 108870 8.70 MB/sec execute 547 sec latency 2127.815 ms 2024-06-06T07:26:03.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:03 smithi073 ceph-mon[32303]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-06-06T07:26:03.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:03 smithi136 ceph-mon[35901]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-06-06T07:26:03.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:03 smithi170 ceph-mon[36108]: pgmap v786: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 4.8 MiB/s wr, 271 op/s 2024-06-06T07:26:03.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109051 8.71 MB/sec execute 548 sec latency 3127.846 ms 2024-06-06T07:26:04.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109158 8.71 MB/sec execute 549 sec latency 4127.881 ms 2024-06-06T07:26:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:05 smithi073 ceph-mon[32303]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:26:05.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:05 smithi073 ceph-mon[32303]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T07:26:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:05 smithi136 ceph-mon[35901]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:26:05.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:05 smithi136 ceph-mon[35901]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T07:26:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:05 smithi170 ceph-mon[36108]: pgmap v787: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 7.8 MiB/s wr, 408 op/s 2024-06-06T07:26:05.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:05 smithi170 ceph-mon[36108]: osdmap e145: 12 total, 12 up, 12 in 2024-06-06T07:26:05.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109328 8.72 MB/sec execute 550 sec latency 5127.916 ms 2024-06-06T07:26:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 -- 172.21.15.73:0/199200634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c06ab90 msgr2=0x7fef7c10e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 --2- 172.21.15.73:0/199200634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c06ab90 0x7fef7c10e840 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fef74009c10 tx=0x7fef7402a770 comp rx=0 tx=0).stop 2024-06-06T07:26:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 -- 172.21.15.73:0/199200634 shutdown_connections 2024-06-06T07:26:06.117 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 --2- 172.21.15.73:0/199200634 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c06ab90 0x7fef7c10e840 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 --2- 172.21.15.73:0/199200634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06a1d0 0x7fef7c06a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 --2- 172.21.15.73:0/199200634 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c107ba0 0x7fef7c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 -- 172.21.15.73:0/199200634 >> 172.21.15.73:0/199200634 conn(0x7fef7c0766a0 msgr2=0x7fef7c076ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 -- 172.21.15.73:0/199200634 shutdown_connections 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.116+0000 7fef82664640 1 -- 172.21.15.73:0/199200634 wait complete. 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 Processor -- start 2024-06-06T07:26:06.118 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 -- start start 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c06a1d0 0x7fef7c19a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 0x7fef7c19eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111c70 con 0x7fef7c06ab90 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111af0 con 0x7fef7c107ba0 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef82664640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c111df0 con 0x7fef7c06a1d0 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47218/0 (socket says 172.21.15.73:47218) 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef73fff640 1 -- 172.21.15.73:0/3872120125 learned_addr learned my addr 172.21.15.73:0/3872120125 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.117+0000 7fef80c01640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 0x7fef7c19eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.120 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c06a1d0 0x7fef7c19a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 -- 172.21.15.73:0/3872120125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c06a1d0 msgr2=0x7fef7c19a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c06a1d0 0x7fef7c19a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 -- 172.21.15.73:0/3872120125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 msgr2=0x7fef7c19eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 0x7fef7c19eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 -- 172.21.15.73:0/3872120125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c19f250 con 0x7fef7c06ab90 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef80c01640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 0x7fef7c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef73fff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9c0 tx=0x7fef6400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef79ffb640 1 -- 172.21.15.73:0/3872120125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6400cde0 con 0x7fef7c06ab90 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef79ffb640 1 -- 172.21.15.73:0/3872120125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef640035f0 con 0x7fef7c06ab90 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c19f540 con 0x7fef7c06ab90 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c1a6dd0 con 0x7fef7c06ab90 2024-06-06T07:26:06.121 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.118+0000 7fef79ffb640 1 -- 172.21.15.73:0/3872120125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef64010690 con 0x7fef7c06ab90 2024-06-06T07:26:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.121+0000 7fef79ffb640 1 -- 172.21.15.73:0/3872120125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fef640037a0 con 0x7fef7c06ab90 2024-06-06T07:26:06.122 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.122+0000 7fef79ffb640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 0x7fef680a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.122+0000 7fef85373740 1 Processor -- start 2024-06-06T07:26:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.122+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 0x7fef680a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.123 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.122+0000 7fef79ffb640 1 -- 172.21.15.73:0/3872120125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 9642+0+0 (secure 0 0 0) 0x7fef640cfd20 con 0x7fef7c06ab90 2024-06-06T07:26:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.122+0000 7fef85373740 1 -- start start 2024-06-06T07:26:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 0x7fef680a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009aa0 tx=0x7fef6c005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.124 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x558577390e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577393740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5585772ea4f0 con 0x558577386170 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558577288870 con 0x558577386a90 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef85373740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x5585772e64b0 con 0x558577391350 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47220/0 (socket says 172.21.15.73:47220) 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 -- 172.21.15.73:0/4291897249 learned_addr learned my addr 172.21.15.73:0/4291897249 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 -- 172.21.15.73:0/4291897249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 msgr2=0x558577393740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577393740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 -- 172.21.15.73:0/4291897249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 msgr2=0x558577390e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x558577390e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.123+0000 7fef7bfff640 1 -- 172.21.15.73:0/4291897249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585772a49b0 con 0x558577386170 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.124+0000 7fef7bfff640 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fef6c010dd0 tx=0x7fef6c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185698cf593ab519 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.125 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.124+0000 7fef727fc640 1 -- 172.21.15.73:0/4291897249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c023070 con 0x558577386170 2024-06-06T07:26:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.124+0000 7fef727fc640 1 -- 172.21.15.73:0/4291897249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef6c01f900 con 0x558577386170 2024-06-06T07:26:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.124+0000 7fef727fc640 1 -- 172.21.15.73:0/4291897249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c011820 con 0x558577386170 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 -- 172.21.15.73:0/4291897249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 msgr2=0x558577386550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fef6c010dd0 tx=0x7fef6c013000 comp rx=0 tx=0).stop 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 -- 172.21.15.73:0/4291897249 shutdown_connections 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577393740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x558577390e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 --2- 172.21.15.73:0/4291897249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x558577386550 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 -- 172.21.15.73:0/4291897249 >> 172.21.15.73:0/4291897249 conn(0x558577382680 msgr2=0x558577382a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.125+0000 7fef85373740 1 -- 172.21.15.73:0/4291897249 shutdown_connections 2024-06-06T07:26:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 -- 172.21.15.73:0/4291897249 wait complete. 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 -- start start 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 Processor -- start 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577386170 0x55857738e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577391350 0x55857738ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x558577393f80 con 0x558577386a90 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x558577273ae0 con 0x558577391350 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x558577394100 con 0x558577386170 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef85373740 10 client.0 _ll_register_callbacks cb 0x5585772ac9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.126+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47222/0 (socket says 172.21.15.73:47222) 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 -- 172.21.15.73:0/3465564384 learned_addr learned my addr 172.21.15.73:0/3465564384 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef80c01640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577391350 0x55857738ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577386170 0x55857738e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 -- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577386170 msgr2=0x55857738e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577386170 0x55857738e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 -- 172.21.15.73:0/3465564384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577391350 msgr2=0x55857738ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577391350 0x55857738ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 -- 172.21.15.73:0/3465564384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558576ffe210 con 0x558577386a90 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef73fff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fef640cd0c0 tx=0x7fef64003b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791c34a2f1186497 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6409f070 con 0x558577386a90 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9642+0+0 (secure 0 0 0) 0x7fef640ba770 con 0x558577386a90 2024-06-06T07:26:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.127+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef6c01f900 con 0x558577386a90 2024-06-06T07:26:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.128+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fef640a0190 con 0x558577386a90 2024-06-06T07:26:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.128+0000 7fef85373740 10 client.17847 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.128+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x5585770ca620 con 0x558577386a90 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.128+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef6c01f900 con 0x558577386a90 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.128+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fef640ba3f0 con 0x558577386a90 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 10 client.17847 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 10 client.17847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 20 client.17847 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x5585773a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 4 client.17847 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585772fdf10 con 0x5585773a14f0 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x5585773a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef5b7fe640 10 client.17847 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.129+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x5585773a38e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2727970a9459e7 server_cookie=c24dfc4292a44a85 in_seq=0 out_seq=0 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef5b7fe640 1 -- 172.21.15.73:0/3465564384 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5585772fdf10 con 0x5585773a14f0 2024-06-06T07:26:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef5b7fe640 10 client.17847 handle_command_reply: tid=0 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef85373740 1 client.17847 shutdown 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 msgr2=0x55857738e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fef640cd0c0 tx=0x7fef64003b10 comp rx=0 tx=0).stop 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 shutdown_connections 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.130+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x5585773a38e0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef5b7fe640 0 client.17847 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577391350 0x55857738ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x55857738e830 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 --2- 172.21.15.73:0/3465564384 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577386170 0x55857738e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 >> 172.21.15.73:0/3465564384 conn(0x558577382680 msgr2=0x558577388400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 shutdown_connections 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 1 -- 172.21.15.73:0/3465564384 wait complete. 2024-06-06T07:26:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.131+0000 7fef85373740 20 client.17847 trim_cache size 0 max 16384 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.152+0000 7fef85373740 1 Processor -- start 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.152+0000 7fef85373740 1 -- start start 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.152+0000 7fef85373740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.152+0000 7fef85373740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x5585773a9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef85373740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x55857738b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef85373740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1170 con 0x558577386170 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef85373740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b2cb0 con 0x558577386a90 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef85373740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1840 con 0x558577391350 2024-06-06T07:26:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef7bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47224/0 (socket says 172.21.15.73:47224) 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef7bfff640 1 -- 172.21.15.73:0/3635981638 learned_addr learned my addr 172.21.15.73:0/3635981638 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x5585773a9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 -- 172.21.15.73:0/3635981638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 msgr2=0x55857738b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x55857738b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 -- 172.21.15.73:0/3635981638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 msgr2=0x5585773a91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 -- 172.21.15.73:0/3635981638 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585773868c0 con 0x558577386a90 2024-06-06T07:26:06.155 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef7bfff640 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.153+0000 7fef73fff640 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x5585773a9710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef6409a090 tx=0x7fef64003e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b1936fddc6e8eee server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.154+0000 7fef71ffb640 1 -- 172.21.15.73:0/3635981638 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6409f070 con 0x558577386a90 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.154+0000 7fef71ffb640 1 -- 172.21.15.73:0/3635981638 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef64097df0 con 0x558577386a90 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.154+0000 7fef71ffb640 1 -- 172.21.15.73:0/3635981638 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef640a4820 con 0x558577386a90 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- 172.21.15.73:0/3635981638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 msgr2=0x5585773a9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x5585773a9710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fef6409a090 tx=0x7fef64003e10 comp rx=0 tx=0).stop 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- 172.21.15.73:0/3635981638 shutdown_connections 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x55857738b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386a90 0x5585773a9710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 --2- 172.21.15.73:0/3635981638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386170 0x5585773a91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- 172.21.15.73:0/3635981638 >> 172.21.15.73:0/3635981638 conn(0x558577382680 msgr2=0x55857738eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- 172.21.15.73:0/3635981638 shutdown_connections 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- 172.21.15.73:0/3635981638 wait complete. 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 -- start start 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.155+0000 7fef85373740 1 Processor -- start 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386170 0x558577399660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x558577399ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577394aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b25e0 con 0x558577386a90 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1b1f10 con 0x558577386170 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c1af170 con 0x558577391350 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef85373740 10 client.0 _ll_register_callbacks cb 0x7fef640046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x558577399ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef73fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x558577399ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47238/0 (socket says 172.21.15.73:47238) 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef73fff640 1 -- 172.21.15.73:0/2784204026 learned_addr learned my addr 172.21.15.73:0/2784204026 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577394aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 -- 172.21.15.73:0/2784204026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386170 msgr2=0x558577399660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386170 0x558577399660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 -- 172.21.15.73:0/2784204026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 msgr2=0x558577399ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x558577399ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.156+0000 7fef80c01640 1 -- 172.21.15.73:0/2784204026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585773952d0 con 0x558577391350 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef80c01640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577394aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef74034040 tx=0x7fef740306f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b1d9b7d1d3d92f server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.170:3300/0 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef74042070 con 0x558577391350 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef64097df0 con 0x558577391350 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mon.1 v2:172.21.15.170:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 9642+0+0 (secure 0 0 0) 0x7fef74003d80 con 0x558577391350 2024-06-06T07:26:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7fef74050050 con 0x558577391350 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef85373740 10 client.27217 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55857740bdc0 con 0x558577391350 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mon.1 v2:172.21.15.170:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef64097df0 con 0x558577391350 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mon.1 v2:172.21.15.170:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7fef74004540 con 0x558577391350 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.157+0000 7fef85373740 10 client.27217 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef85373740 10 client.27217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef85373740 20 client.27217 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x55857740e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef85373740 4 client.27217 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585772f7c20 con 0x5585773a14f0 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef7bfff640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x55857740e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef5b7fe640 10 client.27217 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:06.158+0000 7fef7bfff640 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x55857740e4d0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42b09a7c261b66d9 server_cookie=1388b1b298720ac9 in_seq=0 out_seq=0 2024-06-06T07:26:06.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109461 8.71 MB/sec execute 551 sec latency 6127.944 ms 2024-06-06T07:26:07.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:07 smithi073 ceph-mon[32303]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-06-06T07:26:07.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:07 smithi136 ceph-mon[35901]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-06-06T07:26:07.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:07 smithi170 ceph-mon[36108]: pgmap v789: 129 pgs: 129 active+clean; 1.6 GiB data, 9.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 6.3 MiB/s wr, 328 op/s 2024-06-06T07:26:07.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109628 8.72 MB/sec execute 552 sec latency 7128.071 ms 2024-06-06T07:26:08.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109790 8.72 MB/sec execute 553 sec latency 8128.107 ms 2024-06-06T07:26:09.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:09 smithi073 ceph-mon[32303]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 332 op/s 2024-06-06T07:26:09.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:09 smithi136 ceph-mon[35901]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 332 op/s 2024-06-06T07:26:09.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:09 smithi170 ceph-mon[36108]: pgmap v790: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 7.0 MiB/s wr, 332 op/s 2024-06-06T07:26:09.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 109917 8.70 MB/sec execute 554 sec latency 9128.154 ms 2024-06-06T07:26:10.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110058 8.70 MB/sec execute 555 sec latency 10128.187 ms 2024-06-06T07:26:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:11 smithi073 ceph-mon[32303]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:11.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:11 smithi136 ceph-mon[35901]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:11.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:11 smithi170 ceph-mon[36108]: pgmap v791: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:11.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110227 8.70 MB/sec execute 556 sec latency 11128.220 ms 2024-06-06T07:26:12.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110412 8.71 MB/sec execute 557 sec latency 12128.254 ms 2024-06-06T07:26:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:13 smithi073 ceph-mon[32303]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:13 smithi073 ceph-mon[32303]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T07:26:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:13 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:26:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:13 smithi136 ceph-mon[35901]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:13 smithi136 ceph-mon[35901]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T07:26:13.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:13 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:26:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:13 smithi170 ceph-mon[36108]: pgmap v792: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.9 MiB/s wr, 334 op/s 2024-06-06T07:26:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:13 smithi170 ceph-mon[36108]: osdmap e146: 12 total, 12 up, 12 in 2024-06-06T07:26:13.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:13 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:26:13.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110554 8.70 MB/sec execute 558 sec latency 13128.288 ms 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.856+0000 7fef5b7fe640 1 -- 172.21.15.73:0/2784204026 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5585772f7c20 con 0x5585773a14f0 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.856+0000 7fef5b7fe640 10 client.27217 handle_command_reply: tid=0 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 client.27217 shutdown 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 msgr2=0x558577394aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577394aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fef74034040 tx=0x7fef740306f0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 shutdown_connections 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x5585773a14f0 0x55857740e4d0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x558577391350 0x558577394aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.858 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558577386a90 0x558577399ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 --2- 172.21.15.73:0/2784204026 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x558577386170 0x558577399660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 >> 172.21.15.73:0/2784204026 conn(0x558577382680 msgr2=0x55857739c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:13.859 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef5b7fe640 0 client.27217 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.857+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 shutdown_connections 2024-06-06T07:26:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.858+0000 7fef85373740 1 -- 172.21.15.73:0/2784204026 wait complete. 2024-06-06T07:26:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.858+0000 7fef85373740 20 client.27217 trim_cache size 0 max 16384 2024-06-06T07:26:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:26:13.860 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:26:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:26:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 msgr2=0x7fef680a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:13.861 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 0x7fef680a5260 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fef6c009aa0 tx=0x7fef6c005f90 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 msgr2=0x7fef7c19a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fef6400e9c0 tx=0x7fef6400ee80 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 shutdown_connections 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fef7c107ba0 0x7fef7c19eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef7c06ab90 0x7fef7c19a7a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fef680a2da0 0x7fef680a5260 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 --2- 172.21.15.73:0/3872120125 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fef7c06a1d0 0x7fef7c19a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 >> 172.21.15.73:0/3872120125 conn(0x7fef7c0766a0 msgr2=0x7fef7c0ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:13.862 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 shutdown_connections 2024-06-06T07:26:13.863 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:13.859+0000 7fef82664640 1 -- 172.21.15.73:0/3872120125 wait complete. 2024-06-06T07:26:14.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110675 8.70 MB/sec execute 559 sec latency 13140.382 ms 2024-06-06T07:26:15.251 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:15 smithi073 ceph-mon[32303]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 325 op/s 2024-06-06T07:26:15.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:15 smithi136 ceph-mon[35901]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 325 op/s 2024-06-06T07:26:15.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:15 smithi170 ceph-mon[36108]: pgmap v794: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 6.6 MiB/s wr, 325 op/s 2024-06-06T07:26:15.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110815 8.70 MB/sec execute 560 sec latency 170.398 ms 2024-06-06T07:26:16.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 110957 8.70 MB/sec execute 561 sec latency 73.512 ms 2024-06-06T07:26:17.491 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:17 smithi073 ceph-mon[32303]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-06-06T07:26:17.552 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:17 smithi136 ceph-mon[35901]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-06-06T07:26:17.573 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:17 smithi170 ceph-mon[36108]: pgmap v795: 129 pgs: 129 active+clean; 1.6 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 6.5 MiB/s wr, 321 op/s 2024-06-06T07:26:17.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111085 8.70 MB/sec execute 562 sec latency 81.611 ms 2024-06-06T07:26:18.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111235 8.70 MB/sec execute 563 sec latency 74.076 ms 2024-06-06T07:26:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:19 smithi136 ceph-mon[35901]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-06-06T07:26:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:19 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:19 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:19 smithi073 ceph-mon[32303]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-06-06T07:26:19.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:19 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:19 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:19 smithi170 ceph-mon[36108]: pgmap v796: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 310 op/s 2024-06-06T07:26:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:19 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:19 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:19.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111363 8.69 MB/sec execute 564 sec latency 101.616 ms 2024-06-06T07:26:20.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111484 8.68 MB/sec execute 565 sec latency 95.448 ms 2024-06-06T07:26:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:21 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.347 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:21 smithi170 ceph-mon[36108]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:21 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:21 smithi073 ceph-mon[32303]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:21 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:21.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:21 smithi136 ceph-mon[35901]: pgmap v797: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:21.840 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111625 8.68 MB/sec execute 566 sec latency 149.151 ms 2024-06-06T07:26:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:26:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:26:22.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:26:22.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:26:22.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:26:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:26:22.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:26:22.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111775 8.67 MB/sec execute 567 sec latency 99.204 ms 2024-06-06T07:26:23.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:23 smithi170 ceph-mon[36108]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:23 smithi073 ceph-mon[32303]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:23 smithi136 ceph-mon[35901]: pgmap v798: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 6.1 MiB/s wr, 308 op/s 2024-06-06T07:26:23.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 111951 8.68 MB/sec execute 568 sec latency 108.709 ms 2024-06-06T07:26:24.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112122 8.68 MB/sec execute 569 sec latency 85.305 ms 2024-06-06T07:26:25.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:25 smithi170 ceph-mon[36108]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 394 op/s 2024-06-06T07:26:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:25 smithi073 ceph-mon[32303]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 394 op/s 2024-06-06T07:26:25.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:25 smithi136 ceph-mon[35901]: pgmap v799: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.0 MiB/s wr, 394 op/s 2024-06-06T07:26:25.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112294 8.69 MB/sec execute 570 sec latency 99.681 ms 2024-06-06T07:26:26.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112445 8.69 MB/sec execute 571 sec latency 83.630 ms 2024-06-06T07:26:26.996 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.995+0000 7f79ee7b7640 1 -- 172.21.15.73:0/3743527719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 msgr2=0x7f79e810d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.995+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/3743527719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810d750 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009cd0 tx=0x7f79e002a8e0 comp rx=0 tx=0).stop 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 -- 172.21.15.73:0/3743527719 shutdown_connections 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/3743527719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810d750 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/3743527719 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e81008d0 0x7f79e8100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/3743527719 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e8106890 0x7f79e8106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:26.997 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 -- 172.21.15.73:0/3743527719 >> 172.21.15.73:0/3743527719 conn(0x7f79e8077a20 msgr2=0x7f79e8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.996+0000 7f79ee7b7640 1 -- 172.21.15.73:0/3743527719 shutdown_connections 2024-06-06T07:26:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.997+0000 7f79ee7b7640 1 -- 172.21.15.73:0/3743527719 wait complete. 2024-06-06T07:26:26.998 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 Processor -- start 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 -- start start 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e8106890 0x7f79e810b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810efc0 con 0x7f79e8101290 2024-06-06T07:26:27.001 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810ee40 con 0x7f79e8106890 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79ee7b7640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79e810f140 con 0x7f79e81008d0 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79e7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79e7fff640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:56860/0 (socket says 172.21.15.73:56860) 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.998+0000 7f79e7fff640 1 -- 172.21.15.73:0/512414345 learned_addr learned my addr 172.21.15.73:0/512414345 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e77fe640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 -- 172.21.15.73:0/512414345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e8106890 msgr2=0x7f79e810b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e8106890 0x7f79e810b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 -- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 msgr2=0x7f79e810b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 -- 172.21.15.73:0/512414345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e810beb0 con 0x7f79e81008d0 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:26.999+0000 7f79e7fff640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79d80097d0 tx=0x7f79d8009c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.002 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.000+0000 7f79e57fa640 1 -- 172.21.15.73:0/512414345 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8005020 con 0x7f79e81008d0 2024-06-06T07:26:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.000+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e810c1a0 con 0x7f79e81008d0 2024-06-06T07:26:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.000+0000 7f79e57fa640 1 -- 172.21.15.73:0/512414345 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79d8005670 con 0x7f79e81008d0 2024-06-06T07:26:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.000+0000 7f79e57fa640 1 -- 172.21.15.73:0/512414345 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8004a00 con 0x7f79e81008d0 2024-06-06T07:26:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.000+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e81ab190 con 0x7f79e81008d0 2024-06-06T07:26:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.003+0000 7f79e57fa640 1 -- 172.21.15.73:0/512414345 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f79d801b020 con 0x7f79e81008d0 2024-06-06T07:26:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.004+0000 7f79e57fa640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 0x7f79d00a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.004+0000 7f79e77fe640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 0x7f79d00a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.005+0000 7f79e57fa640 1 -- 172.21.15.73:0/512414345 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f79d80ce850 con 0x7f79e81008d0 2024-06-06T07:26:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.005+0000 7f79e77fe640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 0x7f79d00a5260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f79e810ce10 tx=0x7f79dc005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 Processor -- start 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 -- start start 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916aa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d29160e4f0 con 0x55d2916b5350 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d2915ac870 con 0x55d2916aa170 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79f15a2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d29160a4b0 con 0x55d2916aaa90 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37092/0 (socket says 172.21.15.73:37092) 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.006+0000 7f79ecd54640 1 -- 172.21.15.73:0/2566911742 learned_addr learned my addr 172.21.15.73:0/2566911742 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79e7fff640 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79e77fe640 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 -- 172.21.15.73:0/2566911742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 msgr2=0x55d2916b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 -- 172.21.15.73:0/2566911742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 msgr2=0x55d2916aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 -- 172.21.15.73:0/2566911742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2915c89b0 con 0x55d2916b5350 2024-06-06T07:26:27.009 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009df0 tx=0x7f79e0005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77050d1cfd2d63b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.007+0000 7f79ce7fc640 1 -- 172.21.15.73:0/2566911742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e003f070 con 0x55d2916b5350 2024-06-06T07:26:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.008+0000 7f79ce7fc640 1 -- 172.21.15.73:0/2566911742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79e0036040 con 0x55d2916b5350 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.008+0000 7f79ce7fc640 1 -- 172.21.15.73:0/2566911742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e00045a0 con 0x55d2916b5350 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.008+0000 7f79f15a2740 1 -- 172.21.15.73:0/2566911742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 msgr2=0x55d2916b7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.008+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f79e0009df0 tx=0x7f79e0005c70 comp rx=0 tx=0).stop 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 -- 172.21.15.73:0/2566911742 shutdown_connections 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916b5350 0x55d2916b7740 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2566911742 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 -- 172.21.15.73:0/2566911742 >> 172.21.15.73:0/2566911742 conn(0x55d2916a6680 msgr2=0x55d2916a6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 -- 172.21.15.73:0/2566911742 shutdown_connections 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 -- 172.21.15.73:0/2566911742 wait complete. 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 -- start start 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.009+0000 7f79f15a2740 1 Processor -- start 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916b5350 0x55d2916b2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d2916b7f20 con 0x55d2916aa170 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x55d291597ae0 con 0x55d2916b5350 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x55d2916b80a0 con 0x55d2916aaa90 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79f15a2740 10 client.0 _ll_register_callbacks cb 0x55d2915d09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37106/0 (socket says 172.21.15.73:37106) 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 -- 172.21.15.73:0/922420935 learned_addr learned my addr 172.21.15.73:0/922420935 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79ecd54640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916b5350 0x55d2916b2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e77fe640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 -- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 msgr2=0x55d2916b2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b2920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 -- 172.21.15.73:0/922420935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916b5350 msgr2=0x55d2916b2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916b5350 0x55d2916b2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 -- 172.21.15.73:0/922420935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d291322210 con 0x55d2916aa170 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79e7fff640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e490 tx=0x7f79d809b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d382f79528a9de6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80b8060 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.010+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f79d800e9c0 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79e0036040 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f79d80a35d0 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 10 client.17865 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55d2913ee620 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79d80b9250 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f79d8004ba0 con 0x55d2916aa170 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 10 client.17865 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 10 client.17865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 20 client.17865 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916c55c0 0x55d2916c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 4 client.17865 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:26:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f79e0036040 con 0x55d2916c55c0 2024-06-06T07:26:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.011+0000 7f79e77fe640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916c55c0 0x55d2916c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.012+0000 7f79e77fe640 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916c55c0 0x55d2916c79b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5876dc1664fc8358 server_cookie=9bfa71cb3ea272dd in_seq=0 out_seq=0 2024-06-06T07:26:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.012+0000 7f79c77fe640 10 client.17865 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79c77fe640 1 -- 172.21.15.73:0/922420935 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f79e0036040 con 0x55d2916c55c0 2024-06-06T07:26:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79c77fe640 10 client.17865 handle_command_reply: tid=0 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79f15a2740 1 client.17865 shutdown 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 msgr2=0x55d2916b23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f79d800e490 tx=0x7f79d809b120 comp rx=0 tx=0).stop 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 shutdown_connections 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.014+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916c55c0 0x55d2916c79b0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916b5350 0x55d2916b2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aaa90 0x55d2916b2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 --2- 172.21.15.73:0/922420935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aa170 0x55d2916b23e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79c77fe640 0 client.17865 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 >> 172.21.15.73:0/922420935 conn(0x55d2916a6680 msgr2=0x55d2916ac3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 shutdown_connections 2024-06-06T07:26:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 1 -- 172.21.15.73:0/922420935 wait complete. 2024-06-06T07:26:27.021 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.015+0000 7f79f15a2740 20 client.17865 trim_cache size 0 max 16384 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 Processor -- start 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 -- start start 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916a6680 0x55d2916acb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b54c0 con 0x55d2916aaa90 2024-06-06T07:26:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b7000 con 0x55d2916aa170 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.038+0000 7f79f15a2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b5b90 con 0x55d2916a6680 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37110/0 (socket says 172.21.15.73:37110) 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79ecd54640 1 -- 172.21.15.73:0/2601986315 learned_addr learned my addr 172.21.15.73:0/2601986315 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e7fff640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916a6680 0x55d2916acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 -- 172.21.15.73:0/2601986315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916a6680 msgr2=0x55d2916acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916a6680 0x55d2916acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 -- 172.21.15.73:0/2601986315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 msgr2=0x55d2916cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 -- 172.21.15.73:0/2601986315 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2916a1fe0 con 0x55d2916aa170 2024-06-06T07:26:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79e77fe640 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916a5660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2916c8210 tx=0x7f79dc014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad12713b7748ec6b server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.039+0000 7f79cdffb640 1 -- 172.21.15.73:0/2601986315 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc032440 con 0x55d2916aa170 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.040+0000 7f79cdffb640 1 -- 172.21.15.73:0/2601986315 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79dc032a90 con 0x55d2916aa170 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.040+0000 7f79cdffb640 1 -- 172.21.15.73:0/2601986315 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79dc02aaf0 con 0x55d2916aa170 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.040+0000 7f79f15a2740 1 -- 172.21.15.73:0/2601986315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 msgr2=0x55d2916a5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.040+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916a5660 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x55d2916c8210 tx=0x7f79dc014000 comp rx=0 tx=0).stop 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- 172.21.15.73:0/2601986315 shutdown_connections 2024-06-06T07:26:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916aa170 0x55d2916a5660 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2601986315 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916a6680 0x55d2916acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- 172.21.15.73:0/2601986315 >> 172.21.15.73:0/2601986315 conn(0x55d2916addb0 msgr2=0x55d2916a4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- 172.21.15.73:0/2601986315 shutdown_connections 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- 172.21.15.73:0/2601986315 wait complete. 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- start start 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 Processor -- start 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916a6680 0x55d2916bd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aa170 0x55d2916b8c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b6930 con 0x55d2916aaa90 2024-06-06T07:26:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b6260 con 0x55d2916a6680 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f79e81b34c0 con 0x55d2916aa170 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.041+0000 7f79f15a2740 10 client.0 _ll_register_callbacks cb 0x7f79d800b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37122/0 (socket says 172.21.15.73:37122) 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79e77fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aa170 0x55d2916b8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 -- 172.21.15.73:0/2822153673 learned_addr learned my addr 172.21.15.73:0/2822153673 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79e7fff640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916a6680 0x55d2916bd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 -- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aa170 msgr2=0x55d2916b8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.050 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aa170 0x55d2916b8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 -- 172.21.15.73:0/2822153673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916a6680 msgr2=0x55d2916bd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916a6680 0x55d2916bd8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.042+0000 7f79ecd54640 1 -- 172.21.15.73:0/2822153673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2916b97a0 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.043+0000 7f79ecd54640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e0031870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7b3c11ff6059f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.044+0000 7f79c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.044+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e004b020 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.044+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f79e00305a0 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.044+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79dc032a90 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.044+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f79e003f040 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.045+0000 7f79f15a2740 10 client.17871 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.045+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x55d2917300c0 con 0x55d2916aaa90 2024-06-06T07:26:27.051 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.045+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79e0036040 con 0x55d2916aaa90 2024-06-06T07:26:27.052 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.046+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f79e0030070 con 0x55d2916aaa90 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 10 client.17871 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 10 client.17871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 20 client.17871 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916b5350 0x55d291732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 4 client.17871 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.047+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f79dc032a90 con 0x55d2916b5350 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.049+0000 7f79e7fff640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916b5350 0x55d291732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.050+0000 7f79e7fff640 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916b5350 0x55d291732410 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b126d48e94011f98 server_cookie=60d25bd421cdb2d7 in_seq=0 out_seq=0 2024-06-06T07:26:27.053 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.050+0000 7f79c77fe640 10 client.17871 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79c77fe640 1 -- 172.21.15.73:0/2822153673 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f79dc032a90 con 0x55d2916b5350 2024-06-06T07:26:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79c77fe640 10 client.17871 handle_command_reply: tid=0 2024-06-06T07:26:27.178 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 client.17871 shutdown 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 msgr2=0x55d2916b9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f79e0006010 tx=0x7f79e0031870 comp rx=0 tx=0).stop 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 shutdown_connections 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x55d2916b5350 0x55d291732410 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d2916aaa90 0x55d2916b9160 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x55d2916aa170 0x55d2916b8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79c77fe640 0 client.17871 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 --2- 172.21.15.73:0/2822153673 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x55d2916a6680 0x55d2916bd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 >> 172.21.15.73:0/2822153673 conn(0x55d2916addb0 msgr2=0x55d2916b2c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 shutdown_connections 2024-06-06T07:26:27.179 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.177+0000 7f79f15a2740 1 -- 172.21.15.73:0/2822153673 wait complete. 2024-06-06T07:26:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.178+0000 7f79f15a2740 20 client.17871 trim_cache size 0 max 16384 2024-06-06T07:26:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:26:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:26:27.180 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 msgr2=0x7f79d00a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 0x7f79d00a5260 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f79e810ce10 tx=0x7f79dc005f90 comp rx=0 tx=0).stop 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 msgr2=0x7f79e810d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f79d80097d0 tx=0x7f79d8009c90 comp rx=0 tx=0).stop 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 shutdown_connections 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f79e8106890 0x7f79e810b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.181 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e8101290 0x7f79e810b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f79d00a2da0 0x7f79d00a5260 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 --2- 172.21.15.73:0/512414345 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f79e81008d0 0x7f79e810d280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 >> 172.21.15.73:0/512414345 conn(0x7f79e8077a20 msgr2=0x7f79e81033b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 shutdown_connections 2024-06-06T07:26:27.182 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:27.179+0000 7f79ee7b7640 1 -- 172.21.15.73:0/512414345 wait complete. 2024-06-06T07:26:27.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:27 smithi170 ceph-mon[36108]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-06-06T07:26:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:27 smithi073 ceph-mon[32303]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-06-06T07:26:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:27 smithi136 ceph-mon[35901]: pgmap v800: 129 pgs: 129 active+clean; 1.6 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 4.8 MiB/s wr, 247 op/s 2024-06-06T07:26:27.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112594 8.69 MB/sec execute 572 sec latency 111.440 ms 2024-06-06T07:26:28.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112765 8.69 MB/sec execute 573 sec latency 80.796 ms 2024-06-06T07:26:29.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:29 smithi170 ceph-mon[36108]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 394 op/s 2024-06-06T07:26:29.735 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:29 smithi073 ceph-mon[32303]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 394 op/s 2024-06-06T07:26:29.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:29 smithi136 ceph-mon[35901]: pgmap v801: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 7.6 MiB/s wr, 394 op/s 2024-06-06T07:26:29.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 112894 8.68 MB/sec execute 574 sec latency 115.726 ms 2024-06-06T07:26:30.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113019 8.67 MB/sec execute 575 sec latency 123.724 ms 2024-06-06T07:26:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:31 smithi073 ceph-mon[32303]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-06-06T07:26:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:31 smithi136 ceph-mon[35901]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-06-06T07:26:31.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113184 8.67 MB/sec execute 576 sec latency 84.178 ms 2024-06-06T07:26:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:31 smithi170 ceph-mon[36108]: pgmap v802: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 270 op/s 2024-06-06T07:26:32.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113368 8.67 MB/sec execute 577 sec latency 76.294 ms 2024-06-06T07:26:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:33 smithi073 ceph-mon[32303]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:26:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:33 smithi136 ceph-mon[35901]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:26:33.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113520 8.67 MB/sec execute 578 sec latency 154.600 ms 2024-06-06T07:26:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:33 smithi170 ceph-mon[36108]: pgmap v803: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 5.3 MiB/s wr, 269 op/s 2024-06-06T07:26:34.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113702 8.68 MB/sec execute 579 sec latency 129.737 ms 2024-06-06T07:26:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:35 smithi073 ceph-mon[32303]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 403 op/s 2024-06-06T07:26:35.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:35 smithi136 ceph-mon[35901]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 403 op/s 2024-06-06T07:26:35.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 113858 8.68 MB/sec execute 580 sec latency 74.075 ms 2024-06-06T07:26:35.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:35 smithi170 ceph-mon[36108]: pgmap v804: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 7.7 MiB/s wr, 403 op/s 2024-06-06T07:26:36.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114061 8.69 MB/sec execute 581 sec latency 89.373 ms 2024-06-06T07:26:37.613 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:37 smithi136 ceph-mon[35901]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 285 op/s 2024-06-06T07:26:37.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:37 smithi170 ceph-mon[36108]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 285 op/s 2024-06-06T07:26:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:37 smithi073 ceph-mon[32303]: pgmap v805: 129 pgs: 129 active+clean; 1.6 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.2 MiB/s wr, 285 op/s 2024-06-06T07:26:37.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114215 8.69 MB/sec execute 582 sec latency 92.715 ms 2024-06-06T07:26:38.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114354 8.69 MB/sec execute 583 sec latency 70.635 ms 2024-06-06T07:26:39.750 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:39 smithi136 ceph-mon[35901]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-06-06T07:26:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:39 smithi073 ceph-mon[32303]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-06-06T07:26:39.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114508 8.68 MB/sec execute 584 sec latency 81.127 ms 2024-06-06T07:26:39.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:39 smithi170 ceph-mon[36108]: pgmap v806: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 424 op/s 2024-06-06T07:26:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 -- 172.21.15.73:0/1841269166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 msgr2=0x7f2830107d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 --2- 172.21.15.73:0/1841269166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f2830107d70 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f2824009930 tx=0x7f282402a420 comp rx=0 tx=0).stop 2024-06-06T07:26:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 -- 172.21.15.73:0/1841269166 shutdown_connections 2024-06-06T07:26:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 --2- 172.21.15.73:0/1841269166 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2830102340 0x7f283010e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 --2- 172.21.15.73:0/1841269166 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2830101980 0x7f2830101e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 --2- 172.21.15.73:0/1841269166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f2830107d70 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 -- 172.21.15.73:0/1841269166 >> 172.21.15.73:0/1841269166 conn(0x7f28300fd450 msgr2=0x7f28300ff870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.310+0000 7f2837231640 1 -- 172.21.15.73:0/1841269166 shutdown_connections 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 -- 172.21.15.73:0/1841269166 wait complete. 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 Processor -- start 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 -- start start 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2830101980 0x7f2830198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2830102340 0x7f2830198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.311+0000 7f2837231640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f2837231640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2830111c50 con 0x7f2830107990 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f2837231640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f2830111ad0 con 0x7f2830101980 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f2837231640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f2830111dd0 con 0x7f2830102340 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44816/0 (socket says 172.21.15.73:44816) 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 -- 172.21.15.73:0/3961539568 learned_addr learned my addr 172.21.15.73:0/3961539568 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 -- 172.21.15.73:0/3961539568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2830102340 msgr2=0x7f2830198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2830102340 0x7f2830198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 -- 172.21.15.73:0/3961539568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2830101980 msgr2=0x7f2830198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2830101980 0x7f2830198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 -- 172.21.15.73:0/3961539568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f283019d120 con 0x7f2830107990 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f28357ce640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f282800eaa0 tx=0x7f282800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f282dffb640 1 -- 172.21.15.73:0/3961539568 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282800cdb0 con 0x7f2830107990 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f282dffb640 1 -- 172.21.15.73:0/3961539568 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2828005020 con 0x7f2830107990 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.312+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f283019d410 con 0x7f2830107990 2024-06-06T07:26:40.317 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.313+0000 7f282dffb640 1 -- 172.21.15.73:0/3961539568 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280116d0 con 0x7f2830107990 2024-06-06T07:26:40.318 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.313+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28301a4ca0 con 0x7f2830107990 2024-06-06T07:26:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.315+0000 7f282dffb640 1 -- 172.21.15.73:0/3961539568 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f28280051d0 con 0x7f2830107990 2024-06-06T07:26:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 Processor -- start 2024-06-06T07:26:40.327 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 -- start start 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f282dffb640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 0x7f28200a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b49741220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49743b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b496964f0 con 0x560b49741760 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b49634870 con 0x560b49732030 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.316+0000 7f283a01c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560b496924b0 con 0x560b49732950 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f282ffff640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 0x7f28200a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f282dffb640 1 -- 172.21.15.73:0/3961539568 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f28280cf9f0 con 0x7f2830107990 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59980/0 (socket says 172.21.15.73:59980) 2024-06-06T07:26:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f2834fcd640 1 -- 172.21.15.73:0/3290051421 learned_addr learned my addr 172.21.15.73:0/3290051421 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49743b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f282ffff640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b49741220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 -- 172.21.15.73:0/3290051421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 msgr2=0x560b49741220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b49741220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 -- 172.21.15.73:0/3290051421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 msgr2=0x560b49732410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 -- 172.21.15.73:0/3290051421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b496509b0 con 0x560b49741760 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f2834fcd640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f282ffff640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b49741220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f282ffff640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 0x7f28200a52e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f281c004650 tx=0x7f281c011380 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f28357ce640 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49743b50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f28280ccd30 tx=0x7f282809bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94125222ad55c8e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.317+0000 7f281affd640 1 -- 172.21.15.73:0/3290051421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28280b4070 con 0x560b49741760 2024-06-06T07:26:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f281affd640 1 -- 172.21.15.73:0/3290051421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28280a4770 con 0x560b49741760 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f281affd640 1 -- 172.21.15.73:0/3290051421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282801f070 con 0x560b49741760 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 -- 172.21.15.73:0/3290051421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 msgr2=0x560b49743b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49743b50 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f28280ccd30 tx=0x7f282809bb00 comp rx=0 tx=0).stop 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 -- 172.21.15.73:0/3290051421 shutdown_connections 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49743b50 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b49741220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 --2- 172.21.15.73:0/3290051421 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b49732410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.318+0000 7f283a01c740 1 -- 172.21.15.73:0/3290051421 >> 172.21.15.73:0/3290051421 conn(0x560b4972e680 msgr2=0x560b4972ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.319+0000 7f283a01c740 1 -- 172.21.15.73:0/3290051421 shutdown_connections 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.319+0000 7f283a01c740 1 -- 172.21.15.73:0/3290051421 wait complete. 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.319+0000 7f283a01c740 1 -- start start 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.319+0000 7f283a01c740 1 Processor -- start 2024-06-06T07:26:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.319+0000 7f283a01c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b4973ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b4973f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560b497442d0 con 0x560b49741760 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:59986/0 (socket says 172.21.15.73:59986) 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f2834fcd640 1 -- 172.21.15.73:0/3728280598 learned_addr learned my addr 172.21.15.73:0/3728280598 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f282ffff640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b4973ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x560b4961fae0 con 0x560b49732030 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x560b49744450 con 0x560b49732950 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b4973f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f283a01c740 10 client.0 _ll_register_callbacks cb 0x560b496589b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 -- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 msgr2=0x560b4973ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b4973ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 -- 172.21.15.73:0/3728280598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 msgr2=0x560b4973e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f28357ce640 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b493aa210 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.320+0000 7f2834fcd640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f28357ce640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b4973f200 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f28280984d0 tx=0x7f2828098380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd94799347257562 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f282801f050 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28280a4770 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f28280ba300 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f2828082050 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 10 client.17883 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x560b49476620 con 0x560b49741760 2024-06-06T07:26:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28280a4770 con 0x560b49741760 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f282800f040 con 0x560b49741760 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 10 client.17883 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 10 client.17883 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 20 client.17883 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b49751810 0x560b49753c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 4 client.17883 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.321+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b496a9f10 con 0x560b49751810 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.322+0000 7f2834fcd640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b49751810 0x560b49753c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.322+0000 7f280ffff640 10 client.17883 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.322+0000 7f2834fcd640 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b49751810 0x560b49753c00 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7132dd4f3ff563d7 server_cookie=7822b31c7bb1a29 in_seq=0 out_seq=0 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f280ffff640 1 -- 172.21.15.73:0/3728280598 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560b496a9f10 con 0x560b49751810 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f280ffff640 10 client.17883 handle_command_reply: tid=0 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 client.17883 shutdown 2024-06-06T07:26:40.333 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 msgr2=0x560b4973f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b4973f200 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f28280984d0 tx=0x7f2828098380 comp rx=0 tx=0).stop 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 shutdown_connections 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b49751810 0x560b49753c00 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b4973f200 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f280ffff640 0 client.17883 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49732950 0x560b4973ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 --2- 172.21.15.73:0/3728280598 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732030 0x560b4973e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 >> 172.21.15.73:0/3728280598 conn(0x560b4972e680 msgr2=0x560b49734420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 shutdown_connections 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 1 -- 172.21.15.73:0/3728280598 wait complete. 2024-06-06T07:26:40.334 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.324+0000 7f283a01c740 20 client.17883 trim_cache size 0 max 16384 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 Processor -- start 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 -- start start 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b49759a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49741760 0x560b4973e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f28301ace70 con 0x560b49732030 2024-06-06T07:26:40.347 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f28301ae9b0 con 0x560b49732950 2024-06-06T07:26:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f283a01c740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28301ad540 con 0x560b49741760 2024-06-06T07:26:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f2834fcd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44846/0 (socket says 172.21.15.73:44846) 2024-06-06T07:26:40.348 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.346+0000 7f2834fcd640 1 -- 172.21.15.73:0/993983990 learned_addr learned my addr 172.21.15.73:0/993983990 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 -- 172.21.15.73:0/993983990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49741760 msgr2=0x560b4973e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49741760 0x560b4973e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 -- 172.21.15.73:0/993983990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 msgr2=0x560b49759a30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b49759a30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 -- 172.21.15.73:0/993983990 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b497515c0 con 0x560b49732030 2024-06-06T07:26:40.349 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f2834fcd640 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f282402a940 tx=0x7f2824031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8183060f7c258a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f281a7fc640 1 -- 172.21.15.73:0/993983990 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824035040 con 0x560b49732030 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f281a7fc640 1 -- 172.21.15.73:0/993983990 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2824005040 con 0x560b49732030 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.347+0000 7f281a7fc640 1 -- 172.21.15.73:0/993983990 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2824004950 con 0x560b49732030 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 -- 172.21.15.73:0/993983990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 msgr2=0x560b497594f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f282402a940 tx=0x7f2824031400 comp rx=0 tx=0).stop 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 -- 172.21.15.73:0/993983990 shutdown_connections 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49741760 0x560b4973e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b49759a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 --2- 172.21.15.73:0/993983990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49732030 0x560b497594f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.348+0000 7f283a01c740 1 -- 172.21.15.73:0/993983990 >> 172.21.15.73:0/993983990 conn(0x560b4972e680 msgr2=0x560b49739a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.349+0000 7f283a01c740 1 -- 172.21.15.73:0/993983990 shutdown_connections 2024-06-06T07:26:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.349+0000 7f283a01c740 1 -- 172.21.15.73:0/993983990 wait complete. 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.349+0000 7f283a01c740 1 -- start start 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.349+0000 7f283a01c740 1 Processor -- start 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.349+0000 7f283a01c740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b497389b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49751810 0x560b497352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f282ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f282ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44858/0 (socket says 172.21.15.73:44858) 2024-06-06T07:26:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f282ffff640 1 -- 172.21.15.73:0/569189175 learned_addr learned my addr 172.21.15.73:0/569189175 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f2834fcd640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b497389b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f28301ae2e0 con 0x560b49741760 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f28301adc10 con 0x560b49732950 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f28301aae70 con 0x560b49751810 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.350+0000 7f283a01c740 10 client.0 _ll_register_callbacks cb 0x7f28280b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 -- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49751810 msgr2=0x560b497352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49751810 0x560b497352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 -- 172.21.15.73:0/569189175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 msgr2=0x560b497389b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b497389b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b49735a00 con 0x560b49741760 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f282ffff640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f281c005d00 tx=0x7f281c005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9897b958bfb686cb server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f280ffff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f281c023070 con 0x560b49741760 2024-06-06T07:26:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.351+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2824005040 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.352+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f281c028660 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.352+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7f281c031050 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.352+0000 7f283a01c740 10 client.17889 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.352+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=84,monmap=4+}) -- 0x560b497499f0 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.352+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2824005040 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7f281c01f960 con 0x560b49741760 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 10 client.17889 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 10 client.17889 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 20 client.17889 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b497c0cb0 0x560b497c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:40.355 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 4 client.17889 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:26:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560b496a3c20 con 0x560b497c0cb0 2024-06-06T07:26:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.353+0000 7f2834fcd640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b497c0cb0 0x560b497c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.354+0000 7f280ffff640 10 client.17889 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:40.356 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.355+0000 7f2834fcd640 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b497c0cb0 0x560b497c30a0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=743b3b78e88fdb72 server_cookie=38c69c038b3fa10c in_seq=0 out_seq=0 2024-06-06T07:26:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f280ffff640 1 -- 172.21.15.73:0/569189175 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560b496a3c20 con 0x560b497c0cb0 2024-06-06T07:26:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f280ffff640 10 client.17889 handle_command_reply: tid=0 2024-06-06T07:26:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 client.17889 shutdown 2024-06-06T07:26:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 msgr2=0x560b49734db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.461 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f281c005d00 tx=0x7f281c005c90 comp rx=0 tx=0).stop 2024-06-06T07:26:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 shutdown_connections 2024-06-06T07:26:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x560b497c0cb0 0x560b497c30a0 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x560b49751810 0x560b497352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560b49741760 0x560b49734db0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.462 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f280ffff640 0 client.17889 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 --2- 172.21.15.73:0/569189175 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x560b49732950 0x560b497389b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 >> 172.21.15.73:0/569189175 conn(0x560b4972e680 msgr2=0x560b4972cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.459+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 shutdown_connections 2024-06-06T07:26:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f283a01c740 1 -- 172.21.15.73:0/569189175 wait complete. 2024-06-06T07:26:40.463 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f283a01c740 20 client.17889 trim_cache size 0 max 16384 2024-06-06T07:26:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:26:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:26:40.464 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 msgr2=0x7f28200a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 0x7f28200a52e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f281c004650 tx=0x7f281c011380 comp rx=0 tx=0).stop 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 msgr2=0x7f283019c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f282800eaa0 tx=0x7f282800ef60 comp rx=0 tx=0).stop 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 shutdown_connections 2024-06-06T07:26:40.465 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2830107990 0x7f283019c9f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f28200a2e20 0x7f28200a52e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f2830102340 0x7f2830198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 --2- 172.21.15.73:0/3961539568 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f2830101980 0x7f2830198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 >> 172.21.15.73:0/3961539568 conn(0x7f28300fd450 msgr2=0x7f28300fdc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 shutdown_connections 2024-06-06T07:26:40.466 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:40.460+0000 7f2837231640 1 -- 172.21.15.73:0/3961539568 wait complete. 2024-06-06T07:26:40.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114655 8.68 MB/sec execute 585 sec latency 147.619 ms 2024-06-06T07:26:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:41 smithi170 ceph-mon[36108]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-06-06T07:26:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:41 smithi073 ceph-mon[32303]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-06-06T07:26:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:41 smithi136 ceph-mon[35901]: pgmap v807: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.5 MiB/s wr, 279 op/s 2024-06-06T07:26:41.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114822 8.68 MB/sec execute 586 sec latency 95.691 ms 2024-06-06T07:26:42.841 INFO:tasks.workunit.client.0.smithi073.stdout: 10 114980 8.68 MB/sec execute 587 sec latency 93.445 ms 2024-06-06T07:26:43.776 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:43 smithi073 ceph-mon[32303]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-06T07:26:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:43 smithi136 ceph-mon[35901]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-06T07:26:43.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115147 8.68 MB/sec execute 588 sec latency 62.441 ms 2024-06-06T07:26:43.880 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:43 smithi170 ceph-mon[36108]: pgmap v808: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 5.4 MiB/s wr, 274 op/s 2024-06-06T07:26:44.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115300 8.67 MB/sec execute 589 sec latency 93.990 ms 2024-06-06T07:26:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:45 smithi073 ceph-mon[32303]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2024-06-06T07:26:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:45 smithi136 ceph-mon[35901]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2024-06-06T07:26:45.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115451 8.67 MB/sec execute 590 sec latency 93.384 ms 2024-06-06T07:26:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:45 smithi170 ceph-mon[36108]: pgmap v809: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 8.2 MiB/s wr, 418 op/s 2024-06-06T07:26:46.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115622 8.68 MB/sec execute 591 sec latency 95.868 ms 2024-06-06T07:26:47.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:47 smithi073 ceph-mon[32303]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 284 op/s 2024-06-06T07:26:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:47 smithi136 ceph-mon[35901]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 284 op/s 2024-06-06T07:26:47.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115789 8.68 MB/sec execute 592 sec latency 91.538 ms 2024-06-06T07:26:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:47 smithi170 ceph-mon[36108]: pgmap v810: 129 pgs: 129 active+clean; 1.6 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.8 MiB/s wr, 284 op/s 2024-06-06T07:26:48.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 115924 8.67 MB/sec execute 593 sec latency 80.800 ms 2024-06-06T07:26:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:49 smithi073 ceph-mon[32303]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 418 op/s 2024-06-06T07:26:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:49 smithi136 ceph-mon[35901]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 418 op/s 2024-06-06T07:26:49.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116055 8.67 MB/sec execute 594 sec latency 64.147 ms 2024-06-06T07:26:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:49 smithi170 ceph-mon[36108]: pgmap v811: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 8.5 MiB/s wr, 418 op/s 2024-06-06T07:26:50.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116237 8.68 MB/sec execute 595 sec latency 85.494 ms 2024-06-06T07:26:51.738 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:51 smithi170 ceph-mon[36108]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 282 op/s 2024-06-06T07:26:51.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:51 smithi073 ceph-mon[32303]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 282 op/s 2024-06-06T07:26:51.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:51 smithi136 ceph-mon[35901]: pgmap v812: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.7 MiB/s wr, 282 op/s 2024-06-06T07:26:51.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116415 8.68 MB/sec execute 596 sec latency 104.968 ms 2024-06-06T07:26:52.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116585 8.68 MB/sec execute 597 sec latency 230.155 ms 2024-06-06T07:26:53.596 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 -- 172.21.15.73:0/3812774801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324100b10 msgr2=0x7ff324100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 --2- 172.21.15.73:0/3812774801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324100b10 0x7ff324100f70 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7ff318009900 tx=0x7ff31802a410 comp rx=0 tx=0).stop 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 -- 172.21.15.73:0/3812774801 shutdown_connections 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 --2- 172.21.15.73:0/3812774801 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff3241014b0 0x7ff32410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 --2- 172.21.15.73:0/3812774801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324100b10 0x7ff324100f70 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 --2- 172.21.15.73:0/3812774801 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff324106940 0x7ff324106d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 -- 172.21.15.73:0/3812774801 >> 172.21.15.73:0/3812774801 conn(0x7ff324077780 msgr2=0x7ff324077b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 -- 172.21.15.73:0/3812774801 shutdown_connections 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.595+0000 7ff329cc1640 1 -- 172.21.15.73:0/3812774801 wait complete. 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 Processor -- start 2024-06-06T07:26:53.597 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 -- start start 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff324100b10 0x7ff3241980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324106940 0x7ff32419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff324111d80 con 0x7ff324106940 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff32406aa70 con 0x7ff324100b10 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff329cc1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff324111f00 con 0x7ff3241014b0 2024-06-06T07:26:53.599 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55588/0 (socket says 172.21.15.73:55588) 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff322ffd640 1 -- 172.21.15.73:0/1503008204 learned_addr learned my addr 172.21.15.73:0/1503008204 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.596+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff324100b10 0x7ff3241980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff323fff640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324106940 0x7ff32419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 -- 172.21.15.73:0/1503008204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff324100b10 msgr2=0x7ff3241980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff324100b10 0x7ff3241980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 -- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324106940 msgr2=0x7ff32419c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324106940 0x7ff32419c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 -- 172.21.15.73:0/1503008204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32419d070 con 0x7ff3241014b0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff318005ab0 tx=0x7ff318003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff320ff9640 1 -- 172.21.15.73:0/1503008204 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31803f070 con 0x7ff3241014b0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32419d300 con 0x7ff3241014b0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3241a4bf0 con 0x7ff3241014b0 2024-06-06T07:26:53.600 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff320ff9640 1 -- 172.21.15.73:0/1503008204 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff318031b60 con 0x7ff3241014b0 2024-06-06T07:26:53.601 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.597+0000 7ff320ff9640 1 -- 172.21.15.73:0/1503008204 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318035030 con 0x7ff3241014b0 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.600+0000 7ff320ff9640 1 -- 172.21.15.73:0/1503008204 <== mon.1 v2:172.21.15.170:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff318042070 con 0x7ff3241014b0 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff320ff9640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 0x7ff30c0a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 0x7ff30c0a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff320ff9640 1 -- 172.21.15.73:0/1503008204 <== mon.1 v2:172.21.15.170:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7ff3180efe50 con 0x7ff3241014b0 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 Processor -- start 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 -- start start 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78f170 0x56121b78f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.603 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b79c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56121b6f34f0 con 0x56121b79a350 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56121b691870 con 0x56121b78f170 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56121b6ef4b0 con 0x56121b78fa90 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 0x7ff30c0a52e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff314009b50 tx=0x7ff314005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff323fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b79c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78f170 0x56121b78f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55602/0 (socket says 172.21.15.73:55602) 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.601+0000 7ff322ffd640 1 -- 172.21.15.73:0/3137717033 learned_addr learned my addr 172.21.15.73:0/3137717033 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 -- 172.21.15.73:0/3137717033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78f170 msgr2=0x56121b78f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78f170 0x56121b78f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.604 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 -- 172.21.15.73:0/3137717033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 msgr2=0x56121b79c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b79c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 -- 172.21.15.73:0/3137717033 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121b6ad9b0 con 0x56121b78fa90 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff323fff640 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b79c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff322ffd640 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff3180c46a0 tx=0x7ff31802a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30df4fbbe65af5c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff309ffb640 1 -- 172.21.15.73:0/3137717033 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff318049070 con 0x56121b78fa90 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff309ffb640 1 -- 172.21.15.73:0/3137717033 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff3180c4bd0 con 0x56121b78fa90 2024-06-06T07:26:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.602+0000 7ff309ffb640 1 -- 172.21.15.73:0/3137717033 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3180bc270 con 0x56121b78fa90 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 -- 172.21.15.73:0/3137717033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 msgr2=0x56121b799e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff3180c46a0 tx=0x7ff31802a410 comp rx=0 tx=0).stop 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 -- 172.21.15.73:0/3137717033 shutdown_connections 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b79c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78fa90 0x56121b799e10 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/3137717033 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78f170 0x56121b78f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 -- 172.21.15.73:0/3137717033 >> 172.21.15.73:0/3137717033 conn(0x56121b78b680 msgr2=0x56121b78ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 -- 172.21.15.73:0/3137717033 shutdown_connections 2024-06-06T07:26:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.603+0000 7ff32c9cd740 1 -- 172.21.15.73:0/3137717033 wait complete. 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 -- start start 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 Processor -- start 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b797300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b797840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b797300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56121b79cd40 con 0x56121b79a350 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b797300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55618/0 (socket says 172.21.15.73:55618) 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37084/0 (socket says 172.21.15.73:37084) 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff3237fe640 1 -- 172.21.15.73:0/1993385394 learned_addr learned my addr 172.21.15.73:0/1993385394 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x56121b67cae0 con 0x56121b78fa90 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x56121b79cec0 con 0x56121b78f170 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b797840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff32c9cd740 10 client.0 _ll_register_callbacks cb 0x56121b6b59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 -- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 msgr2=0x56121b797300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b797300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 -- 172.21.15.73:0/1993385394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 msgr2=0x56121b797840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b797840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.604+0000 7ff323fff640 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121b407210 con 0x56121b79a350 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3180c4bd0 con 0x56121b79a350 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff323fff640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005fd0 tx=0x7ff31c00e780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd8f1136412d5102 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff302ffd640 10 client.17901 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff302ffd640 1 -- 172.21.15.73:0/1993385394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c027060 con 0x56121b79a350 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff302ffd640 1 -- 172.21.15.73:0/1993385394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7ff31c00ed30 con 0x56121b79a350 2024-06-06T07:26:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff302ffd640 1 -- 172.21.15.73:0/1993385394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7ff31c004b20 con 0x56121b79a350 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff32c9cd740 10 client.17901 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=84}) -- 0x56121b4d3620 con 0x56121b79a350 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.605+0000 7ff302ffd640 1 -- 172.21.15.73:0/1993385394 <== mon.0 v2:172.21.15.73:3300/0 4 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7ff31c02a050 con 0x56121b79a350 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 10 client.17901 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 10 client.17901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 20 client.17901 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b7a9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 4 client.17901 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56121b706f10 con 0x56121b7a8fe0 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b7a9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b7a9420 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c759e46c9fe224f server_cookie=f34b336acf1069a0 in_seq=0 out_seq=0 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.606+0000 7ff302ffd640 10 client.17901 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:53.612 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.607+0000 7ff302ffd640 1 -- 172.21.15.73:0/1993385394 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56121b706f10 con 0x56121b7a8fe0 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.607+0000 7ff302ffd640 10 client.17901 handle_command_reply: tid=0 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.607+0000 7ff32c9cd740 1 client.17901 shutdown 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 msgr2=0x56121b797d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005fd0 tx=0x7ff31c00e780 comp rx=0 tx=0).stop 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 shutdown_connections 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b7a9420 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff302ffd640 0 client.17901 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797d80 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b797840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1993385394 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b797300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 >> 172.21.15.73:0/1993385394 conn(0x56121b78b680 msgr2=0x56121b7913b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 shutdown_connections 2024-06-06T07:26:53.613 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1993385394 wait complete. 2024-06-06T07:26:53.614 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.608+0000 7ff32c9cd740 20 client.17901 trim_cache size 0 max 16384 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 Processor -- start 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 -- start start 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b791b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241acca0 con 0x56121b79a350 2024-06-06T07:26:53.635 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241ae7e0 con 0x56121b78fa90 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.633+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241ad370 con 0x56121b78f170 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff323fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.170:3300/0 says I am v2:172.21.15.73:55628/0 (socket says 172.21.15.73:55628) 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 -- 172.21.15.73:0/770111959 learned_addr learned my addr 172.21.15.73:0/770111959 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff322ffd640 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b791b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 -- 172.21.15.73:0/770111959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 msgr2=0x56121b791b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b791b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 -- 172.21.15.73:0/770111959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 msgr2=0x56121b797ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 -- 172.21.15.73:0/770111959 --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56121b78f8c0 con 0x56121b78f170 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff323fff640 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3237fe640 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff314009fd0 tx=0x7ff314014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=611082e381c1ec12 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.636 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3097fa640 1 -- 172.21.15.73:0/770111959 <== mon.1 v2:172.21.15.170:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31402a670 con 0x56121b78f170 2024-06-06T07:26:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3097fa640 1 -- 172.21.15.73:0/770111959 <== mon.1 v2:172.21.15.170:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff31402acc0 con 0x56121b78f170 2024-06-06T07:26:53.637 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.634+0000 7ff3097fa640 1 -- 172.21.15.73:0/770111959 <== mon.1 v2:172.21.15.170:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31401e420 con 0x56121b78f170 2024-06-06T07:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 -- 172.21.15.73:0/770111959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 msgr2=0x56121b7869c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff314009fd0 tx=0x7ff314014000 comp rx=0 tx=0).stop 2024-06-06T07:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 -- 172.21.15.73:0/770111959 shutdown_connections 2024-06-06T07:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b79a350 0x56121b797ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.640 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b78fa90 0x56121b791b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/770111959 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7869c0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.635+0000 7ff32c9cd740 1 -- 172.21.15.73:0/770111959 >> 172.21.15.73:0/770111959 conn(0x56121b78b680 msgr2=0x56121b789f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.636+0000 7ff32c9cd740 1 -- 172.21.15.73:0/770111959 shutdown_connections 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.636+0000 7ff32c9cd740 1 -- 172.21.15.73:0/770111959 wait complete. 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.636+0000 7ff32c9cd740 1 -- start start 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.636+0000 7ff32c9cd740 1 Processor -- start 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7b0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b79a350 0x56121b7ad4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241ae110 con 0x56121b78fa90 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241ada40 con 0x56121b79a350 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff3241aaca0 con 0x56121b78f170 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.641 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37102/0 (socket says 172.21.15.73:37102) 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 -- 172.21.15.73:0/1277290371 learned_addr learned my addr 172.21.15.73:0/1277290371 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff323fff640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b79a350 0x56121b7ad4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff32c9cd740 10 client.0 _ll_register_callbacks cb 0x7ff31c009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 -- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 msgr2=0x56121b7b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 -- 172.21.15.73:0/1277290371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b79a350 msgr2=0x56121b7ad4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b79a350 0x56121b7ad4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.637+0000 7ff322ffd640 1 -- 172.21.15.73:0/1277290371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56121b7adb70 con 0x56121b78fa90 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56121b700c20 con 0x56121b78fa90 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff322ffd640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff318005ec0 tx=0x7ff3180b8890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47c79eb4e5ebcf8c server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff302ffd640 10 client.17907 ms_handle_connect on v2:172.21.15.73:3300/0 2024-06-06T07:26:53.642 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff302ffd640 1 -- 172.21.15.73:0/1277290371 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3180d4020 con 0x56121b78fa90 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff302ffd640 1 -- 172.21.15.73:0/1277290371 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7ff31803b460 con 0x56121b78fa90 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff302ffd640 1 -- 172.21.15.73:0/1277290371 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=84) ==== 24+0+0 (secure 0 0 0) 0x7ff318049070 con 0x56121b78fa90 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 10 client.17907 fetch_fsmap learned FSMap version 84 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=84}) -- 0x56121b8143c0 con 0x56121b78fa90 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff302ffd640 1 -- 172.21.15.73:0/1277290371 <== mon.0 v2:172.21.15.73:3300/0 4 ==== fsmap(e 84) ==== 8362+0+0 (secure 0 0 0) 0x7ff3180bc270 con 0x56121b78fa90 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 10 client.17907 fetch_fsmap finished waiting for FSMap version 84 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 10 client.17907 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.i 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 20 client.17907 populate_metadata read hostname 'smithi073' 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b816710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 4 client.17907 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.638+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 --> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56121b6faf70 con 0x56121b7a8fe0 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.639+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b816710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:26:53.643 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.639+0000 7ff3237fe640 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b816710 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ebd44e8bd0596 server_cookie=6da064c443ef2559 in_seq=0 out_seq=0 2024-06-06T07:26:53.644 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.639+0000 7ff302ffd640 10 client.17907 ms_handle_connect on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff302ffd640 1 -- 172.21.15.73:0/1277290371 <== mds.0 v2:172.21.15.170:6838/3230829204 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56121b6faf70 con 0x56121b7a8fe0 2024-06-06T07:26:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff302ffd640 10 client.17907 handle_command_reply: tid=0 2024-06-06T07:26:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 client.17907 shutdown 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 msgr2=0x56121b7acf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7ff318005ec0 tx=0x7ff3180b8890 comp rx=0 tx=0).stop 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 shutdown_connections 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:6838/3230829204,v1:172.21.15.170:6839/3230829204] conn(0x56121b7a8fe0 0x56121b816710 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x56121b79a350 0x56121b7ad4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff302ffd640 0 client.17907 ms_handle_reset on v2:172.21.15.170:6838/3230829204 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56121b78fa90 0x56121b7acf90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 --2- 172.21.15.73:0/1277290371 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x56121b78f170 0x56121b7b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 >> 172.21.15.73:0/1277290371 conn(0x56121b78b680 msgr2=0x56121b7a5910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 shutdown_connections 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.737+0000 7ff32c9cd740 1 -- 172.21.15.73:0/1277290371 wait complete. 2024-06-06T07:26:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff32c9cd740 20 client.17907 trim_cache size 0 max 16384 2024-06-06T07:26:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:{ 2024-06-06T07:26:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout: "return_code": 0 2024-06-06T07:26:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stdout:} 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 msgr2=0x7ff30c0a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 0x7ff30c0a52e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff314009b50 tx=0x7ff314005f90 comp rx=0 tx=0).stop 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 msgr2=0x7ff324198610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff318005ab0 tx=0x7ff318003c10 comp rx=0 tx=0).stop 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 shutdown_connections 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff324106940 0x7ff32419c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.743 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff30c0a2e20 0x7ff30c0a52e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff3241014b0 0x7ff324198610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 --2- 172.21.15.73:0/1503008204 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff324100b10 0x7ff3241980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:26:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 >> 172.21.15.73:0/1503008204 conn(0x7ff324077780 msgr2=0x7ff3240ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:26:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 shutdown_connections 2024-06-06T07:26:53.744 INFO:teuthology.task.background_exec.ubuntu@smithi073.front.sepia.ceph.com.smithi073.stderr:2024-06-06T07:26:53.738+0000 7ff329cc1640 1 -- 172.21.15.73:0/1503008204 wait complete. 2024-06-06T07:26:53.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:53 smithi073 ceph-mon[32303]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 280 op/s 2024-06-06T07:26:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:53 smithi136 ceph-mon[35901]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 280 op/s 2024-06-06T07:26:53.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116703 8.67 MB/sec execute 598 sec latency 266.017 ms 2024-06-06T07:26:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:53 smithi170 ceph-mon[36108]: pgmap v813: 129 pgs: 129 active+clean; 1.7 GiB data, 9.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 5.6 MiB/s wr, 280 op/s 2024-06-06T07:26:54.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 116853 8.68 MB/sec execute 599 sec latency 68.130 ms 2024-06-06T07:26:55.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:55 smithi073 ceph-mon[32303]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.0 MiB/s wr, 414 op/s 2024-06-06T07:26:55.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:55 smithi136 ceph-mon[35901]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.0 MiB/s wr, 414 op/s 2024-06-06T07:26:55.842 INFO:tasks.workunit.client.0.smithi073.stdout: 10 cleanup 600 sec 2024-06-06T07:26:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:55 smithi170 ceph-mon[36108]: pgmap v814: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 8.0 MiB/s wr, 414 op/s 2024-06-06T07:26:56.842 INFO:tasks.workunit.client.0.smithi073.stdout: 7 cleanup 601 sec 2024-06-06T07:26:57.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:57 smithi073 ceph-mon[32303]: pgmap v815: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2024-06-06T07:26:57.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:57 smithi136 ceph-mon[35901]: pgmap v815: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2024-06-06T07:26:57.842 INFO:tasks.workunit.client.0.smithi073.stdout: 6 cleanup 602 sec 2024-06-06T07:26:57.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:57 smithi170 ceph-mon[36108]: pgmap v815: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 5.2 MiB/s wr, 270 op/s 2024-06-06T07:26:58.842 INFO:tasks.workunit.client.0.smithi073.stdout: 1 cleanup 603 sec 2024-06-06T07:26:58.968 INFO:tasks.workunit.client.0.smithi073.stdout: 0 cleanup 603 sec 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Operation Count AvgLat MaxLat 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: ---------------------------------------- 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: NTCreateX 167162 10.392 807.406 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Close 122794 0.187 94.157 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Rename 7085 30.240 14371.472 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Unlink 33734 9.288 790.186 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Qpathinfo 151540 7.279 626.626 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Qfileinfo 26422 0.340 33.050 2024-06-06T07:26:58.969 INFO:tasks.workunit.client.0.smithi073.stdout: Qfsinfo 27760 0.967 53.986 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: Sfileinfo 13653 3.070 88.682 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: Find 58525 40.207 839.189 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: WriteX 82594 0.737 156.928 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: ReadX 261977 0.022 67.577 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: LockX 544 1.914 29.377 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: UnlockX 544 1.707 9.808 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: Flush 11715 9.113 548.197 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout: 2024-06-06T07:26:58.970 INFO:tasks.workunit.client.0.smithi073.stdout:Throughput 8.67551 MB/sec 10 clients 10 procs max_latency=14371.477 ms 2024-06-06T07:26:58.971 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-06T07:26:58.971 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-06T07:26:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:59 smithi073 ceph-mon[32303]: pgmap v816: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 414 op/s 2024-06-06T07:26:59.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:26:59 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:26:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:59 smithi136 ceph-mon[35901]: pgmap v816: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 414 op/s 2024-06-06T07:26:59.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:26:59 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:26:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:59 smithi170 ceph-mon[36108]: pgmap v816: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 7.7 MiB/s wr, 414 op/s 2024-06-06T07:26:59.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:26:59 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:01.741 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:01 smithi073 ceph-mon[32303]: pgmap v817: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 296 op/s 2024-06-06T07:27:01.741 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:01 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:01.798 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-06T07:27:01.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-06T07:27:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:01 smithi136 ceph-mon[35901]: pgmap v817: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 296 op/s 2024-06-06T07:27:01.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:01 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:01 smithi170 ceph-mon[36108]: pgmap v817: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.3 MiB/s wr, 296 op/s 2024-06-06T07:27:01.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:01 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:02.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:02 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:02.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:02 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:02.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:02 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:03.066 DEBUG:teuthology.parallel:result is None 2024-06-06T07:27:03.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-06T07:27:03.103 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-06T07:27:03.103 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-06T07:27:03.114 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:27:03.152 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-06-06T07:27:03.162 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-06-06T07:27:03.162 INFO:teuthology.task.exec:Executing custom commands... 2024-06-06T07:27:03.162 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:27:03.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-06-06T07:27:03.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-06-06T07:27:03.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-06-06T07:27:03.414 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 -- 172.21.15.73:0/3297005525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c781008d0 msgr2=0x7f7c78100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/3297005525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c781008d0 0x7f7c78100d50 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7c60009930 tx=0x7f7c6002a420 comp rx=0 tx=0).stop 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 -- 172.21.15.73:0/3297005525 shutdown_connections 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/3297005525 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c78101290 0x7f7c7810d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/3297005525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c781008d0 0x7f7c78100d50 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/3297005525 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78106890 0x7f7c78106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 -- 172.21.15.73:0/3297005525 >> 172.21.15.73:0/3297005525 conn(0x7f7c78077a20 msgr2=0x7f7c78077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 -- 172.21.15.73:0/3297005525 shutdown_connections 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.411+0000 7f7c7d04d640 1 -- 172.21.15.73:0/3297005525 wait complete. 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 Processor -- start 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 -- start start 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c781008d0 0x7f7c78198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78101290 0x7f7c78198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c78110b50 con 0x7f7c78106890 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7c781109d0 con 0x7f7c78101290 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7d04d640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7c78110cd0 con 0x7f7c781008d0 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7759e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34650/0 (socket says 172.21.15.73:34650) 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c7759e640 1 -- 172.21.15.73:0/4191001391 learned_addr learned my addr 172.21.15.73:0/4191001391 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c6ffff640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78101290 0x7f7c78198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.412+0000 7f7c76d9d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c781008d0 0x7f7c78198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 -- 172.21.15.73:0/4191001391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c781008d0 msgr2=0x7f7c78198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c781008d0 0x7f7c78198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 -- 172.21.15.73:0/4191001391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78101290 msgr2=0x7f7c78198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78101290 0x7f7c78198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.416 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 -- 172.21.15.73:0/4191001391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7819d130 con 0x7f7c78106890 2024-06-06T07:27:03.417 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c7759e640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c7000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:03.418 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.413+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70017070 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.414+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7c70003e20 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.414+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c70010460 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.414+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7819d420 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.414+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c781a4cb0 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.415+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7804f760 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.416+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7c70003970 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.416+0000 7f7c74d99640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 0x7f7c640a52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.417+0000 7f7c76d9d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 0x7f7c640a52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.417+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f7c700cf4f0 con 0x7f7c78106890 2024-06-06T07:27:03.419 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.417+0000 7f7c76d9d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 0x7f7c640a52e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009b50 tx=0x7f7c68005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:03.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.421+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7c7009b750 con 0x7f7c78106890 2024-06-06T07:27:03.619 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.617+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f7c78109e60 con 0x7f7c640a2e20 2024-06-06T07:27:03.632 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.630+0000 7f7c74d99640 1 -- 172.21.15.73:0/4191001391 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7f7c78109e60 con 0x7f7c640a2e20 2024-06-06T07:27:03.632 INFO:teuthology.orchestra.run.smithi073.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-06-06T00:00:00", "created": "2024-06-06T07:01:33", "first": "2024-06-06T07:02:00", "last": "2024-06-06T07:27:00", "last_pruned": "2024-06-06T07:27:00", "created_count": 26, "pruned_count": 20, "active": true} 2024-06-06T07:27:03.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 msgr2=0x7f7c640a52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 0x7f7c640a52e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009b50 tx=0x7f7c68005f90 comp rx=0 tx=0).stop 2024-06-06T07:27:03.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 msgr2=0x7f7c7819ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009fd0 tx=0x7f7c7000b260 comp rx=0 tx=0).stop 2024-06-06T07:27:03.634 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 shutdown_connections 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c78106890 0x7f7c7819ca00 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7c78101290 0x7f7c78198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7c640a2e20 0x7f7c640a52e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 --2- 172.21.15.73:0/4191001391 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7c781008d0 0x7f7c78198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 >> 172.21.15.73:0/4191001391 conn(0x7f7c78077a20 msgr2=0x7f7c780693f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 shutdown_connections 2024-06-06T07:27:03.635 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.632+0000 7f7c7d04d640 1 -- 172.21.15.73:0/4191001391 wait complete. 2024-06-06T07:27:03.644 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:03 smithi073 ceph-mon[32303]: pgmap v818: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 294 op/s 2024-06-06T07:27:03.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 -- 172.21.15.73:0/2985294043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c1008d0 msgr2=0x7fc08c100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 --2- 172.21.15.73:0/2985294043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c1008d0 0x7fc08c100d50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc074009860 tx=0x7fc07402a340 comp rx=0 tx=0).stop 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 -- 172.21.15.73:0/2985294043 shutdown_connections 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 --2- 172.21.15.73:0/2985294043 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c101290 0x7fc08c10d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 --2- 172.21.15.73:0/2985294043 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c1008d0 0x7fc08c100d50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 --2- 172.21.15.73:0/2985294043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c106890 0x7fc08c106c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 -- 172.21.15.73:0/2985294043 >> 172.21.15.73:0/2985294043 conn(0x7fc08c077a20 msgr2=0x7fc08c077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 -- 172.21.15.73:0/2985294043 shutdown_connections 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.746+0000 7fc090ad4640 1 -- 172.21.15.73:0/2985294043 wait complete. 2024-06-06T07:27:03.751 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 Processor -- start 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 -- start start 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c1008d0 0x7fc08c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c106890 0x7fc08c19ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c110b50 con 0x7fc08c101290 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c1109d0 con 0x7fc08c106890 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc090ad4640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fc08c110cd0 con 0x7fc08c1008d0 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.747+0000 7fc083fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34676/0 (socket says 172.21.15.73:34676) 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 -- 172.21.15.73:0/4024059109 learned_addr learned my addr 172.21.15.73:0/4024059109 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc08ad9d640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c106890 0x7fc08c19ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.752 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc08a59c640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c1008d0 0x7fc08c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 -- 172.21.15.73:0/4024059109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c1008d0 msgr2=0x7fc08c198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c1008d0 0x7fc08c198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 -- 172.21.15.73:0/4024059109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c106890 msgr2=0x7fc08c19ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c106890 0x7fc08c19ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 -- 172.21.15.73:0/4024059109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc08c19d170 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc083fff640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc074009980 tx=0x7fc074030df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074037050 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc074043070 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc08c19d400 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074003d10 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.748+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc08c1a4cf0 con 0x7fc08c101290 2024-06-06T07:27:03.753 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.749+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc08c04f760 con 0x7fc08c101290 2024-06-06T07:27:03.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.750+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fc074004040 con 0x7fc08c101290 2024-06-06T07:27:03.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.751+0000 7fc0837fe640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 0x7fc0780a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:03.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.751+0000 7fc08a59c640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 0x7fc0780a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:03.754 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.751+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7fc0740f0230 con 0x7fc08c101290 2024-06-06T07:27:03.755 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.752+0000 7fc08a59c640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 0x7fc0780a5260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005fd0 tx=0x7fc07c009260 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:03.756 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.755+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc0740bcb50 con 0x7fc08c101290 2024-06-06T07:27:03.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:03 smithi136 ceph-mon[35901]: pgmap v818: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 294 op/s 2024-06-06T07:27:03.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:03 smithi170 ceph-mon[36108]: pgmap v818: 129 pgs: 129 active+clean; 1.7 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 5.2 MiB/s wr, 294 op/s 2024-06-06T07:27:03.953 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.952+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 --> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fc08c061de0 con 0x7fc0780a2da0 2024-06-06T07:27:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.965+0000 7fc0837fe640 1 -- 172.21.15.73:0/4024059109 <== mgr.15282 v2:172.21.15.73:6800/3841884199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fc08c061de0 con 0x7fc0780a2da0 2024-06-06T07:27:03.967 INFO:teuthology.orchestra.run.smithi073.stdout:/ 1m 6m3h 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 msgr2=0x7fc0780a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 0x7fc0780a5260 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc07c005fd0 tx=0x7fc07c009260 comp rx=0 tx=0).stop 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 msgr2=0x7fc08c1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fc074009980 tx=0x7fc074030df0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 shutdown_connections 2024-06-06T07:27:03.969 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fc08c106890 0x7fc08c19ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc08c101290 0x7fc08c1986c0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fc0780a2da0 0x7fc0780a5260 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 --2- 172.21.15.73:0/4024059109 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fc08c1008d0 0x7fc08c198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 >> 172.21.15.73:0/4024059109 conn(0x7fc08c077a20 msgr2=0x7fc08c069390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 shutdown_connections 2024-06-06T07:27:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:03.967+0000 7fc090ad4640 1 -- 172.21.15.73:0/4024059109 wait complete. 2024-06-06T07:27:03.980 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-06T07:27:03.990 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-06T07:27:03.990 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T07:27:03.991 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-06T07:27:04.001 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 -- 172.21.15.73:0/2135687061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 msgr2=0x7f7d7c100cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/2135687061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c100cf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009900 tx=0x7f7d6402a410 comp rx=0 tx=0).stop 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 -- 172.21.15.73:0/2135687061 shutdown_connections 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/2135687061 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d7c101230 0x7f7d7c10d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/2135687061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c100cf0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.101 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/2135687061 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d7c106880 0x7f7d7c106c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 -- 172.21.15.73:0/2135687061 >> 172.21.15.73:0/2135687061 conn(0x7f7d7c076c20 msgr2=0x7f7d7c077030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 -- 172.21.15.73:0/2135687061 shutdown_connections 2024-06-06T07:27:04.102 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.099+0000 7f7d80cf5640 1 -- 172.21.15.73:0/2135687061 wait complete. 2024-06-06T07:27:04.103 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 Processor -- start 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 -- start start 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d7c101230 0x7f7d7c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d7c106880 0x7f7d7c19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c111c40 con 0x7f7d7c100870 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c111ac0 con 0x7f7d7c101230 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d80cf5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f7d7c111dc0 con 0x7f7d7c106880 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34696/0 (socket says 172.21.15.73:34696) 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 -- 172.21.15.73:0/598321932 learned_addr learned my addr 172.21.15.73:0/598321932 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 -- 172.21.15.73:0/598321932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d7c106880 msgr2=0x7f7d7c19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d7c106880 0x7f7d7c19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.104 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 -- 172.21.15.73:0/598321932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d7c101230 msgr2=0x7f7d7c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d7c101230 0x7f7d7c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.100+0000 7f7d7a59c640 1 -- 172.21.15.73:0/598321932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d7c19d1b0 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d7a59c640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400db80 tx=0x7f7d7400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d74017070 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d74003b70 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d7c19d4a0 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d740104c0 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.101+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d7c1a4d30 con 0x7f7d7c100870 2024-06-06T07:27:04.105 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.102+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d7c068c10 con 0x7f7d7c100870 2024-06-06T07:27:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.103+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f7d74004020 con 0x7f7d7c100870 2024-06-06T07:27:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.104+0000 7f7d737fe640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 0x7f7d5c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.104+0000 7f7d79d9b640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 0x7f7d5c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.104+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f7d740d0180 con 0x7f7d7c100870 2024-06-06T07:27:04.106 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.104+0000 7f7d79d9b640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 0x7f7d5c0a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d640096c0 tx=0x7f7d64031400 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:04.109 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.107+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7d7400e350 con 0x7f7d7c100870 2024-06-06T07:27:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.286+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d7c02d2e0 con 0x7f7d7c100870 2024-06-06T07:27:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.287+0000 7f7d737fe640 1 -- 172.21.15.73:0/598321932 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) ==== 74+0+25379 (secure 0 0 0) 0x7f7d7409c930 con 0x7f7d7c100870 2024-06-06T07:27:04.289 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:27:04.289 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":146,"fsid":"b6cc6aaa-23d0-11ef-bc9b-c7b262605968","created":"2024-06-06T06:48:47.254289+0000","modified":"2024-06-06T07:26:12.727220+0000","last_up_change":"2024-06-06T06:56:04.741454+0000","last_in_change":"2024-06-06T06:55:44.556041+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-06T06:52:17.534158+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-06T06:57:03.217358+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-06T06:57:06.475737+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-06-06T06:57:16.584799+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":45,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f1af7d41-9f4c-4b4a-8a05-c38530312412","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6803","nonce":3665933649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6805","nonce":3665933649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6809","nonce":3665933649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3665933649},{"type":"v1","addr":"172.21.15.73:6807","nonce":3665933649}]},"public_addr":"172.21.15.73:6803/3665933649","cluster_addr":"172.21.15.73:6805/3665933649","heartbeat_back_addr":"172.21.15.73:6809/3665933649","heartbeat_front_addr":"172.21.15.73:6807/3665933649","state":["exists","up"]},{"osd":1,"uuid":"ec8a9433-7b32-4f79-8a39-4137d445acfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6801","nonce":2860362689}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6802","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6803","nonce":2860362689}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6807","nonce":2860362689}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":2860362689},{"type":"v1","addr":"172.21.15.136:6805","nonce":2860362689}]},"public_addr":"172.21.15.136:6801/2860362689","cluster_addr":"172.21.15.136:6803/2860362689","heartbeat_back_addr":"172.21.15.136:6807/2860362689","heartbeat_front_addr":"172.21.15.136:6805/2860362689","state":["exists","up"]},{"osd":2,"uuid":"3443c257-8d23-4b70-ab00-5d3add9a8b3d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6801","nonce":2309158853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6803","nonce":2309158853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6807","nonce":2309158853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2309158853},{"type":"v1","addr":"172.21.15.170:6805","nonce":2309158853}]},"public_addr":"172.21.15.170:6801/2309158853","cluster_addr":"172.21.15.170:6803/2309158853","heartbeat_back_addr":"172.21.15.170:6807/2309158853","heartbeat_front_addr":"172.21.15.170:6805/2309158853","state":["exists","up"]},{"osd":3,"uuid":"e503419a-b7b8-4a0e-ab38-9459dbf2fdf9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6811","nonce":3983384250}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6813","nonce":3983384250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6817","nonce":3983384250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3983384250},{"type":"v1","addr":"172.21.15.73:6815","nonce":3983384250}]},"public_addr":"172.21.15.73:6811/3983384250","cluster_addr":"172.21.15.73:6813/3983384250","heartbeat_back_addr":"172.21.15.73:6817/3983384250","heartbeat_front_addr":"172.21.15.73:6815/3983384250","state":["exists","up"]},{"osd":4,"uuid":"f622b141-b741-4f89-9c8c-39615ff7919f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6808","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6809","nonce":846852962}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6810","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6811","nonce":846852962}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6815","nonce":846852962}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":846852962},{"type":"v1","addr":"172.21.15.136:6813","nonce":846852962}]},"public_addr":"172.21.15.136:6809/846852962","cluster_addr":"172.21.15.136:6811/846852962","heartbeat_back_addr":"172.21.15.136:6815/846852962","heartbeat_front_addr":"172.21.15.136:6813/846852962","state":["exists","up"]},{"osd":5,"uuid":"27b58b5d-313a-46d9-a7f5-e6c72db937c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6808","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6809","nonce":686557154}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6810","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6811","nonce":686557154}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6814","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6815","nonce":686557154}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6812","nonce":686557154},{"type":"v1","addr":"172.21.15.170:6813","nonce":686557154}]},"public_addr":"172.21.15.170:6809/686557154","cluster_addr":"172.21.15.170:6811/686557154","heartbeat_back_addr":"172.21.15.170:6815/686557154","heartbeat_front_addr":"172.21.15.170:6813/686557154","state":["exists","up"]},{"osd":6,"uuid":"de018a90-38be-4bea-b847-951585c33b94","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6819","nonce":135647786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6821","nonce":135647786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6825","nonce":135647786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":135647786},{"type":"v1","addr":"172.21.15.73:6823","nonce":135647786}]},"public_addr":"172.21.15.73:6819/135647786","cluster_addr":"172.21.15.73:6821/135647786","heartbeat_back_addr":"172.21.15.73:6825/135647786","heartbeat_front_addr":"172.21.15.73:6823/135647786","state":["exists","up"]},{"osd":7,"uuid":"51b08617-a3c2-4de4-b0f7-3f505b13e7e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6816","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6817","nonce":4094155747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6818","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6819","nonce":4094155747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6823","nonce":4094155747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":4094155747},{"type":"v1","addr":"172.21.15.136:6821","nonce":4094155747}]},"public_addr":"172.21.15.136:6817/4094155747","cluster_addr":"172.21.15.136:6819/4094155747","heartbeat_back_addr":"172.21.15.136:6823/4094155747","heartbeat_front_addr":"172.21.15.136:6821/4094155747","state":["exists","up"]},{"osd":8,"uuid":"edab0c0c-df86-49f8-81b9-90f481b58410","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6816","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6817","nonce":65881112}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6818","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6819","nonce":65881112}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6822","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6823","nonce":65881112}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6820","nonce":65881112},{"type":"v1","addr":"172.21.15.170:6821","nonce":65881112}]},"public_addr":"172.21.15.170:6817/65881112","cluster_addr":"172.21.15.170:6819/65881112","heartbeat_back_addr":"172.21.15.170:6823/65881112","heartbeat_front_addr":"172.21.15.170:6821/65881112","state":["exists","up"]},{"osd":9,"uuid":"cd7bb4f4-bc75-4e2d-9a15-b414a84302ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6827","nonce":2353282105}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6829","nonce":2353282105}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6833","nonce":2353282105}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":2353282105},{"type":"v1","addr":"172.21.15.73:6831","nonce":2353282105}]},"public_addr":"172.21.15.73:6827/2353282105","cluster_addr":"172.21.15.73:6829/2353282105","heartbeat_back_addr":"172.21.15.73:6833/2353282105","heartbeat_front_addr":"172.21.15.73:6831/2353282105","state":["exists","up"]},{"osd":10,"uuid":"cac56766-4505-4e9f-9b82-2dbf14ba3162","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6824","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6825","nonce":2185418171}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6826","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6827","nonce":2185418171}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6831","nonce":2185418171}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":2185418171},{"type":"v1","addr":"172.21.15.136:6829","nonce":2185418171}]},"public_addr":"172.21.15.136:6825/2185418171","cluster_addr":"172.21.15.136:6827/2185418171","heartbeat_back_addr":"172.21.15.136:6831/2185418171","heartbeat_front_addr":"172.21.15.136:6829/2185418171","state":["exists","up"]},{"osd":11,"uuid":"c8610f25-d950-4f87-b337-29b59b3f71a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":97,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6824","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6825","nonce":3729886831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6826","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6827","nonce":3729886831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6830","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6831","nonce":3729886831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6828","nonce":3729886831},{"type":"v1","addr":"172.21.15.170:6829","nonce":3729886831}]},"public_addr":"172.21.15.170:6825/3729886831","cluster_addr":"172.21.15.170:6827/3729886831","heartbeat_back_addr":"172.21.15.170:6831/3729886831","heartbeat_front_addr":"172.21.15.170:6829/3729886831","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:25.892886+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:51:49.615937+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:13.093067+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:52:38.734278+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:03.057039+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:27.700284+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:53:54.095559+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:18.973570+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:54:44.822393+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:11.404817+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:55:37.270092+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-06T06:56:02.951664+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.0","mappings":[{"from":8,"to":4}]},{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.1a","mappings":[{"from":5,"to":7}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]},{"pgid":"3.3a","mappings":[{"from":4,"to":8}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.13","mappings":[{"from":9,"to":10}]},{"pgid":"4.1b","mappings":[{"from":9,"to":8}]},{"pgid":"4.1c","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6800/793833713":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1478961120":"2024-06-07T07:01:21.691366+0000","172.21.15.73:6800/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1159052463":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/1988924098":"2024-06-07T07:01:21.691366+0000","172.21.15.73:6801/3054677757":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/2836597527":"2024-06-07T06:49:51.216738+0000","172.21.15.73:0/3113057040":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1117851147":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/905820354":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/27064626":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/793833713":"2024-06-07T07:01:21.691366+0000","172.21.15.73:0/1626604755":"2024-06-07T06:49:24.637645+0000","172.21.15.73:0/2368928774":"2024-06-07T06:49:51.216738+0000","172.21.15.73:6800/961923295":"2024-06-07T06:49:24.637645+0000","172.21.15.73:6801/961923295":"2024-06-07T06:49:24.637645+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]},{"pool":4,"snaps":[{"begin":32,"length":1},{"begin":45,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-06T07:27:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 msgr2=0x7f7d5c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 0x7f7d5c0a51c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7d640096c0 tx=0x7f7d64031400 comp rx=0 tx=0).stop 2024-06-06T07:27:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 msgr2=0x7f7d7c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400db80 tx=0x7f7d7400b630 comp rx=0 tx=0).stop 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 shutdown_connections 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f7d5c0a2d00 0x7f7d5c0a51c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f7d7c106880 0x7f7d7c19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f7d7c101230 0x7f7d7c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 --2- 172.21.15.73:0/598321932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d7c100870 0x7f7d7c1981c0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 >> 172.21.15.73:0/598321932 conn(0x7f7d7c076c20 msgr2=0x7f7d7c0fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 shutdown_connections 2024-06-06T07:27:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.289+0000 7f7d80cf5640 1 -- 172.21.15.73:0/598321932 wait complete. 2024-06-06T07:27:04.303 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-06T06:52:17.534158+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-06T06:57:03.217358+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-06T06:57:06.475737+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 145, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-06-06T06:57:16.584799+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 45, 'snap_epoch': 145, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-06-06T07:27:04.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-06T07:27:04.409 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.406+0000 7f3c28fc1640 1 -- 172.21.15.73:0/3233460477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24103540 msgr2=0x7f3c2410ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.406+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/3233460477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24103540 0x7f3c2410ba20 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c009900 tx=0x7f3c0c02a510 comp rx=0 tx=0).stop 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- 172.21.15.73:0/3233460477 shutdown_connections 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/3233460477 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2410bf60 0x7f3c2410e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/3233460477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24103540 0x7f3c2410ba20 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/3233460477 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24102c20 0x7f3c24103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- 172.21.15.73:0/3233460477 >> 172.21.15.73:0/3233460477 conn(0x7f3c240779a0 msgr2=0x7f3c24077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- 172.21.15.73:0/3233460477 shutdown_connections 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- 172.21.15.73:0/3233460477 wait complete. 2024-06-06T07:27:04.410 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 Processor -- start 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- start start 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 0x7f3c24198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2410bf60 0x7f3c2419ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24111c10 con 0x7f3c24102c20 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24111a90 con 0x7f3c24103540 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c28fc1640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f3c24111d90 con 0x7f3c2410bf60 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c2259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c2259c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34712/0 (socket says 172.21.15.73:34712) 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.407+0000 7f3c2259c640 1 -- 172.21.15.73:0/561942486 learned_addr learned my addr 172.21.15.73:0/561942486 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c21d9b640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 0x7f3c24198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 -- 172.21.15.73:0/561942486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2410bf60 msgr2=0x7f3c2419ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2410bf60 0x7f3c2419ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.411 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 -- 172.21.15.73:0/561942486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 msgr2=0x7f3c24198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 0x7f3c24198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 -- 172.21.15.73:0/561942486 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c2419d1c0 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c21d9b640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 0x7f3c24198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c2259c640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400eab0 tx=0x7f3c1400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400ce40 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c14011920 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c14010a00 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c2419d4b0 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.408+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c241a4d40 con 0x7f3c24102c20 2024-06-06T07:27:04.412 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.409+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c240687c0 con 0x7f3c24102c20 2024-06-06T07:27:04.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.410+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f3c14011ad0 con 0x7f3c24102c20 2024-06-06T07:27:04.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.411+0000 7f3c1b7fe640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 0x7f3c100a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.411+0000 7f3c21d9b640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 0x7f3c100a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.411+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 9554+0+0 (secure 0 0 0) 0x7f3c140cff30 con 0x7f3c24102c20 2024-06-06T07:27:04.413 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.412+0000 7f3c21d9b640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 0x7f3c100a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c031440 tx=0x7f3c0c005a20 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:04.415 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.415+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3c14016030 con 0x7f3c24102c20 2024-06-06T07:27:04.760 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.759+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3c2402d2e0 con 0x7f3c24102c20 2024-06-06T07:27:04.761 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.760+0000 7f3c1b7fe640 1 -- 172.21.15.73:0/561942486 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) ==== 93+0+10 (secure 0 0 0) 0x7f3c1409c740 con 0x7f3c24102c20 2024-06-06T07:27:04.761 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2024-06-06T07:27:04.764 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 msgr2=0x7f3c100a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 0x7f3c100a51c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3c0c031440 tx=0x7f3c0c005a20 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 msgr2=0x7f3c241981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f3c1400eab0 tx=0x7f3c1400ef70 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 shutdown_connections 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f3c100a2d00 0x7f3c100a51c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f3c2410bf60 0x7f3c2419ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f3c24103540 0x7f3c24198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 --2- 172.21.15.73:0/561942486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c24102c20 0x7f3c241981d0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.765 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.764+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 >> 172.21.15.73:0/561942486 conn(0x7f3c240779a0 msgr2=0x7f3c240fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:04.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.766+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 shutdown_connections 2024-06-06T07:27:04.766 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.766+0000 7f3c28fc1640 1 -- 172.21.15.73:0/561942486 wait complete. 2024-06-06T07:27:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:04 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:04 smithi073 ceph-mon[32303]: from='client.17913 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:04.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:04 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/598321932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:27:04.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-06T07:27:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:04 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:04.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:04 smithi136 ceph-mon[35901]: from='client.17913 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:04.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:04 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/598321932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:27:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:04 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:04 smithi170 ceph-mon[36108]: from='client.17913 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:04.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:04 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/598321932' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-06T07:27:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.971+0000 7f6e0eea5640 1 -- 172.21.15.73:0/3036281297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 msgr2=0x7f6e08114df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.971+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/3036281297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e08114df0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009ca0 tx=0x7f6e0002a980 comp rx=0 tx=0).stop 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.972+0000 7f6e0eea5640 1 -- 172.21.15.73:0/3036281297 shutdown_connections 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.972+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/3036281297 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e08114df0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.972+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/3036281297 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08074660 0x7f6e081124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.972+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/3036281297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08073d40 0x7f6e08074120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.972+0000 7f6e0eea5640 1 -- 172.21.15.73:0/3036281297 >> 172.21.15.73:0/3036281297 conn(0x7f6e080fd500 msgr2=0x7f6e080ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:04.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.974+0000 7f6e0eea5640 1 -- 172.21.15.73:0/3036281297 shutdown_connections 2024-06-06T07:27:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.977+0000 7f6e0eea5640 1 -- 172.21.15.73:0/3036281297 wait complete. 2024-06-06T07:27:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.977+0000 7f6e0eea5640 1 Processor -- start 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 -- start start 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08073d40 0x7f6e080731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 0x7f6e0806c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6e081183e0 con 0x7f6e08074660 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08118260 con 0x7f6e08112a00 2024-06-06T07:27:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.979+0000 7f6e0eea5640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f6e08118560 con 0x7f6e08073d40 2024-06-06T07:27:04.982 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.982+0000 7f6e0d442640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.983+0000 7f6e0d442640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.136:3300/0 says I am v2:172.21.15.73:57248/0 (socket says 172.21.15.73:57248) 2024-06-06T07:27:04.984 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.983+0000 7f6e0d442640 1 -- 172.21.15.73:0/2470007267 learned_addr learned my addr 172.21.15.73:0/2470007267 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:04.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.984+0000 7f6e0cc41640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08073d40 0x7f6e080731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e0d442640 1 -- 172.21.15.73:0/2470007267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08073d40 msgr2=0x7f6e080731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e07fff640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 0x7f6e0806c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e0d442640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08073d40 0x7f6e080731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.987 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e0d442640 1 -- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 msgr2=0x7f6e0806c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:04.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e0d442640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 0x7f6e0806c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:04.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e0d442640 1 -- 172.21.15.73:0/2470007267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e0806cde0 con 0x7f6e08112a00 2024-06-06T07:27:04.988 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.986+0000 7f6e07fff640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 0x7f6e0806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:27:04.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.987+0000 7f6e0d442640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e00006010 tx=0x7f6e0002f970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:04.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.987+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00034070 con 0x7f6e08112a00 2024-06-06T07:27:04.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.988+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e00004040 con 0x7f6e08112a00 2024-06-06T07:27:04.989 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.988+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e0002e7f0 con 0x7f6e08112a00 2024-06-06T07:27:04.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.988+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e0806d070 con 0x7f6e08112a00 2024-06-06T07:27:04.990 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.988+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e08070fa0 con 0x7f6e08112a00 2024-06-06T07:27:04.993 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.992+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f6e00037020 con 0x7f6e08112a00 2024-06-06T07:27:04.994 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.993+0000 7f6e05ffb640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 0x7f6df00a5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:04.995 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.994+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9642+0+0 (secure 0 0 0) 0x7f6e000f0820 con 0x7f6e08112a00 2024-06-06T07:27:04.996 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.994+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e08075480 con 0x7f6e08112a00 2024-06-06T07:27:04.997 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.995+0000 7f6e0cc41640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 0x7f6df00a5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:04.999 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:04.998+0000 7f6e0cc41640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 0x7f6df00a5300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00a7d0 tx=0x7f6dfc008040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:05.004 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.003+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6e000098b0 con 0x7f6e08112a00 2024-06-06T07:27:05.300 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.300+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6e0806d760 con 0x7f6e08112a00 2024-06-06T07:27:05.301 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.300+0000 7f6e05ffb640 1 -- 172.21.15.73:0/2470007267 <== mon.2 v2:172.21.15.136:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) ==== 104+0+11 (secure 0 0 0) 0x7f6e000bcf80 con 0x7f6e08112a00 2024-06-06T07:27:05.301 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 msgr2=0x7f6df00a5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 0x7f6df00a5300 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00a7d0 tx=0x7f6dfc008040 comp rx=0 tx=0).stop 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 msgr2=0x7f6e0806c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e00006010 tx=0x7f6e0002f970 comp rx=0 tx=0).stop 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 shutdown_connections 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f6e08112a00 0x7f6e0806c770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e08074660 0x7f6e0806c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f6df00a2e40 0x7f6df00a5300 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 --2- 172.21.15.73:0/2470007267 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f6e08073d40 0x7f6e080731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 >> 172.21.15.73:0/2470007267 conn(0x7f6e080fd500 msgr2=0x7f6e080fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 shutdown_connections 2024-06-06T07:27:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.302+0000 7f6e0eea5640 1 -- 172.21.15.73:0/2470007267 wait complete. 2024-06-06T07:27:05.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-06T07:27:05.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- 172.21.15.73:0/1256655186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 msgr2=0x7fcacc10e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- 172.21.15.73:0/1256655186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc10e400 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009900 tx=0x7fcac402a510 comp rx=0 tx=0).stop 2024-06-06T07:27:05.420 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- 172.21.15.73:0/1256655186 shutdown_connections 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- 172.21.15.73:0/1256655186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc10e400 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- 172.21.15.73:0/1256655186 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 0x7fcacc10ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- 172.21.15.73:0/1256655186 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 0x7fcacc103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- 172.21.15.73:0/1256655186 >> 172.21.15.73:0/1256655186 conn(0x7fcacc0779a0 msgr2=0x7fcacc077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- 172.21.15.73:0/1256655186 shutdown_connections 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- 172.21.15.73:0/1256655186 wait complete. 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 Processor -- start 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- start start 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 0x7fcacc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 0x7fcacc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc111c10 con 0x7fcacc10bf60 2024-06-06T07:27:05.421 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc111a90 con 0x7fcacc103540 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.418+0000 7fcad4476640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7fcacc111d90 con 0x7fcacc102c20 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50918/0 (socket says 172.21.15.73:50918) 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 -- 172.21.15.73:0/1067778900 learned_addr learned my addr 172.21.15.73:0/1067778900 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2212640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 0x7fcacc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad1a11640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 0x7fcacc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 -- 172.21.15.73:0/1067778900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 msgr2=0x7fcacc1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 0x7fcacc1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 -- 172.21.15.73:0/1067778900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 msgr2=0x7fcacc1985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 0x7fcacc1985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 -- 172.21.15.73:0/1067778900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcacc19d0a0 con 0x7fcacc10bf60 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad2a13640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009230 tx=0x7fcac4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:05.422 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac403f030 con 0x7fcacc10bf60 2024-06-06T07:27:05.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcac4003aa0 con 0x7fcacc10bf60 2024-06-06T07:27:05.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcacc19d330 con 0x7fcacc10bf60 2024-06-06T07:27:05.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4035070 con 0x7fcacc10bf60 2024-06-06T07:27:05.423 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.419+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcacc1a4c20 con 0x7fcacc10bf60 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.420+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcacc0687c0 con 0x7fcacc10bf60 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.421+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7fcac4042030 con 0x7fcacc10bf60 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.422+0000 7fcacb7fe640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 0x7fcab40a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.422+0000 7fcad2212640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 0x7fcab40a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.422+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9642+0+0 (secure 0 0 0) 0x7fcac40efef0 con 0x7fcacc10bf60 2024-06-06T07:27:05.424 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.423+0000 7fcad2212640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 0x7fcab40a51c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005fd0 tx=0x7fcabc009680 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:05.430 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.429+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcac40bc6b0 con 0x7fcacc10bf60 2024-06-06T07:27:05.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.608+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fcacc1070e0 con 0x7fcacc10bf60 2024-06-06T07:27:05.609 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.608+0000 7fcacb7fe640 1 -- 172.21.15.73:0/1067778900 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) ==== 100+0+11 (secure 0 0 0) 0x7fcac40bbfb0 con 0x7fcacc10bf60 2024-06-06T07:27:05.609 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2024-06-06T07:27:05.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 msgr2=0x7fcab40a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 0x7fcab40a51c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fcabc005fd0 tx=0x7fcabc009680 comp rx=0 tx=0).stop 2024-06-06T07:27:05.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 msgr2=0x7fcacc19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009230 tx=0x7fcac4004270 comp rx=0 tx=0).stop 2024-06-06T07:27:05.611 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 shutdown_connections 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcacc10bf60 0x7fcacc19c970 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7fcacc103540 0x7fcacc1985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7fcab40a2d00 0x7fcab40a51c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 --2- 172.21.15.73:0/1067778900 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7fcacc102c20 0x7fcacc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 >> 172.21.15.73:0/1067778900 conn(0x7fcacc0779a0 msgr2=0x7fcacc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 shutdown_connections 2024-06-06T07:27:05.612 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.610+0000 7fcad4476640 1 -- 172.21.15.73:0/1067778900 wait complete. 2024-06-06T07:27:05.621 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:05 smithi073 ceph-mon[32303]: from='client.17919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:05.621 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:05 smithi073 ceph-mon[32303]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-06-06T07:27:05.621 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:05 smithi073 ceph-mon[32303]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T07:27:05.621 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:05 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/561942486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.621 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:05 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2470007267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-06-06T07:27:05.728 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 -- 172.21.15.73:0/413439966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8230103540 msgr2=0x7f823010ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 --2- 172.21.15.73:0/413439966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8230103540 0x7f823010ba20 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8218009900 tx=0x7f821802a510 comp rx=0 tx=0).stop 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 -- 172.21.15.73:0/413439966 shutdown_connections 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 --2- 172.21.15.73:0/413439966 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f823010bf60 0x7f823010e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 --2- 172.21.15.73:0/413439966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8230103540 0x7f823010ba20 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 --2- 172.21.15.73:0/413439966 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230102c20 0x7f8230103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 -- 172.21.15.73:0/413439966 >> 172.21.15.73:0/413439966 conn(0x7f82300779a0 msgr2=0x7f8230077db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 -- 172.21.15.73:0/413439966 shutdown_connections 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.725+0000 7f8236012640 1 -- 172.21.15.73:0/413439966 wait complete. 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 Processor -- start 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 -- start start 2024-06-06T07:27:05.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 0x7f8230198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 0x7f8230198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111c10 con 0x7f823010bf60 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111a90 con 0x7f8230103540 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8236012640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f8230111d90 con 0x7f8230102c20 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f822ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f822ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50936/0 (socket says 172.21.15.73:50936) 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f822ffff640 1 -- 172.21.15.73:0/2446540231 learned_addr learned my addr 172.21.15.73:0/2446540231 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f822f7fe640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 0x7f8230198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.726+0000 7f8227fff640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 0x7f8230198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 -- 172.21.15.73:0/2446540231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 msgr2=0x7f8230198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 0x7f8230198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 -- 172.21.15.73:0/2446540231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 msgr2=0x7f8230198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 0x7f8230198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 -- 172.21.15.73:0/2446540231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f823019d240 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822f7fe640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 0x7f8230198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f8227fff640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 0x7f8230198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822ffff640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f822800eb10 tx=0x7f822800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228004b20 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8228004cc0 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f823019d530 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228010690 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.727+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82301a4dc0 con 0x7f823010bf60 2024-06-06T07:27:05.731 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.728+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82300687c0 con 0x7f823010bf60 2024-06-06T07:27:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.729+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f8228005000 con 0x7f823010bf60 2024-06-06T07:27:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.730+0000 7f822d7fa640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 0x7f821c0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.730+0000 7f822f7fe640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 0x7f821c0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.730+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9642+0+0 (secure 0 0 0) 0x7f82280d0380 con 0x7f823010bf60 2024-06-06T07:27:05.733 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.730+0000 7f822f7fe640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 0x7f821c0a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8220009c20 tx=0x7f8220005f90 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:05.734 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.733+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f822809cb40 con 0x7f823010bf60 2024-06-06T07:27:05.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:05 smithi136 ceph-mon[35901]: from='client.17919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:05.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:05 smithi136 ceph-mon[35901]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-06-06T07:27:05.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:05 smithi136 ceph-mon[35901]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T07:27:05.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:05 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/561942486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:05 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2470007267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:05 smithi170 ceph-mon[36108]: from='client.17919 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-06T07:27:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:05 smithi170 ceph-mon[36108]: pgmap v819: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 6.0 MiB/s wr, 471 op/s 2024-06-06T07:27:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:05 smithi170 ceph-mon[36108]: osdmap e147: 12 total, 12 up, 12 in 2024-06-06T07:27:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:05 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/561942486' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:05 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2470007267' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-06T07:27:05.914 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.913+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8230106b50 con 0x7f823010bf60 2024-06-06T07:27:05.915 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.914+0000 7f822d7fa640 1 -- 172.21.15.73:0/2446540231 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v147) ==== 103+0+11 (secure 0 0 0) 0x7f822809c440 con 0x7f823010bf60 2024-06-06T07:27:05.915 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 32 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 msgr2=0x7f821c0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 0x7f821c0a51c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8220009c20 tx=0x7f8220005f90 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 msgr2=0x7f823019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f822800eb10 tx=0x7f822800c2d0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 shutdown_connections 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f823010bf60 0x7f823019cb10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f8230103540 0x7f8230198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f821c0a2d00 0x7f821c0a51c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 --2- 172.21.15.73:0/2446540231 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f8230102c20 0x7f8230198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 >> 172.21.15.73:0/2446540231 conn(0x7f82300779a0 msgr2=0x7f82300fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:05.917 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 shutdown_connections 2024-06-06T07:27:05.918 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:05.916+0000 7f8236012640 1 -- 172.21.15.73:0/2446540231 wait complete. 2024-06-06T07:27:05.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-06T07:27:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 -- 172.21.15.73:0/2293035037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394101980 msgr2=0x7ff394101e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 --2- 172.21.15.73:0/2293035037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394101980 0x7ff394101e00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff384009930 tx=0x7ff38402a420 comp rx=0 tx=0).stop 2024-06-06T07:27:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 -- 172.21.15.73:0/2293035037 shutdown_connections 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 --2- 172.21.15.73:0/2293035037 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff39410e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 --2- 172.21.15.73:0/2293035037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394101980 0x7ff394101e00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 --2- 172.21.15.73:0/2293035037 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394107990 0x7ff394107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 -- 172.21.15.73:0/2293035037 >> 172.21.15.73:0/2293035037 conn(0x7ff3940fd410 msgr2=0x7ff3940ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 -- 172.21.15.73:0/2293035037 shutdown_connections 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.029+0000 7ff39c363640 1 -- 172.21.15.73:0/2293035037 wait complete. 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 Processor -- start 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 -- start start 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 0x7ff394198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff394198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff394111c50 con 0x7ff394107990 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7ff394111ad0 con 0x7ff394102340 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39c363640 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7ff394111dd0 con 0x7ff394101980 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50942/0 (socket says 172.21.15.73:50942) 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 -- 172.21.15.73:0/1340306521 learned_addr learned my addr 172.21.15.73:0/1340306521 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff3998fe640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff394198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a0ff640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 0x7ff394198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 -- 172.21.15.73:0/1340306521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 msgr2=0x7ff394198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 0x7ff394198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 -- 172.21.15.73:0/1340306521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 msgr2=0x7ff394198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff394198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 -- 172.21.15.73:0/1340306521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff39419d220 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a0ff640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 0x7ff394198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff3998fe640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff394198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff39a900640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.030+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c017070 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.031+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff38c003e20 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.031+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38c010460 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.031+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff39419d510 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.031+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3941a4da0 con 0x7ff394107990 2024-06-06T07:27:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.032+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff394068c10 con 0x7ff394107990 2024-06-06T07:27:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.033+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7ff38c003970 con 0x7ff394107990 2024-06-06T07:27:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.033+0000 7ff38b7fe640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 0x7ff3800a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.033+0000 7ff39a0ff640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 0x7ff3800a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.034+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 9642+0+0 (secure 0 0 0) 0x7ff38c0cf790 con 0x7ff394107990 2024-06-06T07:27:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.034+0000 7ff39a0ff640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 0x7ff3800a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff3900094d0 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:06.038 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.038+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff38c09bf50 con 0x7ff394107990 2024-06-06T07:27:06.262 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.261+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff394109f40 con 0x7ff394107990 2024-06-06T07:27:06.270 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.270+0000 7ff38b7fe640 1 -- 172.21.15.73:0/1340306521 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+740903 (secure 0 0 0) 0x7ff38c09b850 con 0x7ff394107990 2024-06-06T07:27:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-06-06T07:27:06.280 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:27:06.280 INFO:teuthology.orchestra.run.smithi073.stdout: "epoch": 25, 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "active_gid": 15282, 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "active_name": "x", 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "active_addrs": { 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6800", 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3841884199 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v1", 2024-06-06T07:27:06.281 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:6801", 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3841884199 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "active_addr": "172.21.15.73:6801/3841884199", 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "active_change": "2024-06-06T07:01:21.691603+0000", 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "active_mgr_features": 4540701547738038271, 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "available": true, 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "standbys": [ 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 24839, 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "y", 2024-06-06T07:27:06.282 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_features": 4540701547738038271, 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "available_modules": [ 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "alerts", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.283 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.284 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.285 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.287 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.288 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_destination": { 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_destination", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.289 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email address to send alerts to", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_from_name": { 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_from_name", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.290 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "Ceph", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email From: name", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_host": { 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_host", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.291 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP server", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.292 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_password": { 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_password", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Password to authenticate with", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.293 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_port": { 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_port", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "465", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP port", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_sender": { 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_sender", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP envelope sender", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_ssl": { 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_ssl", 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.296 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.297 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_user": { 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_user", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "User to authenticate as", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.298 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "balancer", 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "active": { 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "active", 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.299 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_time": { 2024-06-06T07:27:06.300 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_time", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0000", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.301 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_weekday": { 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_weekday", 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.302 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.303 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_max_iterations": { 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_max_iterations", 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.304 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "25", 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "250", 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.305 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_metrics": { 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_metrics", 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.306 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T07:27:06.307 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_step": { 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_step", 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.308 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.5", 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0.001", 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "0.999", 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.309 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "aggressiveness of optimization", 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: "end_time": { 2024-06-06T07:27:06.310 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_time", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2359", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.311 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "end_weekday": { 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_weekday", 2024-06-06T07:27:06.312 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.313 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.315 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.316 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.317 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.318 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.318 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.319 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.320 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.321 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.322 INFO:teuthology.orchestra.run.smithi073.stdout: "min_score": { 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_score", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.323 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "upmap", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-compat", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "none", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "read", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap", 2024-06-06T07:27:06.324 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap-read" 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Balancer mode", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_ids": { 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_ids", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.325 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.326 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_deviation": { 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_deviation", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.327 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.328 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_optimizations": { 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_optimizations", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.329 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm", 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_down_multiplier": { 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_down_multiplier", 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.330 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_refresh_rate": { 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_refresh_rate", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.331 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "20", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_starting_port": { 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_starting_port", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.332 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4721", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_ptrace": { 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_ptrace", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.333 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_interval": { 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_interval", 2024-06-06T07:27:06.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_memory_target_ratio": { 2024-06-06T07:27:06.335 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.7", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_log_destination": { 2024-06-06T07:27:06.336 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm_log_destination", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "file", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "file,syslog", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "syslog" 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.337 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "cgroups_split": { 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cgroups_split", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T07:27:06.338 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "config_checks_enabled": { 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_checks_enabled", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T07:27:06.339 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "config_dashboard": { 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_dashboard", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T07:27:06.340 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_alertmanager": { 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_alertmanager", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.341 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_base": { 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_base", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.342 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Container image name, without the tag", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_elasticsearch": { 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_elasticsearch", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.343 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "elasticsearch container image", 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_grafana": { 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_grafana", 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.344 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.345 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_haproxy": { 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_haproxy", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.346 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "HAproxy container image", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_agent": { 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_agent", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T07:27:06.347 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger agent container image", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_collector": { 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_collector", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T07:27:06.348 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.349 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.349 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.349 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger collector container image", 2024-06-06T07:27:06.349 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.349 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_query": { 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_query", 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T07:27:06.350 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger query container image", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_keepalived": { 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_keepalived", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.351 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Keepalived container image", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_loki": { 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_loki", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.352 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Loki container image", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.353 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_node_exporter": { 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_node_exporter", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.354 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_nvmeof": { 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_nvmeof", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Nvme-of container image", 2024-06-06T07:27:06.355 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_prometheus": { 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_prometheus", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.356 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_promtail": { 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_promtail", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Promtail container image", 2024-06-06T07:27:06.357 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_samba": { 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_samba", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.358 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Samba/SMB container image", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_snmp_gateway": { 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_snmp_gateway", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.359 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SNMP Gateway container image", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "container_init": { 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_init", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.360 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "daemon_cache_timeout": { 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "daemon_cache_timeout", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.361 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "default_cephadm_command_timeout": { 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.362 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "900", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "default_registry": { 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_registry", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.363 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io", 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "device_cache_timeout": { 2024-06-06T07:27:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_cache_timeout", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1800", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache device inventory", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.365 INFO:teuthology.orchestra.run.smithi073.stdout: "device_enhanced_scan": { 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_enhanced_scan", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "facts_cache_timeout": { 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "facts_cache_timeout", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache host facts data", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.367 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "grafana_dashboards_path": { 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "grafana_dashboards_path", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.368 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "host_check_interval": { 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "host_check_interval", 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.369 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to perform a host check", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "hw_monitoring": { 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hw_monitoring", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.370 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "inventory_list_all": { 2024-06-06T07:27:06.371 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "inventory_list_all", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.373 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "log_refresh_metadata": { 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_refresh_metadata", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.374 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T07:27:06.376 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.377 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.378 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T07:27:06.379 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.380 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.381 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "*", 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.382 INFO:teuthology.orchestra.run.smithi073.stdout: "max_count_per_host": { 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_count_per_host", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of daemons per service per host", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd_draining_count": { 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_osd_draining_count", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.384 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "migration_current": { 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "migration_current", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.385 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "internal - do not modify", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "root", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.386 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm-package", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "root" 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "oob_default_addr": { 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "oob_default_addr", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "169.254.1.1", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "prometheus_alerts_path": { 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus_alerts_path", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_insecure": { 2024-06-06T07:27:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_insecure", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_password": { 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_password", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_url": { 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_url", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.392 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_username": { 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_username", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.393 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "secure_monitoring_stack": { 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "secure_monitoring_stack", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.394 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "service_discovery_port": { 2024-06-06T07:27:06.395 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "service_discovery_port", 2024-06-06T07:27:06.398 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.398 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8765", 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.399 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "cephadm service discovery port", 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "ssh_config_file": { 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssh_config_file", 2024-06-06T07:27:06.400 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.401 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.402 INFO:teuthology.orchestra.run.smithi073.stdout: "use_agent": { 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_agent", 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.403 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "use_repo_digest": { 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_repo_digest", 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.405 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.406 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.406 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.406 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.406 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_failed_host_check": { 2024-06-06T07:27:06.407 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_failed_host_check", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.408 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.409 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_daemons": { 2024-06-06T07:27:06.410 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_daemons", 2024-06-06T07:27:06.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.410 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.410 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.410 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.411 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_hosts": { 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_hosts", 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.412 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T07:27:06.413 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.415 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crash", 2024-06-06T07:27:06.415 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.415 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.415 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.415 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.416 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.417 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.418 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.419 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.420 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.421 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.422 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.423 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.424 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.424 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.424 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.424 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.426 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: "retain_interval": { 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retain_interval", 2024-06-06T07:27:06.427 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "31536000", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_recent_interval": { 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_recent_interval", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1209600", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.429 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dashboard", 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.431 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.432 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_ENABLED": { 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.433 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.434 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.435 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.438 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.439 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.442 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.443 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T07:27:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.447 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_URL": { 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_URL", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.448 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.449 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.450 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T07:27:06.451 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.452 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T07:27:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T07:27:06.453 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.454 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.455 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.456 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.457 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.458 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.459 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.460 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.461 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T07:27:06.462 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.463 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.464 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.466 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.467 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.468 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.469 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.470 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.471 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T07:27:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T07:27:06.472 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.473 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.474 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.475 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.476 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.477 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.478 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.480 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "45", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T07:27:06.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.483 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.484 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.485 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.486 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.487 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.488 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "cross_origin_url": { 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cross_origin_url", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.489 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "crt_file": { 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crt_file", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.490 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "debug": { 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "debug", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable/disable debug options", 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "jwt_token_ttl": { 2024-06-06T07:27:06.492 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "jwt_token_ttl", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "28800", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:06.493 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.495 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.496 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.499 INFO:teuthology.orchestra.run.smithi073.stdout: "motd": { 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "motd", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "The message of the day", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.500 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect_resolve_ip_addr": { 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:06.501 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.502 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8080", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.504 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl_server_port": { 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl_server_port", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8443", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.505 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:06.506 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "redirect", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect" 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:06.508 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "url_prefix": { 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url_prefix", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.509 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_monitoring": { 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_monitoring", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.511 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "monitor device health metrics", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.512 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.513 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.514 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.515 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.516 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.517 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "mark_out_threshold": { 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mark_out_threshold", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2419200", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": { 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_name", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "device_health_metrics", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.519 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "retention_period": { 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retention_period", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15552000", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain device health metrics", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_frequency": { 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_frequency", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.521 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "self_heal": { 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "self_heal", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.522 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T07:27:06.523 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_threshold": { 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_threshold", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7257600", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T07:27:06.524 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "diskprediction_local", 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.525 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.526 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.527 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.528 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.529 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "predict_interval": { 2024-06-06T07:27:06.530 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predict_interval", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "predictor_model": { 2024-06-06T07:27:06.531 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predictor_model", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "prophetstor", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "influx", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": false, 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "influxdb python module not found", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "batch_size": { 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "batch_size", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5000", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.534 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "database": { 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "database", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "ceph", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.535 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": { 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hostname", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.536 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server hostname", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.537 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "30", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "5", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.538 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.539 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.540 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.541 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.542 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "password": { 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "password", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.543 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "password of InfluxDB server user", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "port": { 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "port", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.544 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8086", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server port", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:06.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "false", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "threads": { 2024-06-06T07:27:06.546 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threads", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "32", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T07:27:06.547 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "username": { 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "username", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.548 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "username of InfluxDB server user", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "verify_ssl": { 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "verify_ssl", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.549 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "true", 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "insights", 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.551 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.552 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.554 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.555 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.556 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "iostat", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.558 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.559 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.560 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.561 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.562 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "k8sevents", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.563 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_event_retention_days": { 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ceph_event_retention_days", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "config_check_secs": { 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_check_secs", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "10", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T07:27:06.565 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.566 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.567 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.569 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.570 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "localpool", 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "failure_domain": { 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "failure_domain", 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "host", 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "failure domain for any created local pool", 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.574 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.575 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.576 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.577 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.579 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": { 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_size", 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.580 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.581 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.581 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.581 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.581 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default min_size for any created local pool", 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.582 INFO:teuthology.orchestra.run.smithi073.stdout: "num_rep": { 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "num_rep", 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3", 2024-06-06T07:27:06.583 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.584 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.584 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.584 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default replica count for any created local pool", 2024-06-06T07:27:06.584 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.584 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": { 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_num", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "128", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.585 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "prefix": { 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prefix", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.586 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name prefix for any created local pool", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "subtree": { 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subtree", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "rack", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.587 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mds_autoscaler", 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.588 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.589 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.590 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.592 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.593 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirroring", 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.594 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.595 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.596 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.598 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.601 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.603 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "nfs", 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.605 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.606 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.607 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.608 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.609 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.610 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.611 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.612 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.613 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.614 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.615 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "fail_fs": { 2024-06-06T07:27:06.617 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fail_fs", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.618 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.619 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.620 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.621 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.622 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.623 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.625 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.626 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator": { 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:06.627 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "rook", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "test_orchestrator" 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Orchestrator backend", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_perf_query", 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.631 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.632 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.633 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_support", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.635 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.636 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.638 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.639 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_autoscaler", 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.640 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.641 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.642 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.643 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.644 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.645 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "threshold": { 2024-06-06T07:27:06.646 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threshold", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1.0", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "scaling threshold", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "progress", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_pg_recovery_event": { 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_pg_recovery_event", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.648 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.649 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.650 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.651 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.652 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.653 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.654 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "max_completed_events": { 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_completed_events", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "50", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.655 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "number of past completed events to remember", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.656 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long the module is going to sleep", 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus", 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.657 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "cache": { 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cache", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.658 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "exclude_perf_counters": { 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "exclude_perf_counters", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.659 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.660 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.661 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.662 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.663 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools": { 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "300", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.666 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_interval": { 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_interval", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15.0", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.667 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.668 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "9283", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "stale_cache_strategy": { 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stale_cache_strategy", 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.670 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "log", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.671 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "default", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "default", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "error" 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.672 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.674 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_snap_create": { 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_snap_create", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mirror_snapshot_schedule": { 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "trash_purge_schedule": { 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "trash_purge_schedule", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "restful", 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_auth": { 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_auth", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rgw", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rook", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "storage_class": { 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "storage_class", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "local", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "selftest", 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "roption1": { 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption1", 2024-06-06T07:27:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "roption2": { 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption2", 2024-06-06T07:27:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "xyz", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption1": { 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption1", 2024-06-06T07:27:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption2": { 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption2", 2024-06-06T07:27:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption3": { 2024-06-06T07:27:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption3", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption4": { 2024-06-06T07:27:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption4", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption5": { 2024-06-06T07:27:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption5", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption6": { 2024-06-06T07:27:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption6", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption7": { 2024-06-06T07:27:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption7", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "42", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "testkey": { 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testkey", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "testlkey": { 2024-06-06T07:27:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testlkey", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "testnewline": { 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testnewline", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_m_granularity": { 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_m_granularity", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "dump_on_update": { 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dump_on_update", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "dump database to debug log on update", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.724 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stats", 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "status", 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telegraf", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "address": { 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "address", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telemetry", 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_basic": { 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_basic", 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_crash": { 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_crash", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_device": { 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_device", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_ident": { 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_ident", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_perf": { 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_perf", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "contact": { 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "contact", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "description": { 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "description", 2024-06-06T07:27:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "device_url": { 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_url", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "24", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "8", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "last_opt_revision": { 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "last_opt_revision", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard": { 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard_description": { 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard_description", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "organization": { 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "organization", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "proxy": { 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "proxy", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "url": { 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "test_orchestrator", 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-06-06T07:27:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_clones": { 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_clones", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "periodic_async_work": { 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "periodic_async_work", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Periodically check for async work", 2024-06-06T07:27:06.782 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_delay": { 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_delay", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.783 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_no_wait": { 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.784 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix", 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "discovery_interval": { 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "discovery_interval", 2024-06-06T07:27:06.785 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "100", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "identifier": { 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "identifier", 2024-06-06T07:27:06.786 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.788 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.789 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.790 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.791 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_host": { 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_host", 2024-06-06T07:27:06.793 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_port": { 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_port", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.794 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10051", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_sender": { 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_sender", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.795 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.796 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "gid": 24904, 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "z", 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_features": 4540701547738038271, 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "available_modules": [ 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "alerts", 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.797 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.798 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.799 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.800 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.801 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.802 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_destination": { 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_destination", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.807 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email address to send alerts to", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_from_name": { 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_from_name", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "Ceph", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.808 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email From: name", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_host": { 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_host", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.809 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP server", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_password": { 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_password", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.810 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Password to authenticate with", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_port": { 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_port", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "465", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.811 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP port", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_sender": { 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_sender", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.812 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP envelope sender", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_ssl": { 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_ssl", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.813 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_user": { 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_user", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.814 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "User to authenticate as", 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "balancer", 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.815 INFO:teuthology.orchestra.run.smithi073.stdout: "active": { 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "active", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.816 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_time": { 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_time", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0000", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.817 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_weekday": { 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_weekday", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_max_iterations": { 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_max_iterations", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "25", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "250", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_metrics": { 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_metrics", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_step": { 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_step", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.5", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0.001", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "0.999", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "aggressiveness of optimization", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "end_time": { 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_time", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2359", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "end_weekday": { 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_weekday", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.824 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.825 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.826 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.827 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.828 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "min_score": { 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_score", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.829 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "upmap", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-compat", 2024-06-06T07:27:06.830 INFO:teuthology.orchestra.run.smithi073.stdout: "none", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "read", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap-read" 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Balancer mode", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_ids": { 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_ids", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.831 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.832 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_deviation": { 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_deviation", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.833 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_optimizations": { 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_optimizations", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm", 2024-06-06T07:27:06.835 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_down_multiplier": { 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_down_multiplier", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.836 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_refresh_rate": { 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_refresh_rate", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "20", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.837 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_starting_port": { 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_starting_port", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4721", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.838 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_ptrace": { 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_ptrace", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T07:27:06.839 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_interval": { 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_interval", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.840 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_memory_target_ratio": { 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.7", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.841 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_log_destination": { 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm_log_destination", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.842 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "file", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "file,syslog", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "syslog" 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "cgroups_split": { 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cgroups_split", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.843 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "config_checks_enabled": { 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_checks_enabled", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.844 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "config_dashboard": { 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_dashboard", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.845 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_alertmanager": { 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_alertmanager", 2024-06-06T07:27:06.846 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_base": { 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_base", 2024-06-06T07:27:06.847 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Container image name, without the tag", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_elasticsearch": { 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_elasticsearch", 2024-06-06T07:27:06.848 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "elasticsearch container image", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_grafana": { 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_grafana", 2024-06-06T07:27:06.849 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_haproxy": { 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_haproxy", 2024-06-06T07:27:06.850 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "HAproxy container image", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_agent": { 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_agent", 2024-06-06T07:27:06.851 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger agent container image", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.852 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_collector": { 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_collector", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger collector container image", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.853 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_query": { 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_query", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger query container image", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_keepalived": { 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_keepalived", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Keepalived container image", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_loki": { 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_loki", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Loki container image", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_node_exporter": { 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_node_exporter", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_nvmeof": { 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_nvmeof", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Nvme-of container image", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_prometheus": { 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_prometheus", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_promtail": { 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_promtail", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Promtail container image", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_samba": { 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_samba", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Samba/SMB container image", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_snmp_gateway": { 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_snmp_gateway", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SNMP Gateway container image", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.862 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "container_init": { 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_init", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "daemon_cache_timeout": { 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "daemon_cache_timeout", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.864 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "default_cephadm_command_timeout": { 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "900", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.865 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "default_registry": { 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_registry", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.866 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "device_cache_timeout": { 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_cache_timeout", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1800", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache device inventory", 2024-06-06T07:27:06.867 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "device_enhanced_scan": { 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_enhanced_scan", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.868 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "facts_cache_timeout": { 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "facts_cache_timeout", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache host facts data", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.869 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "grafana_dashboards_path": { 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "grafana_dashboards_path", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T07:27:06.870 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "host_check_interval": { 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "host_check_interval", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to perform a host check", 2024-06-06T07:27:06.871 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "hw_monitoring": { 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hw_monitoring", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T07:27:06.872 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "inventory_list_all": { 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "inventory_list_all", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T07:27:06.873 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.874 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "log_refresh_metadata": { 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_refresh_metadata", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.875 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.876 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.877 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.878 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.879 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.880 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "*", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.881 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "max_count_per_host": { 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_count_per_host", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of daemons per service per host", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd_draining_count": { 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_osd_draining_count", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.883 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "migration_current": { 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "migration_current", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.884 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "internal - do not modify", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "root", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.885 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm-package", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "root" 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "oob_default_addr": { 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "oob_default_addr", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.886 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "169.254.1.1", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "prometheus_alerts_path": { 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus_alerts_path", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.887 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_insecure": { 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_insecure", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.888 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_password": { 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_password", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.889 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_url": { 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_url", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.890 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_username": { 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_username", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.891 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "secure_monitoring_stack": { 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "secure_monitoring_stack", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.892 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "service_discovery_port": { 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "service_discovery_port", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.893 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8765", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "cephadm service discovery port", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "ssh_config_file": { 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssh_config_file", 2024-06-06T07:27:06.894 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "use_agent": { 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_agent", 2024-06-06T07:27:06.895 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "use_repo_digest": { 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_repo_digest", 2024-06-06T07:27:06.896 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_failed_host_check": { 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_failed_host_check", 2024-06-06T07:27:06.897 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_daemons": { 2024-06-06T07:27:06.898 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_daemons", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_hosts": { 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_hosts", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.900 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crash", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.901 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.902 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.903 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.904 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.905 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "retain_interval": { 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retain_interval", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.906 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "31536000", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_recent_interval": { 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_recent_interval", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.907 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1209600", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.908 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dashboard", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.909 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.910 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.911 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_ENABLED": { 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.912 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.913 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.914 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.915 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.916 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.917 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.918 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.919 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.920 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.921 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.922 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.923 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.924 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_URL": { 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_URL", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.926 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.927 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.928 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.929 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.937 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.938 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.939 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.940 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.941 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.942 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.943 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.944 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "45", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.945 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.946 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.947 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.948 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.949 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.950 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.951 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "cross_origin_url": { 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cross_origin_url", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.952 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "crt_file": { 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crt_file", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.953 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "debug": { 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "debug", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.954 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable/disable debug options", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "jwt_token_ttl": { 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "jwt_token_ttl", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.955 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "28800", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.957 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.958 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.959 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.960 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.961 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "motd": { 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "motd", 2024-06-06T07:27:06.962 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "The message of the day", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect_resolve_ip_addr": { 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.963 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.964 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.965 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8080", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:06.966 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl_server_port": { 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl_server_port", 2024-06-06T07:27:06.967 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8443", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:06.968 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "redirect", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect" 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.969 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.970 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "url_prefix": { 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url_prefix", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.971 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_monitoring": { 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_monitoring", 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.972 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "monitor device health metrics", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.973 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.974 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.975 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.976 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.977 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.978 INFO:teuthology.orchestra.run.smithi073.stdout: "mark_out_threshold": { 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mark_out_threshold", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2419200", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.979 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": { 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_name", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "device_health_metrics", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.980 INFO:teuthology.orchestra.run.smithi073.stdout: "retention_period": { 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retention_period", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15552000", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain device health metrics", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_frequency": { 2024-06-06T07:27:06.981 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_frequency", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "self_heal": { 2024-06-06T07:27:06.982 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "self_heal", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_threshold": { 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_threshold", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7257600", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "diskprediction_local", 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.986 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.987 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.988 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:06.989 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:06.990 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "predict_interval": { 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predict_interval", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.991 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "predictor_model": { 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predictor_model", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "prophetstor", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.993 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:06.994 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "influx", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": false, 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "influxdb python module not found", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "batch_size": { 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "batch_size", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5000", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T07:27:06.995 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "database": { 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "database", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "ceph", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.996 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": { 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hostname", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.997 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server hostname", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "30", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "5", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.998 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:06.999 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.000 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.001 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.002 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.003 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "password": { 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "password", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.004 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "password of InfluxDB server user", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "port": { 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "port", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8086", 2024-06-06T07:27:07.005 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server port", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.006 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "false", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "threads": { 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threads", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.007 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "32", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "username": { 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "username", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.008 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "username of InfluxDB server user", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "verify_ssl": { 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "verify_ssl", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "true", 2024-06-06T07:27:07.009 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "insights", 2024-06-06T07:27:07.010 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.011 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.012 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.013 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.014 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.015 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "iostat", 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.016 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.017 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.018 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.019 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.020 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "k8sevents", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_event_retention_days": { 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ceph_event_retention_days", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.022 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "config_check_secs": { 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_check_secs", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "10", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.023 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.024 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.025 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.026 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.027 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.028 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "localpool", 2024-06-06T07:27:07.029 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "failure_domain": { 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "failure_domain", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "host", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.030 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "failure domain for any created local pool", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.031 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.032 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.033 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.034 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.035 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": { 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_size", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.036 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default min_size for any created local pool", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "num_rep": { 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "num_rep", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.037 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default replica count for any created local pool", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": { 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_num", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.038 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "128", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "prefix": { 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prefix", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.039 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name prefix for any created local pool", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "subtree": { 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subtree", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.040 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "rack", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mds_autoscaler", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.042 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.043 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.044 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.045 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.046 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirroring", 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.047 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.048 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.049 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.050 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.051 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.052 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "nfs", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.053 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.054 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.055 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.056 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.057 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.058 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "fail_fs": { 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fail_fs", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.059 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.060 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.061 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.062 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.063 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator": { 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "rook", 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "test_orchestrator" 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.065 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Orchestrator backend", 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_perf_query", 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.066 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.067 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.068 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.069 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.070 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.071 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_support", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.072 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.073 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.074 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.075 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.076 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_autoscaler", 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.077 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.078 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.079 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.080 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.081 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.082 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "threshold": { 2024-06-06T07:27:07.083 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threshold", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1.0", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "scaling threshold", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "progress", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_pg_recovery_event": { 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_pg_recovery_event", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.085 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.086 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.087 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.088 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.089 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.090 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.091 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "max_completed_events": { 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_completed_events", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "50", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.092 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "number of past completed events to remember", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.093 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long the module is going to sleep", 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus", 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "cache": { 2024-06-06T07:27:07.094 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cache", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.095 INFO:teuthology.orchestra.run.smithi073.stdout: "exclude_perf_counters": { 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "exclude_perf_counters", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.096 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.097 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.098 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.099 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools": { 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "300", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_interval": { 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_interval", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15.0", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.104 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.105 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "9283", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.106 INFO:teuthology.orchestra.run.smithi073.stdout: "stale_cache_strategy": { 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stale_cache_strategy", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "log", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.107 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "default", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "default", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "error" 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.108 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.109 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.110 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.111 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.112 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.113 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_snap_create": { 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_snap_create", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.115 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mirror_snapshot_schedule": { 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.116 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "trash_purge_schedule": { 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "trash_purge_schedule", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.117 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "restful", 2024-06-06T07:27:07.118 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_auth": { 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_auth", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.119 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.120 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.121 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.122 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.123 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.124 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.125 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.126 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.127 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rgw", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.128 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.131 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.132 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rook", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.133 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.134 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.135 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.136 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.137 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.138 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "storage_class": { 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "storage_class", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "local", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.139 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "selftest", 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.140 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.141 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.143 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.144 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "roption1": { 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption1", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.145 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "roption2": { 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption2", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "xyz", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.146 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption1": { 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption1", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.147 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption2": { 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption2", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.148 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption3": { 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption3", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.149 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption4": { 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption4", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.150 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption5": { 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption5", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.151 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption6": { 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption6", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.152 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption7": { 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption7", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "42", 2024-06-06T07:27:07.153 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "testkey": { 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testkey", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.154 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "testlkey": { 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testlkey", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.155 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "testnewline": { 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testnewline", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.156 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_m_granularity": { 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_m_granularity", 2024-06-06T07:27:07.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "dump_on_update": { 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dump_on_update", 2024-06-06T07:27:07.158 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "dump database to debug log on update", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.159 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.160 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.161 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.162 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stats", 2024-06-06T07:27:07.164 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.165 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.167 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.168 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.169 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "status", 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.171 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.172 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.173 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.174 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telegraf", 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.175 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "address": { 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "address", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.177 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.178 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.179 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.180 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.181 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.182 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telemetry", 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_basic": { 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_basic", 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.183 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_crash": { 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_crash", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.184 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_device": { 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_device", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.185 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_ident": { 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_ident", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.186 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_perf": { 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_perf", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.187 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "contact": { 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "contact", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.188 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "description": { 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "description", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.189 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "device_url": { 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_url", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T07:27:07.190 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.191 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "24", 2024-06-06T07:27:07.192 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "8", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "last_opt_revision": { 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "last_opt_revision", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1", 2024-06-06T07:27:07.193 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard": { 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.194 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard_description": { 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard_description", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.195 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.199 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.200 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "organization": { 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "organization", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.201 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "proxy": { 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "proxy", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.202 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "url": { 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.203 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "test_orchestrator", 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.204 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.205 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.206 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.207 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.208 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.210 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.211 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.212 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.213 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.214 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_clones": { 2024-06-06T07:27:07.215 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_clones", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "periodic_async_work": { 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "periodic_async_work", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Periodically check for async work", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.219 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_delay": { 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_delay", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_no_wait": { 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.221 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "discovery_interval": { 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "discovery_interval", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "100", 2024-06-06T07:27:07.222 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "identifier": { 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "identifier", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.223 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.224 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.225 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.226 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.228 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.229 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_host": { 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_host", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.230 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_port": { 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_port", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10051", 2024-06-06T07:27:07.231 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_sender": { 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_sender", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T07:27:07.232 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "modules": [ 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-06-06T07:27:07.233 INFO:teuthology.orchestra.run.smithi073.stdout: "dashboard", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "iostat", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "nfs", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "restful", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "snap_schedule" 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "available_modules": [ 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "alerts", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.234 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How frequently to reexamine health status", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.236 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.237 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.238 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.239 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.240 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_destination": { 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_destination", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email address to send alerts to", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_from_name": { 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_from_name", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "Ceph", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Email From: name", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_host": { 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_host", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP server", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_password": { 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_password", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Password to authenticate with", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_port": { 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_port", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "465", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP port", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_sender": { 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_sender", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SMTP envelope sender", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_ssl": { 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_ssl", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "smtp_user": { 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "smtp_user", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "User to authenticate as", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "balancer", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "active": { 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "active", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically balance PGs across cluster", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_time": { 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_time", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0000", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "beginning time of day to automatically balance", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "begin_weekday": { 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "begin_weekday", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_max_iterations": { 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_max_iterations", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "25", 2024-06-06T07:27:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "250", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_metrics": { 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_metrics", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "pgs,objects,bytes", 2024-06-06T07:27:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_compat_step": { 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crush_compat_step", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.5", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0.001", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "0.999", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "aggressiveness of optimization", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "end_time": { 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_time", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2359", 2024-06-06T07:27:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ending time of day to automatically balance", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "end_weekday": { 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "end_weekday", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "0", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "6", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "min_score": { 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_score", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "upmap", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "crush-compat", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "none", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "read", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap-read" 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Balancer mode", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_ids": { 2024-06-06T07:27:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_ids", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_deviation": { 2024-06-06T07:27:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_deviation", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "upmap_max_optimizations": { 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "upmap_max_optimizations", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_down_multiplier": { 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_down_multiplier", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_refresh_rate": { 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_refresh_rate", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "20", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "agent_starting_port": { 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "agent_starting_port", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4721", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_ptrace": { 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_ptrace", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_interval": { 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_interval", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "autotune_memory_target_ratio": { 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "autotune_memory_target_ratio", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0.7", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_log_destination": { 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephadm_log_destination", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "file", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "file,syslog", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "syslog" 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "cgroups_split": { 2024-06-06T07:27:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cgroups_split", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "config_checks_enabled": { 2024-06-06T07:27:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_checks_enabled", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "config_dashboard": { 2024-06-06T07:27:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_dashboard", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_alertmanager": { 2024-06-06T07:27:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_alertmanager", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_base": { 2024-06-06T07:27:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_base", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Container image name, without the tag", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_elasticsearch": { 2024-06-06T07:27:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_elasticsearch", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "elasticsearch container image", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_grafana": { 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_grafana", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_haproxy": { 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_haproxy", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "HAproxy container image", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_agent": { 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_agent", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger agent container image", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_collector": { 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_collector", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger collector container image", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_jaeger_query": { 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_jaeger_query", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Jaeger query container image", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_keepalived": { 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_keepalived", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Keepalived container image", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_loki": { 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_loki", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Loki container image", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_node_exporter": { 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_node_exporter", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_nvmeof": { 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_nvmeof", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Nvme-of container image", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_prometheus": { 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_prometheus", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Prometheus container image", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_promtail": { 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_promtail", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Promtail container image", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_samba": { 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_samba", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Samba/SMB container image", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image_snmp_gateway": { 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_image_snmp_gateway", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "SNMP Gateway container image", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "container_init": { 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "container_init", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Run podman/docker with `--init`", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "daemon_cache_timeout": { 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "daemon_cache_timeout", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "default_cephadm_command_timeout": { 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_cephadm_command_timeout", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "900", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "default_registry": { 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "default_registry", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "docker.io", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.299 INFO:teuthology.orchestra.run.smithi073.stdout: "device_cache_timeout": { 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_cache_timeout", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1800", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache device inventory", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.300 INFO:teuthology.orchestra.run.smithi073.stdout: "device_enhanced_scan": { 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_enhanced_scan", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.301 INFO:teuthology.orchestra.run.smithi073.stdout: "facts_cache_timeout": { 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "facts_cache_timeout", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "seconds to cache host facts data", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.302 INFO:teuthology.orchestra.run.smithi073.stdout: "grafana_dashboards_path": { 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "grafana_dashboards_path", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "host_check_interval": { 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "host_check_interval", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to perform a host check", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "hw_monitoring": { 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hw_monitoring", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "inventory_list_all": { 2024-06-06T07:27:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "inventory_list_all", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "log_refresh_metadata": { 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_refresh_metadata", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "*", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "max_count_per_host": { 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_count_per_host", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of daemons per service per host", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "max_osd_draining_count": { 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_osd_draining_count", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-06T07:27:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "migration_current": { 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "migration_current", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "internal - do not modify", 2024-06-06T07:27:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "mode": { 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mode", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "root", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm-package", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "root" 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "mode for remote execution of cephadm", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "oob_default_addr": { 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "oob_default_addr", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "169.254.1.1", 2024-06-06T07:27:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "prometheus_alerts_path": { 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus_alerts_path", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-06T07:27:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_insecure": { 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_insecure", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_password": { 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_password", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_url": { 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_url", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "registry_username": { 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "registry_username", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "secure_monitoring_stack": { 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "secure_monitoring_stack", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "service_discovery_port": { 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "service_discovery_port", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8765", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "cephadm service discovery port", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "ssh_config_file": { 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssh_config_file", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "use_agent": { 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_agent", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "use_repo_digest": { 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "use_repo_digest", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_failed_host_check": { 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_failed_host_check", 2024-06-06T07:27:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if the host check fails", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_daemons": { 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_daemons", 2024-06-06T07:27:07.331 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_on_stray_hosts": { 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_on_stray_hosts", 2024-06-06T07:27:07.332 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.333 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crash", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.334 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.335 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.336 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.337 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.338 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "retain_interval": { 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retain_interval", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "31536000", 2024-06-06T07:27:07.339 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_recent_interval": { 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_recent_interval", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1209600", 2024-06-06T07:27:07.340 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dashboard", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.341 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.342 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_HOST": { 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.343 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.344 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_ENABLED": { 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_ENABLED", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.345 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.346 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "ENABLE_BROWSABLE_API": { 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.347 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.348 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.349 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.350 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.351 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.352 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.353 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.354 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.355 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_PASSWORD": { 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.356 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.357 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.358 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_URL": { 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_URL", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.362 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_API_USERNAME": { 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.363 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.364 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.365 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.366 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.367 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_HOST": { 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.368 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.369 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.370 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.371 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.372 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.373 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.374 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.375 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.376 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_ENABLED": { 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.377 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.378 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.379 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.380 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.381 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "45", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ACCESS_KEY": { 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.382 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.383 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "admin", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SECRET_KEY": { 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.384 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "RGW_API_SSL_VERIFY": { 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-06T07:27:07.385 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-06T07:27:07.386 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-06T07:27:07.387 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-06T07:27:07.388 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.389 INFO:teuthology.orchestra.run.smithi073.stdout: "cross_origin_url": { 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cross_origin_url", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "crt_file": { 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "crt_file", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.391 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "debug": { 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "debug", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Enable/disable debug options", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "jwt_token_ttl": { 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "jwt_token_ttl", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "28800", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.395 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.396 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.398 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.399 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "motd": { 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "motd", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.400 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "The message of the day", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect_resolve_ip_addr": { 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "redirect_resolve_ip_addr", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.401 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.402 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8080", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.403 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.404 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl_server_port": { 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl_server_port", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8443", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.405 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "redirect", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.406 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "redirect" 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.407 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "url_prefix": { 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url_prefix", 2024-06-06T07:27:07.408 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_monitoring": { 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_monitoring", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.410 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "monitor device health metrics", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.411 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.412 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.413 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.414 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.415 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "mark_out_threshold": { 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mark_out_threshold", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "2419200", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.416 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "pool_name": { 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pool_name", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "device_health_metrics", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.417 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "retention_period": { 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "retention_period", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15552000", 2024-06-06T07:27:07.418 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long to retain device health metrics", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_frequency": { 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_frequency", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:07.419 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "self_heal": { 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "self_heal", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.420 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:07.421 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how frequently to wake up and check device health", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "warn_threshold": { 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "warn_threshold", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7257600", 2024-06-06T07:27:07.422 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "diskprediction_local", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.423 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.424 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.425 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.426 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "predict_interval": { 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predict_interval", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "86400", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.429 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "predictor_model": { 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "predictor_model", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "prophetstor", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.430 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "600", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.431 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "influx", 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": false, 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "influxdb python module not found", 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "batch_size": { 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "batch_size", 2024-06-06T07:27:07.432 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5000", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "database": { 2024-06-06T07:27:07.433 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "database", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "ceph", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "hostname": { 2024-06-06T07:27:07.434 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "hostname", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server hostname", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.435 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "30", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "5", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.436 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.437 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.438 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.439 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.440 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.441 INFO:teuthology.orchestra.run.smithi073.stdout: "password": { 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "password", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "password of InfluxDB server user", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "port": { 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "port", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "8086", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "InfluxDB server port", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "ssl": { 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ssl", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "false", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "threads": { 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threads", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "32", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "username": { 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "username", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "username of InfluxDB server user", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.446 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "verify_ssl": { 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "verify_ssl", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "true", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.447 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "insights", 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.448 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.450 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.451 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.452 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "iostat", 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.453 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.454 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.455 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.456 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.457 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.458 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "k8sevents", 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_event_retention_days": { 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "ceph_event_retention_days", 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.459 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "7", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "config_check_secs": { 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "config_check_secs", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.460 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "10", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.461 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.462 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.463 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.464 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.465 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "localpool", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "failure_domain": { 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "failure_domain", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "host", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.467 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "failure domain for any created local pool", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.468 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.469 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.470 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.471 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.472 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "min_size": { 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "min_size", 2024-06-06T07:27:07.473 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default min_size for any created local pool", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "num_rep": { 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "num_rep", 2024-06-06T07:27:07.474 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default replica count for any created local pool", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_num": { 2024-06-06T07:27:07.475 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_num", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "128", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "default pg_num for any created local pool", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "prefix": { 2024-06-06T07:27:07.476 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prefix", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "name prefix for any created local pool", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "subtree": { 2024-06-06T07:27:07.477 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "subtree", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "rack", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mds_autoscaler", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.479 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.480 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.481 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.482 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.483 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirroring", 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.484 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.485 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.486 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.487 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.488 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.489 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "nfs", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.490 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.491 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.492 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.493 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.494 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.495 INFO:teuthology.orchestra.run.smithi073.stdout: "fail_fs": { 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "fail_fs", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.496 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.497 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.498 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.499 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.500 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator": { 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "orchestrator", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "rook", 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: "test_orchestrator" 2024-06-06T07:27:07.502 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Orchestrator backend", 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_perf_query", 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.503 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.504 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.505 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.506 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.507 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "osd_support", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.509 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.510 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.511 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.512 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.513 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "pg_autoscaler", 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.514 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.515 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.516 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.517 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.518 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.519 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.520 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "threshold": { 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "threshold", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "3.0", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1.0", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "scaling threshold", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-06T07:27:07.521 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "progress", 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_pg_recovery_event": { 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_pg_recovery_event", 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.522 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.523 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.525 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.526 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.527 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "max_completed_events": { 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_completed_events", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.529 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "50", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "number of past completed events to remember", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "sleep_interval": { 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "sleep_interval", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.530 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "5", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "how long the module is going to sleep", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "prometheus", 2024-06-06T07:27:07.531 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "cache": { 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cache", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.532 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "exclude_perf_counters": { 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "exclude_perf_counters", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-06T07:27:07.533 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.534 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.535 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.536 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.537 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.538 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools": { 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.539 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "300", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.540 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "scrape_interval": { 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "scrape_interval", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15.0", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.541 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "::", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.542 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "9283", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.543 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "stale_cache_strategy": { 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stale_cache_strategy", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "log", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.544 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_behaviour": { 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_behaviour", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "default", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "default", 2024-06-06T07:27:07.545 INFO:teuthology.orchestra.run.smithi073.stdout: "error" 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "standby_error_status_code": { 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "standby_error_status_code", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "500", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "400", 2024-06-06T07:27:07.546 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "599", 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.547 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.548 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.549 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.550 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.551 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_snap_create": { 2024-06-06T07:27:07.552 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_snap_create", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "mirror_snapshot_schedule": { 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mirror_snapshot_schedule", 2024-06-06T07:27:07.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "trash_purge_schedule": { 2024-06-06T07:27:07.554 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "trash_purge_schedule", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "restful", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_auth": { 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enable_auth", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.556 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "key_file": { 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "key_file", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.557 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.558 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.560 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.561 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.562 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "server_addr": { 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_addr", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.563 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "server_port": { 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "server_port", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.564 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rgw", 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.565 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.566 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.567 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.568 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.569 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rook", 2024-06-06T07:27:07.570 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.571 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.573 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.574 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.575 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "storage_class": { 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "storage_class", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "local", 2024-06-06T07:27:07.576 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.577 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "selftest", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.578 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.579 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.580 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.581 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.582 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "roption1": { 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption1", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.583 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "roption2": { 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "roption2", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.584 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "xyz", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption1": { 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption1", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.585 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.586 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption2": { 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption2", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.587 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption3": { 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption3", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "float", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.588 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption4": { 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption4", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.589 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption5": { 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption5", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.590 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption6": { 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption6", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.591 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "rwoption7": { 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rwoption7", 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.592 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "1", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "42", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "testkey": { 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testkey", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.593 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "testlkey": { 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testlkey", 2024-06-06T07:27:07.594 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "testnewline": { 2024-06-06T07:27:07.595 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "testnewline", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.596 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "allow_m_granularity": { 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "allow_m_granularity", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.597 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "allow minute scheduled snapshots", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "dump_on_update": { 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "dump_on_update", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.598 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "dump database to debug log on update", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.599 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.600 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.601 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.602 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.603 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "stats", 2024-06-06T07:27:07.604 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.605 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.606 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.607 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.608 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.609 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "status", 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.610 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.611 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.612 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.613 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.614 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.615 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telegraf", 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "address": { 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "address", 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.616 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.617 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "15", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.620 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.621 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.622 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.626 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "telemetry", 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.627 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_basic": { 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_basic", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_crash": { 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_crash", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.629 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_device": { 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_device", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.630 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_ident": { 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_ident", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.631 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "channel_perf": { 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "channel_perf", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.632 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "contact": { 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "contact", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.633 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "description": { 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "description", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.634 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "device_url": { 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "device_url", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.635 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "enabled": { 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "enabled", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.636 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "24", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "8", 2024-06-06T07:27:07.637 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "last_opt_revision": { 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "last_opt_revision", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "1", 2024-06-06T07:27:07.638 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard": { 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.639 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "leaderboard_description": { 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "leaderboard_description", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.640 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.641 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.642 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.643 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.644 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.645 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "organization": { 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "organization", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.646 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "proxy": { 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "proxy", 2024-06-06T07:27:07.647 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "url": { 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "url", 2024-06-06T07:27:07.648 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "test_orchestrator", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.650 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.651 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.652 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.653 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.654 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.655 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.656 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.657 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.658 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.659 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.660 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "max_concurrent_clones": { 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "max_concurrent_clones", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.661 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "4", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "periodic_async_work": { 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "periodic_async_work", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.662 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Periodically check for async work", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_delay": { 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_delay", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.663 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "0", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "snapshot_clone_no_wait": { 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snapshot_clone_no_wait", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.664 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "True", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "can_run": true, 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "error_string": "", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "module_options": { 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "discovery_interval": { 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "discovery_interval", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "uint", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "100", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.666 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "identifier": { 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "identifier", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.667 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "interval": { 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "interval", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "secs", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "60", 2024-06-06T07:27:07.668 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "log_level": { 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_level", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.669 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.670 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster": { 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.671 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_cluster_level": { 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_cluster_level", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "info", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [ 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "critical", 2024-06-06T07:27:07.672 INFO:teuthology.orchestra.run.smithi073.stdout: "debug", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "error", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "info", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "warning" 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": { 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "log_to_file", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "bool", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.673 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 1, 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "False", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_host": { 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_host", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.674 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_port": { 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_port", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "int", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.675 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "10051", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "zabbix_sender": { 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "zabbix_sender", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "str", 2024-06-06T07:27:07.676 INFO:teuthology.orchestra.run.smithi073.stdout: "level": "advanced", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flags": 0, 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "min": "", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "max": "", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "enum_allowed": [], 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "desc": "", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "long_desc": "", 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tags": [], 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "see_also": [] 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.677 INFO:teuthology.orchestra.run.smithi073.stdout: "services": { 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "dashboard": "https://172.21.15.73:8443/" 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "always_on_modules": { 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "octopus": [ 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-06-06T07:27:07.678 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "pacific": [ 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-06-06T07:27:07.679 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "quincy": [ 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "reef": [ 2024-06-06T07:27:07.680 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "squid": [ 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "balancer", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "crash", 2024-06-06T07:27:07.681 INFO:teuthology.orchestra.run.smithi073.stdout: "devicehealth", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "orchestrator", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "pg_autoscaler", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "progress", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_support", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "status", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "telemetry", 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "volumes" 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "last_failure_osd_epoch": 101, 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "active_clients": [ 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.682 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "devicehealth", 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 3654032405 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "libcephsqlite", 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 4205651725 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "rbd_support", 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2271649775 2024-06-06T07:27:07.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 1850794971 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "snap_schedule", 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 644584542 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.686 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "volumes", 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: "addrvec": [ 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: "type": "v2", 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: "addr": "172.21.15.73:0", 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: "nonce": 2535268557 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-06-06T07:27:07.687 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:06 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:06 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1067778900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:06 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/2446540231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:06 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/1340306521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:06 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:06 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1067778900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.689 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:06 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/2446540231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.690 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:06 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/1340306521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T07:27:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 msgr2=0x7ff3800a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:07.690 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 0x7ff3800a51c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff390005fd0 tx=0x7ff3900094d0 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 msgr2=0x7ff39419caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff38c009fd0 tx=0x7ff38c00b260 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 shutdown_connections 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff394107990 0x7ff39419caf0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7ff394102340 0x7ff394198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7ff3800a2d00 0x7ff3800a51c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 --2- 172.21.15.73:0/1340306521 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7ff394101980 0x7ff394198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 >> 172.21.15.73:0/1340306521 conn(0x7ff3940fd410 msgr2=0x7ff3940fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 shutdown_connections 2024-06-06T07:27:07.691 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:06.275+0000 7ff39c363640 1 -- 172.21.15.73:0/1340306521 wait complete. 2024-06-06T07:27:07.698 DEBUG:tasks.check_counter:Getting stats from a 2024-06-06T07:27:07.698 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-06T07:27:07.699 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:06 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.699 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:06 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1067778900' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.699 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:06 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/2446540231' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-06-06T07:27:07.699 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:06 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/1340306521' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-06T07:27:07.702 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:07 smithi136 ceph-mon[35901]: pgmap v821: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 404 op/s 2024-06-06T07:27:07.703 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:07 smithi136 ceph-mon[35901]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.725 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:07 smithi073 ceph-mon[32303]: pgmap v821: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 404 op/s 2024-06-06T07:27:07.725 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:07 smithi073 ceph-mon[32303]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:07 smithi170 ceph-mon[36108]: pgmap v821: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.3 MiB/s wr, 404 op/s 2024-06-06T07:27:07.748 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:07 smithi170 ceph-mon[36108]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-06T07:27:07.982 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:09.696 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:09 smithi073 ceph-mon[32303]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:27:09.744 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 1396, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 1396, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 117799251, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 443167, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.186939868, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.026053708, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.121320442, 2024-06-06T07:27:09.745 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.030518858, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1395, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.035685309, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000025580 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T07:27:09.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2238, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2166, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 163914958, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 826328, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.298163733, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.060822929, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.188762161, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.047635793, 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:09.747 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2165, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.058399628, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000026974 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1992400, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 307664 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:09.748 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3533, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 3899, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 156225334, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1919338, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.394398546, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.116727477, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.162924658, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.090897803, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3896, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.113236685, 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000029064 2024-06-06T07:27:09.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 796480 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.750 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 11103, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.229529097, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000020672 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.751 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-06-06T07:27:09.752 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 15314, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 1905, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 13409, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 12593, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 4, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-06-06T07:27:09.753 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:09.754 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T07:27:09.755 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-06-06T07:27:09.756 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:09.757 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 113239, 2024-06-06T07:27:09.758 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 134, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 11, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 35014044, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 438331218, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 438331157, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.759 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 113239 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 1624, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 16126, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 14502, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 63, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 131, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 68, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 15316, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 16568, 2024-06-06T07:27:09.760 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 1252, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 74244, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215420 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-06-06T07:27:09.761 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.762 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.763 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.764 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.765 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.766 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-06-06T07:27:09.767 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.769 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:09.770 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.771 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-06-06T07:27:09.773 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 3985, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:09.774 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:09.775 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:09.776 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 243345, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 88, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:09.777 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9944, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 192, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 14668571, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 69609, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-06-06T07:27:09.778 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 6554, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 6554, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6554, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5.123885201, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000781795 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6554, 2024-06-06T07:27:09.779 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6555 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 6554, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 6554, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3512, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 3040, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-06-06T07:27:09.780 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-06-06T07:27:09.781 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-06-06T07:27:09.782 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 147, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 49, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 1, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-06-06T07:27:09.783 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-06-06T07:27:09.784 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 7163, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 437261193, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:09.785 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 7163, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 7163, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 437261193, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:09.786 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6554, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 434136919, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6554, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 434136919, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6554, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 6554, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6554, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 6554, 2024-06-06T07:27:09.788 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:09.789 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x557c1afb46e0": { 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:09.790 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:09.791 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:27:09.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:09 smithi136 ceph-mon[35901]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:27:09.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:09 smithi170 ceph-mon[36108]: pgmap v822: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 249 op/s 2024-06-06T07:27:10.582 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-06T07:27:10.582 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-06T07:27:10.582 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-06-06T07:27:10.582 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:10.582 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:10.582 DEBUG:tasks.check_counter:Getting stats from d 2024-06-06T07:27:10.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-06T07:27:10.877 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:11.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:11 smithi073 ceph-mon[32303]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:11.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:11 smithi136 ceph-mon[35901]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:11.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:11 smithi170 ceph-mon[36108]: pgmap v823: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2767, 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2695, 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 69791684, 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 996302, 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 7, 2024-06-06T07:27:12.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 3, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.302362697, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.100029432, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.124682917, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.070086616, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2694, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.092596650, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000034371 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1994368, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 311648 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2663, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 3031, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 105542790, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1642615, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 9, 2024-06-06T07:27:12.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.258331117, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.060773387, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.123284132, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.058876540, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3029, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.083687601, 2024-06-06T07:27:12.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000027628 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 797632 2024-06-06T07:27:12.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 1004, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 1006, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 148507948, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 319284, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.180136246, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.019355242, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.128700701, 2024-06-06T07:27:12.644 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.025827796, 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1004, 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.032119378, 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000031991 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:12.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.646 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-06-06T07:27:12.647 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:12.647 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:12.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9834, 2024-06-06T07:27:12.647 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.228761098, 2024-06-06T07:27:12.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000023262 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-06-06T07:27:12.648 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-06-06T07:27:12.649 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 14846, 2024-06-06T07:27:12.650 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 3097, 2024-06-06T07:27:12.651 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 11749, 2024-06-06T07:27:12.651 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-06-06T07:27:12.651 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 10421, 2024-06-06T07:27:12.651 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 1, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-06-06T07:27:12.652 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:12.653 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-06-06T07:27:12.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-06-06T07:27:12.655 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:12.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 149067, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-06-06T07:27:12.657 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 141, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 14, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 68660828, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 324412479, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 324404715, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.658 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 149067 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 766, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 17444, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 16678, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 35, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 165, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 130, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 14846, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 17932, 2024-06-06T07:27:12.659 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 3086, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 77900, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215420 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-06-06T07:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.661 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.663 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.664 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-06-06T07:27:12.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.665 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.665 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-06-06T07:27:12.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:12.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 2, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 1, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 2, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 3987, 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:12.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:12.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:12.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 241794, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 85, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:12.676 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9944, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 192, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 12338951, 2024-06-06T07:27:12.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 64525, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 5814, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 5814, 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-06-06T07:27:12.678 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5814, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 4.950889070, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000851546 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5814, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 5815 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 5814, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 5814, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-06-06T07:27:12.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3335, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 2477, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:12.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-06-06T07:27:12.681 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-06-06T07:27:12.682 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 147, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 49, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 13, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 2, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-06-06T07:27:12.683 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-06-06T07:27:12.684 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6430, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 323229678, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6430, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6430, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 323229678, 2024-06-06T07:27:12.685 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:12.686 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5814, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 320224084, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5814, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 320224084, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:12.687 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 5814, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 5814, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 5814, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 5814, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.688 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.689 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55f6c704e6e0": { 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:12.690 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:12.691 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:27:13.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:13 smithi073 ceph-mon[32303]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:13.525 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:13 smithi073 ceph-mon[32303]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T07:27:13.584 INFO:tasks.check_counter:Daemon mds.d mds.exported=0 2024-06-06T07:27:13.584 INFO:tasks.check_counter:Daemon mds.d mds.imported=0 2024-06-06T07:27:13.585 INFO:tasks.check_counter:Daemon mds.d mds.root_rsnaps=0 2024-06-06T07:27:13.585 INFO:tasks.check_counter:Daemon mds.d mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:13.585 INFO:tasks.check_counter:Daemon mds.d mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:13.585 DEBUG:tasks.check_counter:Getting stats from g 2024-06-06T07:27:13.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-06T07:27:13.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:13 smithi136 ceph-mon[35901]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:13.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:13 smithi136 ceph-mon[35901]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T07:27:13.835 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:13 smithi170 ceph-mon[36108]: pgmap v824: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.0 MiB/s wr, 229 op/s 2024-06-06T07:27:13.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:13 smithi170 ceph-mon[36108]: osdmap e148: 12 total, 12 up, 12 in 2024-06-06T07:27:15.592 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:15 smithi073 ceph-mon[32303]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-06-06T07:27:15.634 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:27:15.634 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:15.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3807, 2024-06-06T07:27:15.634 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 3807, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 132840902, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1209803, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.374716318, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.087873411, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.167704705, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.095473067, 2024-06-06T07:27:15.635 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3806, 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.097083300, 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000025507 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 989, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 917, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 52422168, 2024-06-06T07:27:15.637 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 430032, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.137944400, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.033731011, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.090109171, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.019096941, 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:15.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 916, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.029240726, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000031922 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.639 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1993728, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 309760 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 1930, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2298, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 113729328, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1498770, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 10, 2024-06-06T07:27:15.640 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.247853057, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.057635698, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.120566960, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.054205923, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2295, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.061869556, 2024-06-06T07:27:15.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000026958 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-06T07:27:15.642 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 885984 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:15.643 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 10218, 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.218459495, 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000021379 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.644 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-06-06T07:27:15.646 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 6072, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 735, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 5337, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 5028, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-06-06T07:27:15.647 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 1, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:15.648 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-06-06T07:27:15.649 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:15.650 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-06-06T07:27:15.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:15.652 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:15.653 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-06-06T07:27:15.654 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 40597, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 83, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 5, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 299494571, 2024-06-06T07:27:15.655 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 299479694, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 40597 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 2393, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 6384, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 3991, 2024-06-06T07:27:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 83, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 136, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 53, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 6072, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 7038, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 966, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 65464, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215420 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-06-06T07:27:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-06-06T07:27:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 2, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 3, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-06-06T07:27:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 3380, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:15.674 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:15.675 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:15.676 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:15.677 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 241793, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 85, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:15.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 9944, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 192, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 9705994, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 35982, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-06-06T07:27:15.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 6108, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 6108, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6108, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 4.795892606, 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000785182 2024-06-06T07:27:15.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6108, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 6109 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 6108, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 6108, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 3471, 2024-06-06T07:27:15.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 2635, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:15.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-06-06T07:27:15.683 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-06-06T07:27:15.684 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 148, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 50, 2024-06-06T07:27:15.685 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 11, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 14, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 3, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-06-06T07:27:15.686 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-06-06T07:27:15.687 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6722, 2024-06-06T07:27:15.688 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 298353493, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6722, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6722, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 298353493, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6108, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 295300272, 2024-06-06T07:27:15.690 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6108, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 295300272, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:15.691 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 6108, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 6108, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6108, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 6108, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:15.693 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x5590db5be6e0": { 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:15.694 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:15.695 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:15.696 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:27:15.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:15 smithi136 ceph-mon[35901]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-06-06T07:27:15.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:15 smithi170 ceph-mon[36108]: pgmap v826: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 66 KiB/s wr, 31 op/s 2024-06-06T07:27:16.452 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-06T07:27:16.452 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-06T07:27:16.452 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-06-06T07:27:16.452 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:16.452 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:16.452 DEBUG:tasks.check_counter:Getting stats from j 2024-06-06T07:27:16.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-06T07:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:17.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:17 smithi073 ceph-mon[32303]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-06-06T07:27:17.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:17 smithi136 ceph-mon[35901]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-06-06T07:27:17.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:17 smithi170 ceph-mon[36108]: pgmap v827: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 64 KiB/s wr, 30 op/s 2024-06-06T07:27:18.511 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-06-06T07:27:18.511 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 0, 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2, 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 0, 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 0, 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 4, 2024-06-06T07:27:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 0, 2024-06-06T07:27:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.001165906, 2024-06-06T07:27:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.000000000, 2024-06-06T07:27:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.000296079, 2024-06-06T07:27:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T07:27:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T07:27:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T07:27:18.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.516 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:18.516 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 505, 2024-06-06T07:27:18.516 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 493, 2024-06-06T07:27:18.516 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 1526080, 2024-06-06T07:27:18.517 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 304576, 2024-06-06T07:27:18.517 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 2, 2024-06-06T07:27:18.517 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 1, 2024-06-06T07:27:18.517 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.068482123, 2024-06-06T07:27:18.517 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.025551491, 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.037758924, 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.009690690, 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 493, 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.017296714, 2024-06-06T07:27:18.518 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000035084 2024-06-06T07:27:18.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.519 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:18.519 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:18.519 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:18.519 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:18.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.520 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 1526080, 2024-06-06T07:27:18.520 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 304576 2024-06-06T07:27:18.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.520 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3, 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 376, 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 480, 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 299888, 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:18.521 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 3, 2024-06-06T07:27:18.522 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.025371656, 2024-06-06T07:27:18.522 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.019595704, 2024-06-06T07:27:18.522 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.001971067, 2024-06-06T07:27:18.522 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T07:27:18.522 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 373, 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.020991207, 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000056276 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:18.523 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 299888 2024-06-06T07:27:18.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 0, 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-06-06T07:27:18.525 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:18.526 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:18.526 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:18.526 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:18.526 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:18.526 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:18.527 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:18.527 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:18.527 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:18.527 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:18.527 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:18.528 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:18.531 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 148732, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 31, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:18.532 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-06-06T07:27:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-06-06T07:27:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:18.533 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:18.534 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-06-06T07:27:18.535 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 928, 2024-06-06T07:27:18.535 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 2, 2024-06-06T07:27:18.535 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:18.535 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-06-06T07:27:18.535 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-06-06T07:27:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-06-06T07:27:18.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 504, 2024-06-06T07:27:18.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 1452343, 2024-06-06T07:27:18.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:18.537 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 504, 2024-06-06T07:27:18.537 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-06-06T07:27:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-06-06T07:27:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 504, 2024-06-06T07:27:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1452343, 2024-06-06T07:27:18.538 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-06-06T07:27:18.539 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-06-06T07:27:18.539 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-06-06T07:27:18.539 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-06-06T07:27:18.539 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:18.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-06-06T07:27:18.540 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-06-06T07:27:19.239 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-06T07:27:19.239 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-06T07:27:19.239 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-06-06T07:27:19.239 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-06-06T07:27:19.239 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-06-06T07:27:19.239 DEBUG:tasks.check_counter:Getting stats from b 2024-06-06T07:27:19.240 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-06T07:27:19.524 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:19 smithi073 ceph-mon[32303]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-06-06T07:27:19.642 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:27:19.764 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:19 smithi136 ceph-mon[35901]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-06-06T07:27:19.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:19 smithi170 ceph-mon[36108]: pgmap v828: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-06-06T07:27:21.686 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:21 smithi136 ceph-mon[35901]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:21.687 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 139832, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 113109, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 47579256, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 142463710, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 13, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 6.267568514, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 2.316747860, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 3.966201089, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.766075271, 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:21.688 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 113107, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.315420436, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000020471 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 204712, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.026790863, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000130 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.689 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 605689, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 686565, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 183390914, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 565903350, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 14, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 6, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 30.548590390, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 12.401659415, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 21.483370650, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.926672256, 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:21.690 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 686564, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 14.247130196, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000020751 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1211981, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.184280538, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000152 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2017968, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 316224 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 55827, 2024-06-06T07:27:21.691 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 56028, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 17226215, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 188794617, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 16, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 4.046512075, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 1.724571213, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.715061887, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.767014981, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 56026, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.595711082, 2024-06-06T07:27:21.692 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000028481 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 59476, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.009510736, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000159 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 4192, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 801312 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.693 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 77774, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 46.686161854, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000600279 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 30005, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.334927423, 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000077817 2024-06-06T07:27:21.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 460006, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 450365, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 450365, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 361.492727414, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000802666 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 9641, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 21, 2024-06-06T07:27:21.695 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 43, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 4, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 1805, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 353, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 1314, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 138, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 1296, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 1650, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 1, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 1, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 6, 2024-06-06T07:27:21.696 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 897283, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 847093, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 1714, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 140, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 4, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 5, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 1229, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 43, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 1220, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 43, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 1046, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:21.697 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:21.698 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 2790, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 40814, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 57068, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 66817, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 18624, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 1102, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 50432 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 9, 2024-06-06T07:27:21.699 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 57, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 40, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 166, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 202, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 20, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 13404, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 13384, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:21.700 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 44, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 4, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-06T07:27:21.701 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 113247, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 29879, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 24581, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 88666, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 5298, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 165, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 37, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 31, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 134, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 8, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 6, 2024-06-06T07:27:21.702 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 35014044, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 438428327, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 38966, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 128.620895095, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003300849 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 1525, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 16272, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 14747, 2024-06-06T07:27:21.703 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 102, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 198, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 96, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 1882, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 19579, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 17697, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 1, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 13562, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 13561, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 84692, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223612 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 530172, 2024-06-06T07:27:21.704 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 8004, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 460006, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 1244, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 30323, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13206, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 28.398068218, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002150391 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 368155, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 131.041390731, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000355940 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:21.705 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13903, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 56.931861927, 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004094933 2024-06-06T07:27:21.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.707 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:21.708 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.709 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.710 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 9954, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6.481390592, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000651134 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 27759, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 61.231618167, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.002205829 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3492, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 35.605199155, 2024-06-06T07:27:21.711 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.010196219 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-06T07:27:21.712 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 20, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.143628097, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.007181404 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 492, 2024-06-06T07:27:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.206872708, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000420472 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-06T07:27:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13384, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 41.452697819, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003097183 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 3, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 2, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 49072, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 49072, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1560, 2024-06-06T07:27:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 465079, 2024-06-06T07:27:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 152, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 186, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 5234184, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 23126, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 72586, 2024-06-06T07:27:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 437049471, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 28, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 72586, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 72586, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 282.144885728, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003887042 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 72586, 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 75138 2024-06-06T07:27:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 72586, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 2077, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 70509, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 21, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 2056, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 41022, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 1164, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-06T07:27:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 4, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 26890, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 10, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 42, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-06T07:27:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 148, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 50, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 2627, 2024-06-06T07:27:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 42, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 1260 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-06T07:27:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 10, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 5, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 26768, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 13384, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 801344, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 176039773, 2024-06-06T07:27:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 801344, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 801344, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 176039773, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 72586, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 438518959, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 72586, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 438518959, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 72586, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 72586, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 72586, 2024-06-06T07:27:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 72586, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 13384, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1469824, 2024-06-06T07:27:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 13384, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 2056, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1469824, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55c057458500": { 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 113247, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 434234023, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 113247, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 38966, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 434234023, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:21.736 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-06T07:27:21.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:21 smithi073 ceph-mon[32303]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:21.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:21 smithi170 ceph-mon[36108]: pgmap v829: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:22.498 INFO:tasks.check_counter:Daemon mds.b mds.exported=43 2024-06-06T07:27:22.499 INFO:tasks.check_counter:Daemon mds.b mds.imported=43 2024-06-06T07:27:22.499 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-06-06T07:27:22.499 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:22.499 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:22.499 DEBUG:tasks.check_counter:Getting stats from e 2024-06-06T07:27:22.499 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-06T07:27:22.745 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:27:22.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:22 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:27:22.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:22 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:27:22.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:22 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-06T07:27:23.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:23 smithi073 ceph-mon[32303]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:23.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:23 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:27:23.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:23 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:27:23.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:23 smithi073 ceph-mon[32303]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:27:23.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:23 smithi136 ceph-mon[35901]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:23.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:23 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:27:23.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:23 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:27:23.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:23 smithi136 ceph-mon[35901]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:27:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:23 smithi170 ceph-mon[36108]: pgmap v830: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 19 op/s 2024-06-06T07:27:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:23 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-06T07:27:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:23 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-06T07:27:23.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:23 smithi170 ceph-mon[36108]: from='mgr.15282 172.21.15.73:0/1328263495' entity='mgr.x' 2024-06-06T07:27:24.444 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-06T07:27:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 743, 2024-06-06T07:27:24.444 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 728, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 21198311, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 259102, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 2, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.068732178, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.019815222, 2024-06-06T07:27:24.445 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.028944648, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.016295697, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 727, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.019589712, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000026945 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.446 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 294960, 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 67344 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:24.447 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 584, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 663, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 54674838, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 409355, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.072377275, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.012331634, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.043920746, 2024-06-06T07:27:24.448 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.012481934, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 661, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.020702325, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000031319 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T07:27:24.449 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 223904 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 355, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 355, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 30927305, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 112082, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.042580724, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.006374051, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.026705686, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.007500718, 2024-06-06T07:27:24.450 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 354, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.012614798, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000035635 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.451 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2638, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.078529669, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029768 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.452 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.453 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 3686, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 2125, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 1561, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-06-06T07:27:24.454 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 650, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 2, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-06T07:27:24.455 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:24.456 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-06-06T07:27:24.457 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-06-06T07:27:24.458 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-06T07:27:24.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 16817, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-06-06T07:27:24.460 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 2, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 27, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 346035102, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 349598426, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 349594875, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.461 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 16817 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 1340, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 3286, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 1946, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 77, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 79, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 2, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 3686, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 3686, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 0, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-06-06T07:27:24.462 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 56420, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215396 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.463 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-06T07:27:24.464 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.465 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-06T07:27:24.468 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.470 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.472 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.473 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.474 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.475 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 2, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 0, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 0, 2024-06-06T07:27:24.476 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 2781, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:24.477 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:24.478 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:24.479 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 222343, 2024-06-06T07:27:24.480 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 66, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:24.481 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 186, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 5691871, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 22090, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:24.482 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 1537, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 26, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 4, 2024-06-06T07:27:24.483 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 1537, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1537, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.396970124, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000908894 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1537, 2024-06-06T07:27:24.484 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1538 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 1537, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 1537, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 760, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-06-06T07:27:24.485 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 774, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:24.486 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 2, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-06T07:27:24.487 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-06T07:27:24.488 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 148, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 14, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 9, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 9, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-06-06T07:27:24.489 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:24.490 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 1678, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 106639418, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 1678, 2024-06-06T07:27:24.491 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1678, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 106639418, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:24.492 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 1537, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 106075054, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1537, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 106075054, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 1537, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 1537, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1537, 2024-06-06T07:27:24.494 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1537, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:24.495 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:24.496 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x559eb7fde6e0": { 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:24.497 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.498 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:24.498 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-06T07:27:25.115 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-06-06T07:27:25.116 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-06-06T07:27:25.116 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-06-06T07:27:25.116 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:25.116 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:25.116 DEBUG:tasks.check_counter:Getting stats from h 2024-06-06T07:27:25.117 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-06T07:27:25.359 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:27:25.750 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:25 smithi136 ceph-mon[35901]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-06-06T07:27:25.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:25 smithi073 ceph-mon[32303]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-06-06T07:27:25.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:25 smithi170 ceph-mon[36108]: pgmap v831: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 24 op/s 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 50093, 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 50574, 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 17099090, 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 95061493, 2024-06-06T07:27:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 12, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 3.680703128, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 1.558480003, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.589452769, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.719930815, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 50573, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.641718362, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000032462 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 59291, 2024-06-06T07:27:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.009875252, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000166 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 648880 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 38804, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 43956, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 14321814, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 154892470, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 25, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 8, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 3.756678131, 2024-06-06T07:27:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 1.157028777, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 1.012499306, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.543538221, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 43952, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 1.495890276, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000034034 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 31761, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.005340564, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000168 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 2008816, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 320112 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 481477, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 548773, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 151844163, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 445448967, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 13, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 24.491051658, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 9.759048096, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 16.970626708, 2024-06-06T07:27:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 1.088356791, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 548773, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 12.300042709, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000022413 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 969465, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.148625528, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000153 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 153776 2024-06-06T07:27:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 59763, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 46.820679779, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000783439 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 28676, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 2.565798891, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000089475 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 287831, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 287825, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 287825, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 319.038308586, 2024-06-06T07:27:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001108445 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 6, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 13, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 169, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 11, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 621, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 233, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 368, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 20, 2024-06-06T07:27:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 288, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 3439, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 6, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 552070, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 479350, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 6, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 136, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 5, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 5, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 1000, 2024-06-06T07:27:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 1, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 88, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 2994, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 26, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 766, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 4, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 234, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 6, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 1097, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 28066, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 51792, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 6, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 53404, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 17721, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 690, 2024-06-06T07:27:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 48353 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 3, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 45, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 31, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 157, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 223, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 10, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 13944, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 13934, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 88, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 11, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-06T07:27:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 149067, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 48338, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 36871, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 112196, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 11467, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 188, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 60, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 47, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 141, 2024-06-06T07:27:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 10, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 13, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 68660828, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 324412479, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 0, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 30501, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 138.339522577, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004535573 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 0 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-06T07:27:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 505, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 17544, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 17039, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 163, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 201, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 38, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 890, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 25148, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 24258, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 14060, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 14060, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 83352, 2024-06-06T07:27:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 223612 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 369721, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 2234, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 287831, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 797, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 23561, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13609, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 44.719017514, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.003285988 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-06T07:27:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 183336, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 67.660798222, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000369053 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 12150, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6.658897850, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000548057 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 10457, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 8.555192478, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000818130 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 51643, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 67.883595217, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.001314478 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-06T07:27:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 2326, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 53.848758177, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.023150798 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 10, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.062075856, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.006207585 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 360, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.174782301, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000485506 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 13934, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 69.475190971, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.004986019 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 1, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 36543, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 36543, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 1567, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 377732, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 124, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 186, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 2410122, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 13627, 2024-06-06T07:27:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 62668, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 324681272, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 33, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 62668, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 62668, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 321.488050158, 2024-06-06T07:27:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.005130019 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 62668, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 64429 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 62668, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 1301, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 61367, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 70, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 27, 2024-06-06T07:27:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 1288, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 31789, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 701, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 11, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 27971, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 10, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 51, 2024-06-06T07:27:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-06T07:27:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 148, 2024-06-06T07:27:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 50, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 12, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 19, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 7, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 1716, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 26, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 769 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 1, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 3 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 10, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 5, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 27868, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 13934, 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 570370, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 131895928, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 570370, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 570370, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 131895928, 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 62668, 2024-06-06T07:27:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 326215726, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 62668, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 326215726, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 62668, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 62668, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 62668, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 62668, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 13934, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 1534662, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 13934, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 1288, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 1534662, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x5647cfada500": { 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 149067, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 320218175, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 149067, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 30501, 2024-06-06T07:27:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 320218175, 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:27.077 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-06T07:27:27.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:27 smithi073 ceph-mon[32303]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-06-06T07:27:27.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:27 smithi136 ceph-mon[35901]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-06-06T07:27:27.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:27 smithi170 ceph-mon[36108]: pgmap v832: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-06-06T07:27:27.885 INFO:tasks.check_counter:Daemon mds.h mds.exported=88 2024-06-06T07:27:27.885 INFO:tasks.check_counter:Daemon mds.h mds.imported=26 2024-06-06T07:27:27.885 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-06-06T07:27:27.885 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:27.885 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:27.885 DEBUG:tasks.check_counter:Getting stats from k 2024-06-06T07:27:27.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-06T07:27:28.127 INFO:teuthology.orchestra.run.smithi136.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.b/config 2024-06-06T07:27:29.551 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:29 smithi136 ceph-mon[35901]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-06-06T07:27:29.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:29 smithi073 ceph-mon[32303]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-06-06T07:27:29.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:29 smithi170 ceph-mon[36108]: pgmap v833: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.1 KiB/s wr, 17 op/s 2024-06-06T07:27:29.874 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-06-06T07:27:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:29.874 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 3877, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 3879, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 198350297, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1232893, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 8, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.385873624, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.094464926, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.181578564, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.085897580, 2024-06-06T07:27:29.875 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 3877, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.105141187, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000027119 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 0 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:29.876 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 1435, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1809, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 159749694, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 1259636, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 9, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 7, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.192045684, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.040726702, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.110668432, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.029782667, 2024-06-06T07:27:29.877 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1807, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.052434898, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000029017 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 805152 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.878 INFO:teuthology.orchestra.run.smithi136.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_messages": 2043, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages": 1971, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_bytes": 182714276, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_bytes": 768198, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_active_connections": 4, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_total_time": 0.268142355, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_send_time": 0.053393333, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_recv_time": 0.175346116, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_running_fast_dispatch_time": 0.039065951, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 1970, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.050904949, 2024-06-06T07:27:29.879 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000025840 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_recv_encrypted_bytes": 1996784, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "msgr_send_encrypted_bytes": 316320 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "cct": { 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "total_workers": 1, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "unhealthy_workers": 0 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.880 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-MDSRank": { 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 11369, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.223851374, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000019689 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "queue_len": 0, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "complete_latency": { 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.881 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "mds": { 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "request": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "reply": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "reply_latency": { 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_reply": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "forward": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_complete": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:29.882 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_commit": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_split": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_merge": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes": 17099, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_top": 2769, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_bottom": 14330, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pin_tail": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_pinned": 13142, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_expired": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "inodes_with_caps": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "caps": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "subtrees": 1, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_hit": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_forward": 0, 2024-06-06T07:27:29.883 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_discover": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "traverse_lock": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "load_cent": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "q": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "exported": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "exported_inodes": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "imported": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "imported_inodes": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rfiles": 0, 2024-06-06T07:27:29.884 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rbytes": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "root_rsnaps": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_inode_file_caps": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_revoke": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_grant": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:29.885 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_caps_dirty": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_cap_release": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "process_request_cap_release": 0 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_cache": { 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_update_receipt": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_try_discover": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_send_discover": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "dir_handle_discover": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays": 0, 2024-06-06T07:27:29.886 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_created": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_enqueued": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "strays_migrated": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_started": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "recovery_completed": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:29.887 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_exportdir": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_flush": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ireq_inodestats": 0 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_log": { 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evlrg": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evadd": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evex": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evtrm": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "ev": 122729, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evexg": 0, 2024-06-06T07:27:29.888 INFO:teuthology.orchestra.run.smithi136.stdout: "evexd": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segadd": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segex": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segtrm": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "seg": 137, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segmjr": 11, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segexg": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "segexd": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "expos": 47453919, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "wrpos": 541150385, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "rdpos": 541146613, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "jlat": { 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.889 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "replayed": 122729 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_mem": { 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "ino": 2283, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "ino+": 17770, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "ino-": 15487, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dir": 96, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dir+": 171, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dir-": 75, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dn": 17099, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dn+": 18166, 2024-06-06T07:27:29.890 INFO:teuthology.orchestra.run.smithi136.stdout: "dn-": 1067, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "cap": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "cap+": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "cap-": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "rss": 77328, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "heap": 215420 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_server": { 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_client_request": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "dispatch_server_request": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_request": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_client_session": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "handle_peer_request": 0, 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "req_create_latency": { 2024-06-06T07:27:29.891 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getattr_latency": { 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.892 INFO:teuthology.orchestra.run.smithi136.stdout: "req_link_latency": { 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookup_latency": { 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.893 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupino_latency": { 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupname_latency": { 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.894 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "req_lssnap_latency": { 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.895 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mkdir_latency": { 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.896 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mknod_latency": { 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "req_mksnap_latency": { 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "req_open_latency": { 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.897 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "req_readdir_latency": { 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rename_latency": { 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.898 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmdir_latency": { 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.899 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setattr_latency": { 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.900 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setlayout_latency": { 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.901 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "req_setxattr_latency": { 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "req_symlink_latency": { 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "req_unlink_latency": { 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.902 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_sessions": { 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "session_count": 1, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "session_add": 2, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "session_remove": 1, 2024-06-06T07:27:29.903 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_open": 2, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "sessions_stale": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "total_load": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "average_load": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "avg_session_uptime": 4009, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "mempool": { 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:29.904 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:29.905 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:29.906 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_bytes": 242551, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_anon_items": 85, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_bytes": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_items": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_items": 186, 2024-06-06T07:27:29.907 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "pgmap_items": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_bytes": 17412960, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "mds_co_items": 80491, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_1_items": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "unittest_2_items": 0 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "objecter": { 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "op_active": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "op_laggy": 0, 2024-06-06T07:27:29.908 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send": 6727, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "op_send_bytes": 21, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "op_resend": 3, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "op_reply": 6727, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "op_latency": { 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6727, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 4.857894711, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000722148 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "op_inflight": 0, 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "oplen_avg": { 2024-06-06T07:27:29.909 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 6727, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 6728 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "op": 6727, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "op_r": 6727, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "op_w": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "op_rmw": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "op_pg": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_stat": 3595, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_create": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_read": 3130, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_write": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writefull": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_writesame": 0, 2024-06-06T07:27:29.910 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_append": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_zero": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_truncate": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_delete": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_mapext": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_getxattr": 1, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_setxattr": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_call": 0, 2024-06-06T07:27:29.911 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_watch": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_notify": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "osdop_other": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_active": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_send": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_resend": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "linger_ping": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_active": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_send": 0, 2024-06-06T07:27:29.912 INFO:teuthology.orchestra.run.smithi136.stdout: "poolop_resend": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_active": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_send": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "poolstat_resend": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_active": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_send": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "statfs_resend": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "command_active": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "command_send": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "command_resend": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "map_epoch": 148, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "map_full": 0, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "map_inc": 50, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_sessions": 11, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_open": 12, 2024-06-06T07:27:29.913 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_session_close": 1, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "osd_laggy": 0, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_wr": 0, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_rd": 2, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_del": 0 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "oft": { 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_objs": 0, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_updates": 0, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "omap_total_removes": 0 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "osd-slow-ops": { 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: "slow_ops_count": 0 2024-06-06T07:27:29.914 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "purge_queue": { 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_ops_high_water": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executing_high_water": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed_ops": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_executed": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:29.915 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 7351, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 540118223, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 7351, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 7351, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 540118223, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.916 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 104857600, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6727, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 536959291, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6727, 2024-06-06T07:27:29.917 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 536959291, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 1024, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:29.918 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 6727, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 6727, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 6727, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 6727, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:29.919 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.920 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "throttle-write_buf_throttle-0x55ded29ec6e0": { 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "val": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "max": 3758096384, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "get_started": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "get": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "get_sum": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "take": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "take_sum": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "put": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "put_sum": 0, 2024-06-06T07:27:29.921 INFO:teuthology.orchestra.run.smithi136.stdout: "wait": { 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgcount": 0, 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "sum": 0.000000000, 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: "avgtime": 0.000000000 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-06-06T07:27:29.922 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-06-06T07:27:30.647 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-06T07:27:30.647 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-06T07:27:30.647 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-06-06T07:27:30.647 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:30.647 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:30.648 DEBUG:tasks.check_counter:Getting stats from c 2024-06-06T07:27:30.648 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-06T07:27:30.956 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:27:31.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:31 smithi073 ceph-mon[32303]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:31.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:31 smithi136 ceph-mon[35901]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:31.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:31 smithi170 ceph-mon[36108]: pgmap v834: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 131353, 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 145367, 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 37840128, 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 170034754, 2024-06-06T07:27:32.743 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 6, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 8.991779694, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 4.172750042, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 4.929065616, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.726784798, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 145364, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 4.339835878, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000029854 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 229016, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.039356079, 2024-06-06T07:27:32.744 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000171 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2023584, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 341696 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 592554, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 692769, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 176583513, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 642554322, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 17, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 33.936456196, 2024-06-06T07:27:32.745 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 13.802040986, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 23.705484312, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.928504058, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 692769, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 21.486239348, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000031015 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1216738, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.214662240, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000176 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.746 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 5248, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 159040 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 47989, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 43088, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 14906069, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 241256425, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 11, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 3.455239425, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.171934119, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.255255070, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 1.057183198, 2024-06-06T07:27:32.747 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 43086, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.741069364, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000040409 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31874, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.005762453, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000180 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 160, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 650256 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-06-06T07:27:32.748 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 67921, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 52.713261495, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000776096 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:32.749 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 32525, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.520899104, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000077506 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 466311, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 463628, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 463628, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 365.218780965, 2024-06-06T07:27:32.750 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000787740 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 2683, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 15, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 98, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 10, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 5, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2531, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 609, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 1827, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 95, 2024-06-06T07:27:32.751 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 1660, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1319, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 2, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 2, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 886804, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 827970, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 176, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 115, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 1, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 190, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-06-06T07:27:32.752 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 31, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 1018, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 20, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 1310, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 11, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 11, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:32.753 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 11, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 2607, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 37011, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 57904, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 61266, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 17680, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 1218, 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 52052 2024-06-06T07:27:32.754 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 14, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 34, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 24, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 131, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 108, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 20, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 14541, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 14521, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-06-06T07:27:32.755 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 47, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 15, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.756 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 122729, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 44593, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 36871, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 85858, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 7722, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 181, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 53, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 44, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 137, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 7, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-06-06T07:27:32.757 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 9, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 47453919, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 541150385, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 33678, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 107.942968907, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003205147 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 2241, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 17857, 2024-06-06T07:27:32.758 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 15616, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 120, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 194, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 74, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 2533, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 24851, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 22318, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 2, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 14617, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 14615, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 94220, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223612 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-06-06T07:27:32.759 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 538847, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 3308, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 466311, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 1318, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 18784, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14378, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 30.089640013, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002092755 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 355501, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 149.068819167, 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000419320 2024-06-06T07:27:32.760 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 12234, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 7.815025726, 2024-06-06T07:27:32.761 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000638795 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-06-06T07:27:32.762 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:32.763 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-06-06T07:27:32.764 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 9947, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 8.188845681, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000823247 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 53556, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 97.850672807, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001827072 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2828, 2024-06-06T07:27:32.766 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 28.800661889, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.010184109 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.767 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.102112015, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.005105600 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.768 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 99, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.494097401, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004990882 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 544, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.206520453, 2024-06-06T07:27:32.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000379633 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-06-06T07:27:32.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 14521, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 42.602385813, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002933846 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 1, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 2, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 1, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 1, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 49594, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 49594, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1569, 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:32.772 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:32.773 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:32.774 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 477368, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 169, 2024-06-06T07:27:32.775 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 186, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-06-06T07:27:32.776 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 7193311, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 30895, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 70292, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 540360776, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 28, 2024-06-06T07:27:32.777 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 70292, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 70292, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 267.566611601, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003806501 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 70292, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 72865 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 70292, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 2333, 2024-06-06T07:27:32.778 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 67959, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 21, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 2307, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 35985, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 1282, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 8, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 29204, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-06-06T07:27:32.779 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 12, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 42, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:32.780 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-06-06T07:27:32.781 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 148, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 50, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 2712, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 30, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 1262 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.782 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 0 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 5, 2024-06-06T07:27:32.783 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 29042, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 14521, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 771892, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 159823196, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 771892, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:32.784 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 771892, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 159823196, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:32.785 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 70292, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 541954693, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 70292, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 541954693, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.786 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 70292, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 70292, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 70292, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 70292, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 14521, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1594157, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 14521, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:32.788 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 2307, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1594157, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x559f2a7a0500": { 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 122729, 2024-06-06T07:27:32.789 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 536956081, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 122729, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 33678, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 536956081, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:32.790 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-06-06T07:27:33.534 INFO:tasks.check_counter:Daemon mds.c mds.exported=31 2024-06-06T07:27:33.535 INFO:tasks.check_counter:Daemon mds.c mds.imported=20 2024-06-06T07:27:33.535 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-06-06T07:27:33.535 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:33.535 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:33.535 DEBUG:tasks.check_counter:Getting stats from f 2024-06-06T07:27:33.535 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-06T07:27:33.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:33 smithi073 ceph-mon[32303]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:33.782 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:27:33.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:33 smithi136 ceph-mon[35901]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:33.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:33 smithi170 ceph-mon[36108]: pgmap v835: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-06-06T07:27:35.743 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:35 smithi170 ceph-mon[36108]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 17 op/s 2024-06-06T07:27:35.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:35 smithi073 ceph-mon[32303]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 17 op/s 2024-06-06T07:27:35.794 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-06-06T07:27:35.794 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:35.794 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 45606, 2024-06-06T07:27:35.794 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 36085, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 18842183, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 108777547, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 11, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 5, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 3.717870488, 2024-06-06T07:27:35.795 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.339982768, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 1.709020272, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 1.021471977, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 36083, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.371325638, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000038004 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:35.796 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 62161, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.011982959, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000192 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 5088, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 30576 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 627122, 2024-06-06T07:27:35.797 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 642296, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 186731402, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 530802701, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 19, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 7, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 33.009156658, 2024-06-06T07:27:35.798 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 13.048744806, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 23.493494770, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 1.243413950, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 642296, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.808948966, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000029283 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.799 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1209815, 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.213307916, 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000176 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2020016, 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 323616 2024-06-06T07:27:35.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 60582, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 59255, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 18688073, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 58601311, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 15, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 3.592934654, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 1.288611325, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 2.241636367, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.467338807, 2024-06-06T07:27:35.801 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59252, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 2.005800508, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000033852 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 106525, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.018200591, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000170 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.802 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 808832 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 46778, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 31.538460271, 2024-06-06T07:27:35.803 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000674215 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8303, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.924547591, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000111351 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 521057, 2024-06-06T07:27:35.804 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 489884, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 489884, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 497.337945322, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001015215 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 31173, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 40, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 0, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 11, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 0, 2024-06-06T07:27:35.805 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 2857, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 152, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 2499, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 206, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 1798, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 1461, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 2, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 2, 2024-06-06T07:27:35.806 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 1000104, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 992253, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 5634, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 205, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 4, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 0, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 34, 2024-06-06T07:27:35.807 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 914, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 75, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 3235, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 0, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-06-06T07:27:35.808 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 0, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 18, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 26749, 2024-06-06T07:27:35.809 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 12626, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 32560, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 4221, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 450, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 11275 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 8, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 94, 2024-06-06T07:27:35.810 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 67, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 246, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 108, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 30, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 3265, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 3235, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-06-06T07:27:35.811 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 36, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-06-06T07:27:35.812 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 0, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 40598, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 0, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 0, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 40598, 2024-06-06T07:27:35.813 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 0, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 83, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 0, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 0, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 83, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 5, 2024-06-06T07:27:35.814 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 0, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 4194304, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 299512312, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24306, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 118.837313403, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004889217 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.815 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 2417, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 6659, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 4242, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 110, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 200, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 90, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 2857, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 7833, 2024-06-06T07:27:35.816 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 4976, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 2, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 3662, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 3660, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 82232, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 223612 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 547235, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 18554, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 521057, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 382, 2024-06-06T07:27:35.817 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 37307, 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3155, 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5.031963986, 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001594917 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-06-06T07:27:35.818 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 457293, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 191.502226516, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000418773 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.819 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-06-06T07:27:35.820 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.820 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.820 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.820 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.820 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 18545, 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 205.769570783, 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.011095689 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.821 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.822 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:35.823 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.824 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.825 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:35 smithi136 ceph-mon[35901]: pgmap v836: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 9.3 KiB/s wr, 17 op/s 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.825 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.826 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1625, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.603285820, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000986637 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5434, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 18.004164575, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003313243 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 543, 2024-06-06T07:27:35.827 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 65.991584246, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.121531462 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.828 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 30, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.269548838, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.008984961 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.829 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:35.830 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 24, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.012299981, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000512499 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.831 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3235, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 9.153300577, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002829459 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:35.832 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 4, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 2, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 54052, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 27026, 2024-06-06T07:27:35.833 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1551, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:35.834 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:35.835 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:35.836 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 560395, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 194, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 186, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:35.837 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 7737160, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 30279, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 33792, 2024-06-06T07:27:35.838 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 295984433, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 33, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 33792, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 33792, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 160.187188366, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004740387 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 33792, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 35024 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.839 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 33792, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 841, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 32951, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 10, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 801, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 25107, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 513, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 0, 2024-06-06T07:27:35.840 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 6552, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 40, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 20, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-06-06T07:27:35.841 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-06-06T07:27:35.842 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 148, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 50, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 1330, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 80, 2024-06-06T07:27:35.843 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 571 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 2 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-06-06T07:27:35.844 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 5, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 6470, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 3235, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 733306, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 158226549, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 733306, 2024-06-06T07:27:35.845 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 733306, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 158226549, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:35.846 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 33792, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 296338872, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 33792, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 296338872, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.847 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 33792, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 33792, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 33792, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 33792, 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:35.848 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 3235, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 355079, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 3235, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:35.849 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 801, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 355079, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x560bfdd22500": { 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 40598, 2024-06-06T07:27:35.850 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 295318008, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 40598, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 24306, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 295318008, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:35.851 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-06-06T07:27:36.617 INFO:tasks.check_counter:Daemon mds.f mds.exported=34 2024-06-06T07:27:36.617 INFO:tasks.check_counter:Daemon mds.f mds.imported=75 2024-06-06T07:27:36.617 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-06-06T07:27:36.617 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-06-06T07:27:36.617 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-06-06T07:27:36.617 DEBUG:tasks.check_counter:Getting stats from i 2024-06-06T07:27:36.618 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-06T07:27:36.880 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:27:37.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:37 smithi170 ceph-mon[36108]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:37.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:37 smithi073 ceph-mon[32303]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:37.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:37 smithi136 ceph-mon[35901]: pgmap v837: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:38.763 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 139440, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 125453, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 51038858, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 129698783, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 17, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 8.884607328, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 4.167406232, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 5.244727365, 2024-06-06T07:27:38.764 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.564581163, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 125454, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 6.005763662, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000047872 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 230364, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.036931298, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000160 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2141664, 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 1024944 2024-06-06T07:27:38.765 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 1149076, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 1184146, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 338220432, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 832406586, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 18, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 9, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 52.557613707, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 21.374385525, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 37.723806418, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 1.077260256, 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:38.766 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1184141, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 32.309956564, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000027285 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2260366, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.324342865, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000143 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 823184, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448976 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:38.767 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 96022, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 113261, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 25899767, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 165224576, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 255, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 10, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 5.739423748, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 2.228687687, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 3.276938571, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.748158113, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 113260, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 5.512032556, 2024-06-06T07:27:38.768 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000048667 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 165793, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.025207673, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000152 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 45488, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 1584 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 1, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.769 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-MDSRank": { 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 78318, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 51.604829522, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000658914 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "finisher-PurgeQueue": { 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "queue_len": 0, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "complete_latency": { 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 11595, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 1.170753422, 2024-06-06T07:27:38.770 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000100970 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "mds": { 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "request": 1002138, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "reply": 942661, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "reply_latency": { 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 942661, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 829.866579163, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000880344 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_reply": 0, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "forward": 59477, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_complete": 33, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_fetch_keys": 2, 2024-06-06T07:27:38.771 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_commit": 2138, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_split": 1, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_merge": 0, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes": 1923, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_top": 925, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_bottom": 988, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pin_tail": 10, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_pinned": 601, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_expired": 3036, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "inodes_with_caps": 3, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "caps": 3, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "subtrees": 6, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse": 1924509, 2024-06-06T07:27:38.772 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_hit": 1898785, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_forward": 24173, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_discover": 135, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_dir_fetch": 0, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_remote_ino": 0, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "traverse_lock": 10, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "load_cent": 16, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "q": 0, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "exported": 48, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "exported_inodes": 2374, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "imported": 80, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "imported_inodes": 2163, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_dir_fetch": 0, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_backtrace_fetch": 106, 2024-06-06T07:27:38.773 INFO:teuthology.orchestra.run.smithi170.stdout: "openino_peer_discover": 0, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rfiles": 4, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rbytes": 234, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "root_rsnaps": 6, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_fetch": 106, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_set_tag": 0, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_backtrace_repaired": 0, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_inotable_repaired": 0, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_inodes": 36, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dir_base_inodes": 3, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_dirfrag_rstats": 33, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "scrub_file_inodes": 70, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_inode_file_caps": 27, 2024-06-06T07:27:38.774 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_revoke": 48302, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_grant": 22876, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_trunc": 0, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps": 58539, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_caps_dirty": 6719, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_cap_release": 597, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "process_request_cap_release": 18223 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_cache": { 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update": 38, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_update_receipt": 12, 2024-06-06T07:27:38.775 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_try_discover": 8, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_send_discover": 139, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "dir_handle_discover": 198, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays": 33, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_delayed": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_strays_enqueuing": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_created": 4644, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_enqueued": 4611, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_reintegrated": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "strays_migrated": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_processing": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_enqueued": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "num_recovering_prioritized": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_started": 0, 2024-06-06T07:27:38.776 INFO:teuthology.orchestra.run.smithi170.stdout: "recovery_completed": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_path": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_quiesce_inode": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_enqueue_scrub": 3, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_exportdir": 57, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_flush": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragmentdir": 1, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_fragstats": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "ireq_inodestats": 0 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_log": { 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "evlrg": 0, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "evadd": 61199, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "evex": 60676, 2024-06-06T07:27:38.777 INFO:teuthology.orchestra.run.smithi170.stdout: "evtrm": 60676, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "ev": 523, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evexg": 0, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "evexd": 784, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segadd": 201, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segex": 199, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segtrm": 199, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "seg": 2, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segmjr": 1, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segexg": 0, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "segexd": 2, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "expos": 346035102, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "wrpos": 349623892, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "rdpos": 0, 2024-06-06T07:27:38.778 INFO:teuthology.orchestra.run.smithi170.stdout: "jlat": { 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 39234, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 163.827580046, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004175653 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "replayed": 0 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_mem": { 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "ino": 1581, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "ino+": 8620, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "ino-": 7039, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "dir": 95, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "dir+": 293, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "dir-": 198, 2024-06-06T07:27:38.779 INFO:teuthology.orchestra.run.smithi170.stdout: "dn": 1923, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dn+": 11728, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dn-": 9805, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "cap": 3, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "cap+": 5551, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "cap-": 5548, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "rss": 82888, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "heap": 231804 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_server": { 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_client_request": 1049988, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "dispatch_server_request": 27536, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_request": 1002138, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_client_session": 685, 2024-06-06T07:27:38.780 INFO:teuthology.orchestra.run.smithi170.stdout: "handle_peer_request": 58597, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "req_create_latency": { 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 5341, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 10.583474329, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001981552 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getattr_latency": { 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 888432, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 411.452939854, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000463122 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getfilelock_latency": { 2024-06-06T07:27:38.781 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_link_latency": { 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookup_latency": { 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 31225, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 229.899200754, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.007362664 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.782 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookuphash_latency": { 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupino_latency": { 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupname_latency": { 2024-06-06T07:27:38.783 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupparent_latency": { 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lookupsnap_latency": { 2024-06-06T07:27:38.784 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "req_lssnap_latency": { 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26, 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.007828502, 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000301096 2024-06-06T07:27:38.785 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mkdir_latency": { 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 119, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.404699337, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.003400834 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mknod_latency": { 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "req_mksnap_latency": { 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 26, 2024-06-06T07:27:38.786 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 15.839913322, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.609227435 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_open_latency": { 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2743, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 3.804901524, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001387131 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_readdir_latency": { 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8749, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 42.371150850, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.004842970 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rename_latency": { 2024-06-06T07:27:38.787 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 1151, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 101.943371452, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.088569393 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_renamesnap_latency": { 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_snapdiff_latency": { 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.788 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmdir_latency": { 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 44, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.619677130, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.014083571 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmsnap_latency": { 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 20, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.869374709, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.043468735 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_rmxattr_latency": { 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.789 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setattr_latency": { 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 2, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.002389542, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.001194771 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setdirlayout_latency": { 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setfilelock_latency": { 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 164, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.122736880, 2024-06-06T07:27:38.790 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000748395 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setlayout_latency": { 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_setxattr_latency": { 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 8, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.022030071, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002753758 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "req_symlink_latency": { 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.791 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "req_unlink_latency": { 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 4611, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 11.922890907, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.002585749 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_revoke_eviction": 0, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "cap_acquisition_throttle": 0, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "req_getvxattr_latency": { 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.792 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.793 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.793 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_sessions": { 2024-06-06T07:27:38.793 INFO:teuthology.orchestra.run.smithi170.stdout: "session_count": 2, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "session_add": 6, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "session_remove": 4, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_open": 2, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "sessions_stale": 0, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "total_load": 53087, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "average_load": 26543, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "avg_session_uptime": 1554, 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: "mdthresh_evicted": 0 2024-06-06T07:27:38.794 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 360, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 360, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:38.795 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:38.796 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 573679, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 200, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 88, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 1, 2024-06-06T07:27:38.797 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 9704, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 186, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 5217870, 2024-06-06T07:27:38.798 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 21578, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "objecter": { 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_active": 0, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_laggy": 0, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send": 59255, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_send_bytes": 359639739, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_resend": 37, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_reply": 59255, 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "op_latency": { 2024-06-06T07:27:38.799 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59255, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 510.667772469, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.008618138 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_inflight": 0, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "oplen_avg": { 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 59255, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 70576 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op": 59255, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_r": 1212, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_w": 58043, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_rmw": 0, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "op_pg": 0, 2024-06-06T07:27:38.800 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_stat": 2010, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_create": 3246, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_read": 1068, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_write": 40299, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writefull": 1049, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_writesame": 0, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_append": 0, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_zero": 0, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_truncate": 1, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_delete": 9897, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_mapext": 0, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_sparse_read": 0, 2024-06-06T07:27:38.801 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_clonerange": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_getxattr": 144, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_setxattr": 4943, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_cmpxattr": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_rmxattr": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_resetxattrs": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_call": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_watch": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_notify": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_src_cmpxattr": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_pgls_filter": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "osdop_other": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_active": 0, 2024-06-06T07:27:38.802 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_send": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_resend": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "linger_ping": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_active": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_send": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolop_resend": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_active": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_send": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "poolstat_resend": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_active": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_send": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "statfs_resend": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_active": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_send": 0, 2024-06-06T07:27:38.803 INFO:teuthology.orchestra.run.smithi170.stdout: "command_resend": 0, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "map_epoch": 148, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "map_full": 0, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "map_inc": 59, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_sessions": 12, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_open": 12, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_session_close": 0, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_laggy": 0, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_wr": 6624, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_rd": 70, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_del": 1225 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "oft": { 2024-06-06T07:27:38.804 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_objs": 1, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_kv_pairs": 0, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_updates": 0, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "omap_total_removes": 0 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "osd-slow-ops": { 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "slow_ops_count": 2 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "purge_queue": { 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops": 0, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_ops_high_water": 10, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing": 0, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executing_high_water": 5, 2024-06-06T07:27:38.805 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed_ops": 9222, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_executed": 4611, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "pq_item_in_journal": 0 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 1384534, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 290411686, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 1384534, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:38.806 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1384534, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 290411686, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_bytes": { 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:38.807 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 59255, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 360144970, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 59255, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 360144970, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.808 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-objecter_ops": { 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 1024, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 0, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 59255, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 59255, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 59255, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 59255, 2024-06-06T07:27:38.809 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle": { 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 4611, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 505767, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:38.810 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 4611, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 1065, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 505767, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-write_buf_throttle-0x559c8efb4500": { 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 3758096384, 2024-06-06T07:27:38.811 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 61199, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 345429588, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 61199, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 39234, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 345429588, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.812 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:38.813 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-06-06T07:27:39.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:39 smithi170 ceph-mon[36108]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-06-06T07:27:39.635 INFO:tasks.check_counter:Daemon mds.i mds.exported=48 2024-06-06T07:27:39.635 INFO:tasks.check_counter:Daemon mds.i mds.imported=80 2024-06-06T07:27:39.635 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-06-06T07:27:39.635 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=26 2024-06-06T07:27:39.635 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=20 2024-06-06T07:27:39.635 DEBUG:tasks.check_counter:Getting stats from l 2024-06-06T07:27:39.635 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-06T07:27:39.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:39 smithi073 ceph-mon[32303]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-06-06T07:27:39.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:39 smithi136 ceph-mon[35901]: pgmap v838: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.3 KiB/s wr, 16 op/s 2024-06-06T07:27:39.886 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.c/config 2024-06-06T07:27:41.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:41 smithi170 ceph-mon[36108]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:41.730 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-06-06T07:27:41.730 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-0": { 2024-06-06T07:27:41.730 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 4, 2024-06-06T07:27:41.730 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 4, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 2944, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 448, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 4, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 0, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.001429054, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.000036492, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.000632490, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 3, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000752329, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000250776 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.731 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 448 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-1": { 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 509, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 499, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 1525312, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 309248, 2024-06-06T07:27:41.732 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 2, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 1, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.066793451, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.019723237, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.039940037, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.010881581, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 498, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.017283157, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000034705 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:41.733 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 1525312, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 309248 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "AsyncMessenger::Worker-2": { 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_messages": 3, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages": 382, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_bytes": 480, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_bytes": 300944, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_created_connections": 6, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_active_connections": 3, 2024-06-06T07:27:41.734 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_total_time": 0.019552309, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_send_time": 0.014299010, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_recv_time": 0.001898798, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_messages_queue_lat": { 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 379, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.027213609, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000071803 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_handle_ack_lat": { 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.735 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "msgr_send_encrypted_bytes": 300944 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "cct": { 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "total_workers": 0, 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "unhealthy_workers": 0 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "mempool": { 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_bytes": 0, 2024-06-06T07:27:41.736 INFO:teuthology.orchestra.run.smithi170.stdout: "bloom_filter_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_alloc_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_data_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_onode_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_meta_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_other_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_cache_buffer_items": 0, 2024-06-06T07:27:41.737 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_extent_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_blob_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_shared_blob_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_inline_bl_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_fsck_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_txc_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_deferred_items": 0, 2024-06-06T07:27:41.738 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluestore_writing_items": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_items": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_reader_items": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "bluefs_file_writer_items": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_bytes": 149735, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_anon_items": 30, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "buffer_meta_items": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_bytes": 0, 2024-06-06T07:27:41.739 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_bytes": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_mapbl_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_bytes": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osd_pglog_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_bytes": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_bytes": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "osdmap_mapping_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_bytes": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "pgmap_items": 0, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_bytes": 928, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "mds_co_items": 2, 2024-06-06T07:27:41.740 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_bytes": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_1_items": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_bytes": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "unittest_2_items": 0 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: }, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "val": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "max": 104857600, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "get_started": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "get": 512, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "get_sum": 1453383, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_fail": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "get_or_fail_success": 512, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "take": 0, 2024-06-06T07:27:41.741 INFO:teuthology.orchestra.run.smithi170.stdout: "take_sum": 0, 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "put": 512, 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "put_sum": 1453383, 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "wait": { 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgcount": 0, 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "sum": 0.000000000, 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: "avgtime": 0.000000000 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout: } 2024-06-06T07:27:41.742 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-06-06T07:27:41.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:41 smithi073 ceph-mon[32303]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:41.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:41 smithi136 ceph-mon[35901]: pgmap v839: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:42.509 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-06-06T07:27:42.510 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-06-06T07:27:42.510 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-06-06T07:27:42.510 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-06-06T07:27:42.510 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-06-06T07:27:42.510 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-06-06T07:27:42.521 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-06-06T07:27:42.522 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T07:27:42.522 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:27:42.562 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T07:27:42.563 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:27:42.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T07:27:42.914 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:43.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:43 smithi073 ceph-mon[32303]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:43.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:43 smithi136 ceph-mon[35901]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:43.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:43 smithi170 ceph-mon[36108]: pgmap v840: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.3 KiB/s wr, 11 op/s 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 -- 172.21.15.73:0/778001677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 msgr2=0x7f49541077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 --2- 172.21.15.73:0/778001677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f49541077b0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4948009990 tx=0x7f4948009ca0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 -- 172.21.15.73:0/778001677 shutdown_connections 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 --2- 172.21.15.73:0/778001677 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 0x7f495410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 --2- 172.21.15.73:0/778001677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f49541077b0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 --2- 172.21.15.73:0/778001677 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 0x7f49541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 -- 172.21.15.73:0/778001677 >> 172.21.15.73:0/778001677 conn(0x7f49540782c0 msgr2=0x7f49540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.572+0000 7f495aabd700 1 -- 172.21.15.73:0/778001677 shutdown_connections 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 -- 172.21.15.73:0/778001677 wait complete. 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 Processor -- start 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 -- start start 2024-06-06T07:27:44.575 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 0x7f495419e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 0x7f495419edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f49541115c0 con 0x7f49541073d0 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f4954111440 con 0x7f4954101390 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495aabd700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f4954111740 con 0x7f4954101d30 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.573+0000 7f495905a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54502/0 (socket says 172.21.15.73:54502) 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 -- 172.21.15.73:0/658527002 learned_addr learned my addr 172.21.15.73:0/658527002 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f4958859700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 0x7f495419e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f4953fff700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 0x7f495419edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 -- 172.21.15.73:0/658527002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 msgr2=0x7f495419edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 0x7f495419edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.576 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 -- 172.21.15.73:0/658527002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 msgr2=0x7f495419e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 0x7f495419e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 -- 172.21.15.73:0/658527002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49541991f0 con 0x7f49541073d0 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495905a700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b970 tx=0x7f494c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c019070 con 0x7f49541073d0 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49541994e0 con 0x7f49541073d0 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f494c014440 con 0x7f49541073d0 2024-06-06T07:27:44.577 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f494c00ce60 con 0x7f49541073d0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.574+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49541a5750 con 0x7f49541073d0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.575+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4954069180 con 0x7f49541073d0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.576+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f494c01c070 con 0x7f49541073d0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.577+0000 7f4951ffb700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 0x7f49440a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.577+0000 7f4958859700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 0x7f49440a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.577+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9554+0+0 (secure 0 0 0) 0x7f494c0ceb50 con 0x7f49541073d0 2024-06-06T07:27:44.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.577+0000 7f4958859700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 0x7f49440a5250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f494800b540 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:44.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.582+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f494c09b470 con 0x7f49541073d0 2024-06-06T07:27:44.793 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.793+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f49541a59e0 con 0x7f49541073d0 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.793+0000 7f4951ffb700 1 -- 172.21.15.73:0/658527002 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v148) ==== 81+0+939 (secure 0 0 0) 0x7f494c09ad70 con 0x7f49541073d0 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/793833713 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1478961120 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1159052463 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1988924098 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:44.794 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2836597527 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/3113057040 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1117851147 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/905820354 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/27064626 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/793833713 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1626604755 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2368928774 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:44.795 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 msgr2=0x7f49440a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 0x7f49440a5250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4948006010 tx=0x7f494800b540 comp rx=0 tx=0).stop 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 msgr2=0x7f4954198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f494c00b970 tx=0x7f494c00bd30 comp rx=0 tx=0).stop 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 shutdown_connections 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f49440a2d90 0x7f49440a5250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49541073d0 0x7f4954198a90 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f4954101390 0x7f495419e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 --2- 172.21.15.73:0/658527002 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f4954101d30 0x7f495419edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.795+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 >> 172.21.15.73:0/658527002 conn(0x7f49540782c0 msgr2=0x7f495406a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.796+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 shutdown_connections 2024-06-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:44.796+0000 7f495aabd700 1 -- 172.21.15.73:0/658527002 wait complete. 2024-06-06T07:27:44.798 INFO:teuthology.orchestra.run.smithi073.stderr:listed 16 entries 2024-06-06T07:27:45.503 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-06-06T07:27:45.503 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-06T07:27:45.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-06T07:27:45.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:45 smithi073 ceph-mon[32303]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 16 op/s 2024-06-06T07:27:45.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:45 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/658527002' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:45 smithi136 ceph-mon[35901]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 16 op/s 2024-06-06T07:27:45.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:45 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/658527002' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:45.806 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/mon.a/config 2024-06-06T07:27:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:45 smithi170 ceph-mon[36108]: pgmap v841: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.8 KiB/s wr, 16 op/s 2024-06-06T07:27:45.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:45 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/658527002' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:47.634 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:47 smithi073 ceph-mon[32303]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- 172.21.15.73:0/2572961478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c101390 msgr2=0x7f113c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 --2- 172.21.15.73:0/2572961478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c101390 0x7f113c101810 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f1130009990 tx=0x7f1130009ca0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- 172.21.15.73:0/2572961478 shutdown_connections 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 --2- 172.21.15.73:0/2572961478 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 0x7f113c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 --2- 172.21.15.73:0/2572961478 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c107400 0x7f113c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 --2- 172.21.15.73:0/2572961478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c101390 0x7f113c101810 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- 172.21.15.73:0/2572961478 >> 172.21.15.73:0/2572961478 conn(0x7f113c078340 msgr2=0x7f113c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- 172.21.15.73:0/2572961478 shutdown_connections 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- 172.21.15.73:0/2572961478 wait complete. 2024-06-06T07:27:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 Processor -- start 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.647+0000 7f11413a4700 1 -- start start 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 --2- >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c101390 0x7f113c199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 --2- >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 0x7f113c19a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111620 con 0x7f113c107400 2024-06-06T07:27:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 -- --> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1114a0 con 0x7f113c101390 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f11413a4700 1 -- --> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1117a0 con 0x7f113c101d50 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52592/0 (socket says 172.21.15.73:52592) 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 -- 172.21.15.73:0/623546885 learned_addr learned my addr 172.21.15.73:0/623546885 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113a7fc700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 0x7f113c19a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113affd700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c101390 0x7f113c199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 -- 172.21.15.73:0/623546885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 msgr2=0x7f113c19a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 0x7f113c19a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 -- 172.21.15.73:0/623546885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c101390 msgr2=0x7f113c199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c101390 0x7f113c199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 -- 172.21.15.73:0/623546885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c1a5100 con 0x7f113c107400 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f113b7fe700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f113400b970 tx=0x7f113400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.648+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134019070 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.649+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c197930 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.649+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f113400cde0 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.649+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1134007370 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.649+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c197e90 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.649+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113c109970 con 0x7f113c107400 2024-06-06T07:27:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.651+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) ==== 137964+0+0 (secure 0 0 0) 0x7f113401c070 con 0x7f113c107400 2024-06-06T07:27:47.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.651+0000 7f1123fff700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 0x7f11240a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-06T07:27:47.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.651+0000 7f113affd700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 0x7f11240a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-06T07:27:47.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.651+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 9554+0+0 (secure 0 0 0) 0x7f11340cebe0 con 0x7f113c107400 2024-06-06T07:27:47.654 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.652+0000 7f113affd700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 0x7f11240a5250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c008040 comp rx=0 tx=0).ready entity=mgr.15282 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-06T07:27:47.657 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.656+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f113409b500 con 0x7f113c107400 2024-06-06T07:27:47.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:47 smithi136 ceph-mon[35901]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:47.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:47 smithi170 ceph-mon[36108]: pgmap v842: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:47.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.870+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f113c069180 con 0x7f113c107400 2024-06-06T07:27:47.872 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.870+0000 7f1123fff700 1 -- 172.21.15.73:0/623546885 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v148) ==== 81+0+939 (secure 0 0 0) 0x7f113409ae00 con 0x7f113c107400 2024-06-06T07:27:47.872 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/793833713 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.872 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1478961120 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1159052463 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1988924098 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/3054677757 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2836597527 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/3113057040 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1117851147 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.873 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/905820354 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/27064626 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/793833713 2024-06-07T07:01:21.691366+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/1626604755 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:0/2368928774 2024-06-07T06:49:51.216738+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6800/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:47.874 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.15.73:6801/961923295 2024-06-07T06:49:24.637645+0000 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.872+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 msgr2=0x7f11240a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.872+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 0x7f11240a5250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f112c006fd0 tx=0x7f112c008040 comp rx=0 tx=0).stop 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.872+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 msgr2=0x7f113c1a4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.872+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f113400b970 tx=0x7f113400bc80 comp rx=0 tx=0).stop 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.872+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 shutdown_connections 2024-06-06T07:27:47.876 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:6800/3841884199,v1:172.21.15.73:6801/3841884199] conn(0x7f11240a2d90 0x7f11240a5250 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f113c107400 0x7f113c1a4bc0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] conn(0x7f113c101390 0x7f113c199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 --2- 172.21.15.73:0/623546885 >> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] conn(0x7f113c101d50 0x7f113c19a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 >> 172.21.15.73:0/623546885 conn(0x7f113c078340 msgr2=0x7f113c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 shutdown_connections 2024-06-06T07:27:47.877 INFO:teuthology.orchestra.run.smithi073.stderr:2024-06-06T07:27:47.873+0000 7f11413a4700 1 -- 172.21.15.73:0/623546885 wait complete. 2024-06-06T07:27:47.878 INFO:teuthology.orchestra.run.smithi073.stderr:listed 16 entries 2024-06-06T07:27:48.656 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:48 smithi073 ceph-mon[32303]: from='client.? 172.21.15.73:0/623546885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:48.684 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi073.front.sepia.ceph.com... 2024-06-06T07:27:48.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:27:48.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-06-06T07:27:48.724 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi073.stderr:ceph-fuse[91947]: fuse finished with error 0 and tester_r 0 2024-06-06T07:27:48.725 INFO:teuthology.orchestra.run:waiting for 300 2024-06-06T07:27:48.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:48 smithi136 ceph-mon[35901]: from='client.? 172.21.15.73:0/623546885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:48.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:48 smithi170 ceph-mon[36108]: from='client.? 172.21.15.73:0/623546885' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-06T07:27:49.774 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:49 smithi073 ceph-mon[32303]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-06-06T07:27:49.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:49 smithi136 ceph-mon[35901]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-06-06T07:27:49.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:49 smithi170 ceph-mon[36108]: pgmap v843: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.5 KiB/s wr, 16 op/s 2024-06-06T07:27:51.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:51 smithi073 ceph-mon[32303]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:51.802 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:51 smithi136 ceph-mon[35901]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:51.848 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:51 smithi170 ceph-mon[36108]: pgmap v844: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:53.775 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:53 smithi073 ceph-mon[32303]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:53.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:53 smithi136 ceph-mon[35901]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:53.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:53 smithi170 ceph-mon[36108]: pgmap v845: 129 pgs: 129 active+clean; 1.7 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.5 KiB/s wr, 11 op/s 2024-06-06T07:27:54.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T07:27:54.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:27:54.827 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-06T07:27:54.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:27:54.864 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2024-06-06T07:27:54.944 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-06T07:27:54.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:27:54.944 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-06T07:27:55.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-06T07:27:55.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2024-06-06T07:27:55.111 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-06T07:27:55.122 INFO:tasks.cephadm:Teardown begin 2024-06-06T07:27:55.122 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/cephadm.py", line 2240, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T07:27:55.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:27:55.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:27:55.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:27:55.260 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-06T07:27:55.260 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T07:27:55.277 DEBUG:teuthology.orchestra.run.smithi136:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T07:27:55.295 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-06T07:27:55.319 INFO:tasks.cephadm:Stopping all daemons... 2024-06-06T07:27:55.319 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-06T07:27:55.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a 2024-06-06T07:27:55.661 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:55 smithi073 systemd[1]: Stopping Ceph mon.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:27:55.662 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:55 smithi073 ceph-mon[32303]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 18 op/s 2024-06-06T07:27:55.801 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:55 smithi136 ceph-mon[35901]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 18 op/s 2024-06-06T07:27:55.847 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:55 smithi170 ceph-mon[36108]: pgmap v846: 129 pgs: 129 active+clean; 1.7 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.5 KiB/s wr, 18 op/s 2024-06-06T07:27:56.024 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:27:55.660+0000 7f12e2383700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:27:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a[32299]: 2024-06-06T07:27:55.660+0000 7f12e2383700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-06T07:27:56.025 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:55 smithi073 podman[119157]: 2024-06-06 07:27:55.743521803 +0000 UTC m=+0.214794643 container died ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-06-06T07:27:56.412 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:56 smithi073 podman[119157]: 2024-06-06 07:27:56.161363633 +0000 UTC m=+0.632636466 container cleanup ddd01b50ec7948067e613445e10e6e99a746bf1abf826327797993de345aa0ba (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T07:27:56.413 INFO:journalctl@ceph.mon.a.smithi073.stdout:Jun 06 07:27:56 smithi073 bash[119157]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-a 2024-06-06T07:27:56.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-06T07:27:56.685 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:27:56.685 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-06T07:27:56.686 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-06T07:27:56.686 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.b 2024-06-06T07:27:56.998 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:56 smithi136 systemd[1]: Stopping Ceph mon.b for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:27:57.301 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-b[35897]: 2024-06-06T07:27:56.997+0000 7f4144c37700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:27:57.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-b[35897]: 2024-06-06T07:27:56.997+0000 7f4144c37700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-06T07:27:57.302 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:57 smithi136 podman[80156]: 2024-06-06 07:27:57.018279287 +0000 UTC m=+0.123527109 container died 93d43cff2aa56148bec10e031a66852a2b87ecbd32d48c8694c2f74429bd02a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-06-06T07:27:57.642 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:57 smithi136 podman[80156]: 2024-06-06 07:27:57.398329303 +0000 UTC m=+0.503577124 container cleanup 93d43cff2aa56148bec10e031a66852a2b87ecbd32d48c8694c2f74429bd02a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-b, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux ) 2024-06-06T07:27:57.642 INFO:journalctl@ceph.mon.b.smithi136.stdout:Jun 06 07:27:57 smithi136 bash[80156]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-b 2024-06-06T07:27:57.768 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-06T07:27:57.873 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:27:57.873 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-06T07:27:57.874 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-06T07:27:57.874 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.c 2024-06-06T07:27:58.232 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:57 smithi170 systemd[1]: Stopping Ceph mon.c for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:27:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-c[36092]: 2024-06-06T07:27:58.231+0000 7fa206811700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:27:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-c[36092]: 2024-06-06T07:27:58.231+0000 7fa206811700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-06T07:27:58.597 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:58 smithi170 podman[78127]: 2024-06-06 07:27:58.435792896 +0000 UTC m=+0.326930211 container died 6dabbe5f80af62b3a3bb0bc1e6d10981a57e93cd1bb262dc4d66fd8464c13e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-06T07:27:59.059 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:58 smithi170 podman[78127]: 2024-06-06 07:27:58.840720082 +0000 UTC m=+0.731857396 container cleanup 6dabbe5f80af62b3a3bb0bc1e6d10981a57e93cd1bb262dc4d66fd8464c13e86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-c, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-06T07:27:59.059 INFO:journalctl@ceph.mon.c.smithi170.stdout:Jun 06 07:27:58 smithi170 bash[78127]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mon-c 2024-06-06T07:27:59.193 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-06T07:27:59.241 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:27:59.241 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-06T07:27:59.241 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-06T07:27:59.241 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x 2024-06-06T07:27:59.524 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:27:59 smithi073 systemd[1]: Stopping Ceph mgr.x for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:27:59.868 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:27:59 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x[32530]: 2024-06-06T07:27:59.556+0000 7f99041a1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:27:59.869 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:27:59 smithi073 podman[119278]: 2024-06-06 07:27:59.588260505 +0000 UTC m=+0.145655894 container died 80dbb3aa06b0d2940badff88c36416861d936b6d678021b5b970026f8c064cf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:00.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:27:59 smithi073 podman[119278]: 2024-06-06 07:27:59.983002315 +0000 UTC m=+0.540397706 container cleanup 80dbb3aa06b0d2940badff88c36416861d936b6d678021b5b970026f8c064cf2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:00.275 INFO:journalctl@ceph.mgr.x.smithi073.stdout:Jun 06 07:27:59 smithi073 bash[119278]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-x 2024-06-06T07:28:00.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-06T07:28:00.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:00.446 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-06T07:28:00.446 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-06T07:28:00.447 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.y 2024-06-06T07:28:00.783 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:28:00 smithi136 systemd[1]: Stopping Ceph mgr.y for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:01.051 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:28:00 smithi136 podman[80280]: 2024-06-06 07:28:00.782366791 +0000 UTC m=+0.143309725 container died b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-06T07:28:01.548 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:28:01 smithi136 podman[80280]: 2024-06-06 07:28:01.296093223 +0000 UTC m=+0.657036144 container cleanup b8c0e20d91ae10636781adf47fdd16ddac9c1a153c158b5ec120e846a66629c4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:01.548 INFO:journalctl@ceph.mgr.y.smithi136.stdout:Jun 06 07:28:01 smithi136 bash[80280]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-y 2024-06-06T07:28:01.687 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-06T07:28:01.722 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:01.722 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-06T07:28:01.722 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-06T07:28:01.722 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.z 2024-06-06T07:28:02.066 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:28:01 smithi170 systemd[1]: Stopping Ceph mgr.z for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:02.347 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:28:02 smithi170 podman[78249]: 2024-06-06 07:28:02.066607001 +0000 UTC m=+0.142051368 container died 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True) 2024-06-06T07:28:02.732 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:28:02 smithi170 podman[78249]: 2024-06-06 07:28:02.471775657 +0000 UTC m=+0.547220024 container cleanup 4d278b441d697a328693d0ca30039bed1bfc70c17d96c7f7ad2435e2d73a90dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-06T07:28:02.732 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Jun 06 07:28:02 smithi170 bash[78249]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mgr-z 2024-06-06T07:28:02.891 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-06T07:28:02.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:02.927 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-06T07:28:02.927 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-06T07:28:02.928 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.0 2024-06-06T07:28:03.232 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:02 smithi073 systemd[1]: Stopping Ceph osd.0 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:03.527 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0[47405]: 2024-06-06T07:28:03.231+0000 7f70ec69e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:03.527 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0[47405]: 2024-06-06T07:28:03.231+0000 7f70ec69e700 -1 osd.0 148 *** Got signal Terminated *** 2024-06-06T07:28:03.527 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0[47405]: 2024-06-06T07:28:03.231+0000 7f70ec69e700 -1 osd.0 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:09.141 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:08 smithi073 podman[119399]: 2024-06-06 07:28:08.850591342 +0000 UTC m=+5.734381154 container died 752566785d0dbc033d982652d6f882d9fd4ce6be1de52f35b3dd6f0824505713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-06T07:28:09.524 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:09 smithi073 podman[119399]: 2024-06-06 07:28:09.240212272 +0000 UTC m=+6.124002031 container cleanup 752566785d0dbc033d982652d6f882d9fd4ce6be1de52f35b3dd6f0824505713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528) 2024-06-06T07:28:09.525 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:09 smithi073 bash[119399]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0 2024-06-06T07:28:09.841 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:09 smithi073 podman[119416]: 2024-06-06 07:28:09.583276445 +0000 UTC m=+0.719997146 container remove 752566785d0dbc033d982652d6f882d9fd4ce6be1de52f35b3dd6f0824505713 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:10.172 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:09.911996137 +0000 UTC m=+0.017556167 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:10.172 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:10.05348065 +0000 UTC m=+0.159040724 container create eb1ff67683c9377daf05fda0fccb09abad840448016ec382e52315bc0c3ac06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-06-06T07:28:10.524 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:10.295620957 +0000 UTC m=+0.401180986 container init eb1ff67683c9377daf05fda0fccb09abad840448016ec382e52315bc0c3ac06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-06-06T07:28:10.524 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:10.298824529 +0000 UTC m=+0.404384562 container start eb1ff67683c9377daf05fda0fccb09abad840448016ec382e52315bc0c3ac06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-06T07:28:10.524 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:10.387313132 +0000 UTC m=+0.492873175 container attach eb1ff67683c9377daf05fda0fccb09abad840448016ec382e52315bc0c3ac06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:11.025 INFO:journalctl@ceph.osd.0.smithi073.stdout:Jun 06 07:28:10 smithi073 podman[119504]: 2024-06-06 07:28:10.674752505 +0000 UTC m=+0.780312535 container died eb1ff67683c9377daf05fda0fccb09abad840448016ec382e52315bc0c3ac06e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:11.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-06T07:28:11.417 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:11.418 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-06T07:28:11.418 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-06T07:28:11.418 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.1 2024-06-06T07:28:11.780 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:11 smithi136 systemd[1]: Stopping Ceph osd.1 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:12.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1[40429]: 2024-06-06T07:28:11.778+0000 7fb084305700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:12.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1[40429]: 2024-06-06T07:28:11.778+0000 7fb084305700 -1 osd.1 148 *** Got signal Terminated *** 2024-06-06T07:28:12.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1[40429]: 2024-06-06T07:28:11.778+0000 7fb084305700 -1 osd.1 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:17.581 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:17 smithi136 podman[80417]: 2024-06-06 07:28:17.30520383 +0000 UTC m=+5.697221000 container died 2dfa3cfd6fec42aea791e951f1d418a2994d593f8c57ba3f1a8486dbe4091e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-06T07:28:17.974 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:17 smithi136 podman[80417]: 2024-06-06 07:28:17.654684608 +0000 UTC m=+6.046701769 container cleanup 2dfa3cfd6fec42aea791e951f1d418a2994d593f8c57ba3f1a8486dbe4091e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-06-06T07:28:17.974 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:17 smithi136 bash[80417]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1 2024-06-06T07:28:18.299 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:17 smithi136 podman[80434]: 2024-06-06 07:28:17.973413082 +0000 UTC m=+0.666502477 container remove 2dfa3cfd6fec42aea791e951f1d418a2994d593f8c57ba3f1a8486dbe4091e6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:18.552 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:18 smithi136 podman[80521]: 2024-06-06 07:28:18.202420054 +0000 UTC m=+0.017739380 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:18.552 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:18 smithi136 podman[80521]: 2024-06-06 07:28:18.33305182 +0000 UTC m=+0.148371133 container create 2c65aef1b8fe6bbf4fbfc360ba2173e985e3ca0409efa36bdf176c800b7bb704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-06-06T07:28:19.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:18 smithi136 podman[80521]: 2024-06-06 07:28:18.610782916 +0000 UTC m=+0.426102234 container init 2c65aef1b8fe6bbf4fbfc360ba2173e985e3ca0409efa36bdf176c800b7bb704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-06T07:28:19.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:18 smithi136 podman[80521]: 2024-06-06 07:28:18.614129637 +0000 UTC m=+0.429448956 container start 2c65aef1b8fe6bbf4fbfc360ba2173e985e3ca0409efa36bdf176c800b7bb704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:28:19.052 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:18 smithi136 podman[80521]: 2024-06-06 07:28:18.685976088 +0000 UTC m=+0.501295420 container attach 2c65aef1b8fe6bbf4fbfc360ba2173e985e3ca0409efa36bdf176c800b7bb704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T07:28:19.464 INFO:journalctl@ceph.osd.1.smithi136.stdout:Jun 06 07:28:19 smithi136 podman[80521]: 2024-06-06 07:28:19.109339961 +0000 UTC m=+0.924659281 container died 2c65aef1b8fe6bbf4fbfc360ba2173e985e3ca0409efa36bdf176c800b7bb704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-06T07:28:19.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-06T07:28:19.852 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:19.853 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-06T07:28:19.853 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-06T07:28:19.853 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.2 2024-06-06T07:28:20.177 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:19 smithi170 systemd[1]: Stopping Ceph osd.2 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:20.597 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2[41209]: 2024-06-06T07:28:20.174+0000 7f4b7a1f0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:20.597 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2[41209]: 2024-06-06T07:28:20.174+0000 7f4b7a1f0700 -1 osd.2 148 *** Got signal Terminated *** 2024-06-06T07:28:20.597 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2[41209]: 2024-06-06T07:28:20.174+0000 7f4b7a1f0700 -1 osd.2 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:25.878 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:25 smithi170 podman[78387]: 2024-06-06 07:28:25.614293788 +0000 UTC m=+5.557611409 container died 5c6ca650074c65e7d891f23bcb8dc1e777dccf6adec1ba5951367f5d124ab819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:28:26.195 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:25 smithi170 podman[78387]: 2024-06-06 07:28:25.968245027 +0000 UTC m=+5.911562645 container cleanup 5c6ca650074c65e7d891f23bcb8dc1e777dccf6adec1ba5951367f5d124ab819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-06T07:28:26.195 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:25 smithi170 bash[78387]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2 2024-06-06T07:28:26.597 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:26 smithi170 podman[78402]: 2024-06-06 07:28:26.19502115 +0000 UTC m=+0.568875304 container remove 5c6ca650074c65e7d891f23bcb8dc1e777dccf6adec1ba5951367f5d124ab819 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:26.919 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:26 smithi170 podman[78497]: 2024-06-06 07:28:26.518487509 +0000 UTC m=+0.017700605 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:26.919 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:26 smithi170 podman[78497]: 2024-06-06 07:28:26.649474697 +0000 UTC m=+0.148687792 container create fcd477e1c38ba8aff5577aaa4f8dde3aa6bc02808ba0a401c4f1657ff0bb277e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-06T07:28:27.272 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:26 smithi170 podman[78497]: 2024-06-06 07:28:26.919369158 +0000 UTC m=+0.418582229 container init fcd477e1c38ba8aff5577aaa4f8dde3aa6bc02808ba0a401c4f1657ff0bb277e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-06T07:28:27.272 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:26 smithi170 podman[78497]: 2024-06-06 07:28:26.922990377 +0000 UTC m=+0.422203454 container start fcd477e1c38ba8aff5577aaa4f8dde3aa6bc02808ba0a401c4f1657ff0bb277e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-06T07:28:27.272 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:27 smithi170 podman[78497]: 2024-06-06 07:28:27.04429024 +0000 UTC m=+0.543503315 container attach fcd477e1c38ba8aff5577aaa4f8dde3aa6bc02808ba0a401c4f1657ff0bb277e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2-deactivate, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-06-06T07:28:27.552 INFO:journalctl@ceph.osd.2.smithi170.stdout:Jun 06 07:28:27 smithi170 podman[78497]: 2024-06-06 07:28:27.272580649 +0000 UTC m=+0.771793727 container died fcd477e1c38ba8aff5577aaa4f8dde3aa6bc02808ba0a401c4f1657ff0bb277e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True) 2024-06-06T07:28:27.901 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-06T07:28:27.940 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:27.940 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-06T07:28:27.940 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-06T07:28:27.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.3 2024-06-06T07:28:28.274 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:27 smithi073 systemd[1]: Stopping Ceph osd.3 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:28.774 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:28 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3[54187]: 2024-06-06T07:28:28.276+0000 7f3061928700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:28.774 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:28 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3[54187]: 2024-06-06T07:28:28.276+0000 7f3061928700 -1 osd.3 148 *** Got signal Terminated *** 2024-06-06T07:28:28.775 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:28 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3[54187]: 2024-06-06T07:28:28.276+0000 7f3061928700 -1 osd.3 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:29.913+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:30.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:30 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:30.413+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:31.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:30.880+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:31.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:31 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:31.416+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:31.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:31.450+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:32.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:31 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:31.884+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:32.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:31.861+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:32.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:31.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:32.775 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:32 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:32.284+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:32.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:32 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:32.378+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:32.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:32.482+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:33.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:32 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:32.867+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:33.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:32.947+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:33.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:32.907+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:33.716 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:33 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:33.324+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:33.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:33 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:33.330+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:33.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:33.516+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:34.026 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:33 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:33.841+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:34.027 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:33 smithi073 podman[119611]: 2024-06-06 07:28:33.715863931 +0000 UTC m=+5.573834620 container died a9ba292d5087bfc7b217800c5bc3d799f16ecaf4a343d4b529fb44c400fe8601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:34.326 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:34 smithi073 podman[119611]: 2024-06-06 07:28:34.110050489 +0000 UTC m=+5.968021177 container cleanup a9ba292d5087bfc7b217800c5bc3d799f16ecaf4a343d4b529fb44c400fe8601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:34.326 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:34 smithi073 bash[119611]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3 2024-06-06T07:28:34.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:33.984+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:34.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:33.955+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:34.735 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:34 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:34.325+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:34.736 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:34 smithi073 podman[119640]: 2024-06-06 07:28:34.370308728 +0000 UTC m=+0.657707315 container remove a9ba292d5087bfc7b217800c5bc3d799f16ecaf4a343d4b529fb44c400fe8601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-06-06T07:28:34.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:34 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:34.312+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:34.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:34 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:34.554+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:35.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:34 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:34.808+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:35.024 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:34 smithi073 podman[119730]: 2024-06-06 07:28:34.63827209 +0000 UTC m=+0.017016426 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:35.024 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:34 smithi073 podman[119730]: 2024-06-06 07:28:34.771243514 +0000 UTC m=+0.149987862 container create 221479ef4d13589f92a04a6be6338b6f4dd209e7e0fc6e8383a42012393bb650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD) 2024-06-06T07:28:35.024 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:35 smithi073 podman[119730]: 2024-06-06 07:28:35.023633924 +0000 UTC m=+0.402378256 container init 221479ef4d13589f92a04a6be6338b6f4dd209e7e0fc6e8383a42012393bb650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:35.275 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:35 smithi073 podman[119730]: 2024-06-06 07:28:35.027043236 +0000 UTC m=+0.405787561 container start 221479ef4d13589f92a04a6be6338b6f4dd209e7e0fc6e8383a42012393bb650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-06T07:28:35.275 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:35 smithi073 podman[119730]: 2024-06-06 07:28:35.149209514 +0000 UTC m=+0.527953849 container attach 221479ef4d13589f92a04a6be6338b6f4dd209e7e0fc6e8383a42012393bb650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-06T07:28:35.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:34 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:34.950+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:35.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:35.011+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:35.594 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:35 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:35.281+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:35.594 INFO:journalctl@ceph.osd.3.smithi073.stdout:Jun 06 07:28:35 smithi073 podman[119730]: 2024-06-06 07:28:35.328908949 +0000 UTC m=+0.707653279 container died 221479ef4d13589f92a04a6be6338b6f4dd209e7e0fc6e8383a42012393bb650 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-3-deactivate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2) 2024-06-06T07:28:35.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:35 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:35.318+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:35.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:35.550+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:35.853 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:35 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:35.839+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:35.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-06T07:28:35.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:35.917 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-06T07:28:35.918 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-06T07:28:35.918 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.4 2024-06-06T07:28:36.243 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:35 smithi136 systemd[1]: Stopping Ceph osd.4 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:36.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:35.986+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:36.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:35.927+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:36.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:36 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:36.272+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:36.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:36 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:36.343+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:36.552 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:36 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:36.242+0000 7f665268b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:36.552 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:36 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:36.242+0000 7f665268b700 -1 osd.4 148 *** Got signal Terminated *** 2024-06-06T07:28:36.552 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:36 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:36.242+0000 7f665268b700 -1 osd.4 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:36.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:36.571+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:37.230 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:36 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:36.794+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:37.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:36.981+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:36.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:37.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:37 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:37.229+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:37.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:37 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:37.314+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:37.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:37 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:37.588+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:38.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:37 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:37.758+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:38.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:38 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:38.008+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:38.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:37 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:37.919+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:38.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:38 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:38.244+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:38.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:38 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:38.294+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:38.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:38 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:38.630+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:39.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:38 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:38.766+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:39.302 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:38 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:38.981+0000 7f664ec7f700 -1 osd.4 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.422781+0000 front 2024-06-06T07:28:16.422565+0000 (oldest deadline 2024-06-06T07:28:38.122703+0000) 2024-06-06T07:28:39.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:39 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:39.016+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:39.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:38 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:38.968+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:39.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:39 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:39.227+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:39.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:39 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:39.227+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:39.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:39 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:39.337+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:39.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:39 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:39.337+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:39.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:39 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:39.590+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:40.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:39 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:39.760+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:40.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:39 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:39.760+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:40.291 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:39 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:39.932+0000 7f664ec7f700 -1 osd.4 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.422781+0000 front 2024-06-06T07:28:16.422565+0000 (oldest deadline 2024-06-06T07:28:38.122703+0000) 2024-06-06T07:28:40.291 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:39 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:39.887+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:40.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:39 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:39.979+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:40.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:39 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:39.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:40.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:39 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:39.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:40.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:40 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:40.290+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:40.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:40 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:40.290+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:40.621 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:40 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:40.576+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:40.716 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:40.277+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:40.716 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:40.277+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:41.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:40.715+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:41.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:40.715+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:41.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:40 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:40.921+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:41.303 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:40 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4[47321]: 2024-06-06T07:28:40.940+0000 7f664ec7f700 -1 osd.4 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.422781+0000 front 2024-06-06T07:28:16.422565+0000 (oldest deadline 2024-06-06T07:28:38.122703+0000) 2024-06-06T07:28:41.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:41 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:41.001+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:41.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:40 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:40.943+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:41.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:40 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:40.943+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:41.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:41 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:41.265+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:41.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:41 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:41.265+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:41.609 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:41 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:41.331+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:41.609 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:41 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:41.331+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:41.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:41 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:41.586+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:41.847 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:41 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:41.586+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:41.885 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:41 smithi136 podman[80626]: 2024-06-06 07:28:41.609114327 +0000 UTC m=+5.485479896 container died 91e6b58100ff83c96cfa507bbfd22bdc20900a4bfabc8fff55fe9514526a4a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=) 2024-06-06T07:28:42.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:41 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:41.674+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:42.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:41 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:41.674+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:42.261 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:41 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:41.884+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:42.261 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80626]: 2024-06-06 07:28:42.026317267 +0000 UTC m=+5.902682826 container cleanup 91e6b58100ff83c96cfa507bbfd22bdc20900a4bfabc8fff55fe9514526a4a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T07:28:42.261 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 bash[80626]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4 2024-06-06T07:28:42.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:42 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:42.048+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:42.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:41 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:41.896+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:42.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:41 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:41.896+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:42.551 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80642]: 2024-06-06 07:28:42.261251183 +0000 UTC m=+0.643802693 container remove 91e6b58100ff83c96cfa507bbfd22bdc20900a4bfabc8fff55fe9514526a4a97 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:28:42.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:42 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:42.373+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:42.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:42 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:42.373+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:42.684 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:42 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:42.274+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:42.684 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:42 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:42.274+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:42.833 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80728]: 2024-06-06 07:28:42.48574246 +0000 UTC m=+0.022626494 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:42.856 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80728]: 2024-06-06 07:28:42.587113383 +0000 UTC m=+0.123997405 container create a4b845e39401f83eb0064f40bf6c37c2db4279bbec00d8f2e2d0f9b2d5ec6308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T07:28:42.857 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80728]: 2024-06-06 07:28:42.832028837 +0000 UTC m=+0.368912875 container init a4b845e39401f83eb0064f40bf6c37c2db4279bbec00d8f2e2d0f9b2d5ec6308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:28:42.933 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:42 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:42.631+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:42.933 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:42 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:42.631+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:43.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:42 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:42.683+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:43.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:42 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:42.683+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:43.121 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:42 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:42.878+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:43.122 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80728]: 2024-06-06 07:28:42.835590353 +0000 UTC m=+0.372474378 container start a4b845e39401f83eb0064f40bf6c37c2db4279bbec00d8f2e2d0f9b2d5ec6308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-3d89acc) 2024-06-06T07:28:43.122 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:42 smithi136 podman[80728]: 2024-06-06 07:28:42.931618196 +0000 UTC m=+0.468502221 container attach a4b845e39401f83eb0064f40bf6c37c2db4279bbec00d8f2e2d0f9b2d5ec6308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0) 2024-06-06T07:28:43.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:43.089+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:43.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:43.089+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:43.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:42 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:42.932+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:43.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:42 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:42.932+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:43.385 INFO:journalctl@ceph.osd.4.smithi136.stdout:Jun 06 07:28:43 smithi136 podman[80728]: 2024-06-06 07:28:43.121659939 +0000 UTC m=+0.658543968 container died a4b845e39401f83eb0064f40bf6c37c2db4279bbec00d8f2e2d0f9b2d5ec6308 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-06T07:28:43.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:43 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:43.273+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:43.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:43 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:43.273+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:43.711 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:43 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:43.384+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:43.711 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:43 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:43.384+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:43.727 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-06T07:28:43.765 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:43.765 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-06T07:28:43.765 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-06T07:28:43.765 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.5 2024-06-06T07:28:43.920 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:43.651+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:43.920 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:43.651+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:43.920 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:43 smithi170 systemd[1]: Stopping Ceph osd.5 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:44.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:43 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:43.711+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:44.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:43 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:43.711+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:44.051 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:43 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:43.876+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:44.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:44.062+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:44.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:44.062+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:44.348 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:44.074+0000 7f6113478700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:44.348 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:44.074+0000 7f6113478700 -1 osd.5 148 *** Got signal Terminated *** 2024-06-06T07:28:44.348 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:44.074+0000 7f6113478700 -1 osd.5 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:44.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:43.918+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:44.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:43 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:43.918+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:44.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:44.231+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:46.061 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:44.231+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:46.062 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:44.684+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:46.062 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:44.684+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:46.063 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:44 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:44.335+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:46.063 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:44 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:44.335+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:46.063 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:44.693+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:46.063 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:44 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:44.693+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:46.063 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:44.915+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:44 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:44.915+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:44 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:44.848+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:45.214+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:45.214+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:45.087+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:46.064 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:45.087+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:46.065 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:45 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:45.323+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:46.065 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:45 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:45.323+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:46.065 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:45.712+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:46.065 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:45.712+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:46.065 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:45 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:45.801+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:46.097 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:45.713+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:46.097 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:45.713+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:46.097 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:45.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:46.097 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:45 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:45.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:46.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:46.256+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:46.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:46.256+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:46.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:46.113+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:46.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:46.113+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:46.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:46 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:46.308+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:46.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:46 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:46.308+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:47.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:46.689+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:47.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:46 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:46.689+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:47.094 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:46.724+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:47.094 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:46.724+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:47.094 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:46.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:47.094 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:46 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:46.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:47.297 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:46 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:46.842+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:47.297 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:46 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:46.842+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:47.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:47.093+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:47.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:47.093+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:47.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:47 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:47.297+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:47.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:47 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:47.297+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:47.698 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:47.277+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:47.698 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:47.277+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:48.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:47.697+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:48.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:47 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:47.697+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:48.089 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:47.691+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:48.089 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:47.691+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:48.090 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:47.893+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:48.090 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:47 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:47.893+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:48.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:47 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:47.834+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:48.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:47 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:47.834+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:48.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:48.088+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:48.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:48.088+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:48.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:48.088+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:48.703 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:48.309+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:48.703 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:48.309+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:48.703 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:48.309+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:48.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:48 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:48.309+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:48.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:48 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:48.309+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:48.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:48 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:48.309+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:49.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:48.702+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:49.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:48.702+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:49.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:48 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:48.702+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:49.097 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:48.731+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.355999+0000 front 2024-06-06T07:28:05.356017+0000 (oldest deadline 2024-06-06T07:28:31.255897+0000) 2024-06-06T07:28:49.097 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5[47929]: 2024-06-06T07:28:48.731+0000 7f610ba64700 -1 osd.5 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:11.256385+0000 front 2024-06-06T07:28:11.256500+0000 (oldest deadline 2024-06-06T07:28:41.257240+0000) 2024-06-06T07:28:49.098 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:48.918+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:49.098 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:48 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:48.918+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:49.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:48 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:48.857+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:49.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:48 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:48.857+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:49.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:49 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:49.118+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:49.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:49 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:49.118+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:49.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:49 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:49.118+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:49.753 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:49.295+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:49.753 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:49.295+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:49.753 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:49.295+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:49.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:49 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:49.338+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:49.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:49 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:49.338+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:49.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:49 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:49.338+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:49.943 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:49 smithi170 podman[78627]: 2024-06-06 07:28:49.689671846 +0000 UTC m=+5.715860169 container died 5cac18921775625a966d63c83db0b06238793638b75461344fa5db19f31510e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-06T07:28:49.944 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:49 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:49.897+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:49.944 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:49 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:49.897+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:50.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:49.752+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:50.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:49.752+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:50.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:49 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:49.752+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:50.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:49 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:49.875+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:50.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:49 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:49.875+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:50.347 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78627]: 2024-06-06 07:28:50.050977467 +0000 UTC m=+6.077165785 container cleanup 5cac18921775625a966d63c83db0b06238793638b75461344fa5db19f31510e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:50.347 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 bash[78627]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5 2024-06-06T07:28:50.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:50.139+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:50.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:50.139+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:50.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:50.139+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:50.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:50.271+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:50.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:50.271+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:50.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:50.271+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:50.687 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78644]: 2024-06-06 07:28:50.36935847 +0000 UTC m=+0.671152100 container remove 5cac18921775625a966d63c83db0b06238793638b75461344fa5db19f31510e2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:28:50.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:50.324+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:50.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:50.324+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:50.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:50.324+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:50.990 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:50.864+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:50.990 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:50.864+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:50.990 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:50 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:50.864+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:50.994 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78733]: 2024-06-06 07:28:50.590034308 +0000 UTC m=+0.017528007 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:50.994 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78733]: 2024-06-06 07:28:50.703803515 +0000 UTC m=+0.131297191 container create 1f2ac15292fd955b927e06ad56f39a68da26d330dfcaadec4987278e67a432e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:28:50.995 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78733]: 2024-06-06 07:28:50.990019608 +0000 UTC m=+0.417513303 container init 1f2ac15292fd955b927e06ad56f39a68da26d330dfcaadec4987278e67a432e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T07:28:51.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:50.731+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:51.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:50.731+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:51.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:50.731+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:51.293 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:50 smithi170 podman[78733]: 2024-06-06 07:28:50.993393158 +0000 UTC m=+0.420886842 container start 1f2ac15292fd955b927e06ad56f39a68da26d330dfcaadec4987278e67a432e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T07:28:51.293 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:51 smithi170 podman[78733]: 2024-06-06 07:28:51.090122806 +0000 UTC m=+0.517616508 container attach 1f2ac15292fd955b927e06ad56f39a68da26d330dfcaadec4987278e67a432e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:51.293 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:51.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:51.293 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:51.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:51.294 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:51.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:51.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:50.916+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:51.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:50 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:50.916+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:51.597 INFO:journalctl@ceph.osd.5.smithi170.stdout:Jun 06 07:28:51 smithi170 podman[78733]: 2024-06-06 07:28:51.293242701 +0000 UTC m=+0.720736392 container died 1f2ac15292fd955b927e06ad56f39a68da26d330dfcaadec4987278e67a432e6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-06T07:28:51.695 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:51.282+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:51.695 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:51.282+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:51.695 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:51.282+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:51.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:51 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:51.313+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:51.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:51 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:51.313+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:51.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:51 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:51.313+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:51.919 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-06T07:28:51.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:28:51.958 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-06T07:28:51.958 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-06T07:28:51.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.6 2024-06-06T07:28:51.985 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:51.694+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:51.985 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:51.694+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:51.985 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:51 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:51.694+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:52.273 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 systemd[1]: Stopping Ceph osd.6 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:28:52.274 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.272+0000 7f95de102700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:28:52.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:51 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:51.937+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:52.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:51 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:51.937+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:52.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:51.882+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:52.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:51.882+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:52.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:51 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:51.882+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:52.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:52.123+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:52.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:52.123+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:52.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:52.123+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:52.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.272+0000 7f95de102700 -1 osd.6 148 *** Got signal Terminated *** 2024-06-06T07:28:52.524 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.272+0000 7f95de102700 -1 osd.6 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:28:52.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.288+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:52.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.288+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:52.525 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:52.288+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:52.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:52 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:52.355+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:52.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:52 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:52.355+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:52.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:52 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:52.355+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:53.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:52.711+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:53.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:52.711+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:53.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:52 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:52.711+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:53.144 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:52.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:53.144 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:52.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:53.144 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:52 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:52.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:53.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:52 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:52.898+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:53.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:52 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:52.898+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:53.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:53.143+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:53.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:53.143+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:53.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:53.143+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:53.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:53.319+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:53.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:53.319+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:53.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:53.319+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:53.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:53 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:53.401+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:53.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:53 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:53.401+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:53.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:53 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:53.401+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:54.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:53.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:54.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:53.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:54.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:53 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:53.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:54.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:53 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:53.871+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:54.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:53 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:53.871+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:54.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:54.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:54.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:54.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:54.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:54.112+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:54.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:53.909+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:54.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:53.909+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:54.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:53.909+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:54.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:53 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:53.909+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:54.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:54.366+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:54.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:54.366+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:54.738 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:54.366+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:54.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:54.413+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:54.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:54.413+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:54.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:54.413+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:54.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:54.413+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:55.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:54.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:55.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:54.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:55.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:54 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:54.737+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:55.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:54.898+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:55.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:54 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:54.898+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:55.096+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:55.096+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:55.096+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:54.900+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:54.900+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:55.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:54.900+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:55.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:54 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:54.900+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:55.741 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:55.338+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:55.741 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:55.338+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:55.742 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:55.338+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:55.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:55.430+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:55.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:55.430+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:55.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:55.430+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:55.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:55.430+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:56.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:55.741+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:56.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:55.741+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:56.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:55.741+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:56.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:55.885+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:56.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:55 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:55.885+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:56.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:56.073+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:56.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:56.073+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:56.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:56.073+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:56.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:55.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:56.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:55.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:56.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:55.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:56.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:55 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:55.870+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:56.774 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:56.359+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.701876+0000 front 2024-06-06T07:28:06.701904+0000 (oldest deadline 2024-06-06T07:28:31.302613+0000) 2024-06-06T07:28:56.774 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:56.359+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.405493+0000 front 2024-06-06T07:28:16.405156+0000 (oldest deadline 2024-06-06T07:28:39.205855+0000) 2024-06-06T07:28:56.775 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:56.360+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.107252+0000 front 2024-06-06T07:28:25.107245+0000 (oldest deadline 2024-06-06T07:28:48.007080+0000) 2024-06-06T07:28:56.775 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6[60740]: 2024-06-06T07:28:56.360+0000 7f95d66ee700 -1 osd.6 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.108156+0000 front 2024-06-06T07:28:32.108078+0000 (oldest deadline 2024-06-06T07:28:55.608204+0000) 2024-06-06T07:28:56.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:56.446+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:56.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:56.446+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:56.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:56.446+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:56.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:56.446+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:57.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:56.785+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:57.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:56.785+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:57.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:56.785+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:57.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:56 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:56.785+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:28:57.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:56.838+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:57.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:56 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:56.838+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:57.080+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:57.080+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:57.080+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:56.885+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:56.885+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:57.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:56.885+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:57.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:56 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:56.885+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:57.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:57.450+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:57.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:57.450+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:57.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:57.450+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:57.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:57.450+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:58.025 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:57 smithi073 podman[119834]: 2024-06-06 07:28:57.748998498 +0000 UTC m=+5.595056095 container died 36724d157a1b05eefb834d62107be8272e35b1ed98eb7bacad160b8fcbe559e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T07:28:58.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:57 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:57.768+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:58.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:57 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:57.768+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:58.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:57 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:57.768+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:58.025 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:57 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:57.768+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:28:58.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:57.859+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:58.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:57 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:57.859+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:58.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.034+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:58.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.034+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:58.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.034+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:58.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:57.905+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:58.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:57.905+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:58.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:57.905+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:58.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:57 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:57.905+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:58.424 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:58 smithi073 podman[119834]: 2024-06-06 07:28:58.172960194 +0000 UTC m=+6.019017781 container cleanup 36724d157a1b05eefb834d62107be8272e35b1ed98eb7bacad160b8fcbe559e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T07:28:58.424 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:58 smithi073 bash[119834]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6 2024-06-06T07:28:58.774 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:58 smithi073 podman[119861]: 2024-06-06 07:28:58.424529023 +0000 UTC m=+0.677665399 container remove 36724d157a1b05eefb834d62107be8272e35b1ed98eb7bacad160b8fcbe559e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-06T07:28:58.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:58.467+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:58.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:58.467+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:58.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:58.467+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:58.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:58.467+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:59.136 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:58 smithi073 podman[119952]: 2024-06-06 07:28:58.756132729 +0000 UTC m=+0.017136946 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:28:59.137 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:58 smithi073 podman[119952]: 2024-06-06 07:28:58.892404789 +0000 UTC m=+0.153408999 container create 80e40ceace66cd522c5c26a0bf573e31ee9226d272a4a8f2c457ff8ae00de76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-06-06T07:28:59.137 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:58 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:58.791+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:59.137 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:58 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:58.791+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:59.137 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:58 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:58.791+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:59.137 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:58 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:58.791+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:28:59.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:58.872+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:28:59.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:58.872+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:28:59.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:58.872+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:28:59.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.996+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:28:59.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.996+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:28:59.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:58.996+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:28:59.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:58.937+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:28:59.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:58.937+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:28:59.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:58.937+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:28:59.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:58 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:58.937+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:28:59.429 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:59 smithi073 podman[119952]: 2024-06-06 07:28:59.136360254 +0000 UTC m=+0.397364468 container init 80e40ceace66cd522c5c26a0bf573e31ee9226d272a4a8f2c457ff8ae00de76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-06T07:28:59.429 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:59 smithi073 podman[119952]: 2024-06-06 07:28:59.140488438 +0000 UTC m=+0.401492656 container start 80e40ceace66cd522c5c26a0bf573e31ee9226d272a4a8f2c457ff8ae00de76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:28:59.429 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:59 smithi073 podman[119952]: 2024-06-06 07:28:59.253187794 +0000 UTC m=+0.514192004 container attach 80e40ceace66cd522c5c26a0bf573e31ee9226d272a4a8f2c457ff8ae00de76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T07:28:59.714 INFO:journalctl@ceph.osd.6.smithi073.stdout:Jun 06 07:28:59 smithi073 podman[119952]: 2024-06-06 07:28:59.430015189 +0000 UTC m=+0.691019406 container died 80e40ceace66cd522c5c26a0bf573e31ee9226d272a4a8f2c457ff8ae00de76c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-06T07:28:59.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:59.471+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:28:59.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:59.471+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:28:59.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:59.471+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:28:59.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:28:59.471+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:28:59.990 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:59 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:59.801+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:28:59.990 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:59 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:59.801+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:28:59.991 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:59 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:59.801+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:28:59.991 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:28:59 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:28:59.801+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:00.011 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-06T07:29:00.092 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:00.092 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-06T07:29:00.092 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-06T07:29:00.092 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.7 2024-06-06T07:29:00.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:59.902+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:00.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:59.902+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:00.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:28:59 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:28:59.902+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:00.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 systemd[1]: Stopping Ceph osd.7 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:59.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:59.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:28:59.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:59.934+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:59.934+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:00.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:59.934+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:00.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:28:59 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:28:59.934+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:00.801 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.403+0000 7fa4a6f8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:00.801 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.403+0000 7fa4a6f8b700 -1 osd.7 148 *** Got signal Terminated *** 2024-06-06T07:29:00.802 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.403+0000 7fa4a6f8b700 -1 osd.7 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:29:00.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:00.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:00.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:00.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:00.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:00.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:00.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:00.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:01.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:00.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:01.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:00.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:01.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:00.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:01.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:00 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:00.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:01.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.909+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:01.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.909+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:01.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:00 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:00.909+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:01.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:01.000+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:01.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:01.000+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:01.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:01.000+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:01.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:00 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:00.951+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:01.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:00 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:00.951+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:01.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:00 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:00.951+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:01.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:00 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:00.951+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:01.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:01.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:01.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:01.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:01.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:01.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:01.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:01.518+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:02.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:01 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:01.834+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:02.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:01 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:01.834+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:02.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:01 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:01.834+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:02.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:01 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:01.834+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:02.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:01.910+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:02.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:01.910+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:02.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:01.911+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:02.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:01 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:01.911+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:40.768199+0000 front 2024-06-06T07:28:40.768177+0000 (oldest deadline 2024-06-06T07:29:01.868330+0000) 2024-06-06T07:29:02.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:02.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:02.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:02.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:02.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:02 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:02.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:02.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:01.987+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:02.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:01.987+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:02.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:01.987+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:02.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:01 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:01.987+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:02.958 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:02.563+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:02.959 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:02.563+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:02.959 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:02.563+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:02.959 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:02.563+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:03.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:02 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:02.822+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:03.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:02 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:02.822+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:03.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:02 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:02.822+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:03.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:02 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:02.822+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:03.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:02 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:02.822+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:03.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:02.957+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:03.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:02.957+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:03.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:02.957+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:03.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:02.957+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:40.768199+0000 front 2024-06-06T07:28:40.768177+0000 (oldest deadline 2024-06-06T07:29:01.868330+0000) 2024-06-06T07:29:03.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:03.025+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:03.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:03.025+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:03.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:03.025+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:03.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.023+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:03.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.023+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:03.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.023+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:03.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.023+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:03.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.023+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:03.918 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:03.562+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:03.918 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:03.562+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:03.918 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:03.562+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:03.918 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:03.562+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:04.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:03.835+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:04.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:03.835+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:04.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:03.835+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:04.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:03.835+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:04.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:03 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:03.835+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:04.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:03.917+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:04.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:03.917+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:04.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:03.917+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:04.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:03 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:03.917+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:40.768199+0000 front 2024-06-06T07:28:40.768177+0000 (oldest deadline 2024-06-06T07:29:01.868330+0000) 2024-06-06T07:29:04.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:04.017+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:04.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:04.017+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:04.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:04.017+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:04.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:04.017+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:04.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:04.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:04.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:04.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:04.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:03 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:03.978+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:04.923 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:04.605+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:04.923 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:04.605+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:04.923 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:04.605+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:04.923 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:04.605+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:04.923 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:04.605+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:05.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:04.829+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:05.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:04.829+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:05.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:04.829+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:05.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:04.829+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:05.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:04 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:04.829+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:05.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:04.921+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.264124+0000 front 2024-06-06T07:28:16.264218+0000 (oldest deadline 2024-06-06T07:28:39.764217+0000) 2024-06-06T07:29:05.301 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:04.921+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:20.265451+0000 front 2024-06-06T07:28:20.265462+0000 (oldest deadline 2024-06-06T07:28:46.165441+0000) 2024-06-06T07:29:05.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:04.921+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:33.167313+0000 front 2024-06-06T07:28:33.167399+0000 (oldest deadline 2024-06-06T07:28:57.867214+0000) 2024-06-06T07:29:05.302 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:04 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7[53962]: 2024-06-06T07:29:04.921+0000 7fa4a357f700 -1 osd.7 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:40.768199+0000 front 2024-06-06T07:28:40.768177+0000 (oldest deadline 2024-06-06T07:29:01.868330+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.009+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.009+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.009+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.009+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:04.974+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:05.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:04.974+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:05.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:04.974+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:05.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:04.974+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:05.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:04 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:04.974+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:05.874 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:05.598+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:05.874 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:05.598+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:05.874 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:05.598+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:05.874 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:05.598+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:05.874 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:05.598+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:06.188 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:05 smithi136 podman[80832]: 2024-06-06 07:29:05.874179465 +0000 UTC m=+5.582580968 container died 098018c69f4a4cb239fac3c4f53575166e86dd297b95c549acc7e1130d5c1142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T07:29:06.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:05.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:06.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:05.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:06.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:05.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:06.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:05.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:06.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:05 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:05.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:06.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.977+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:06.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.977+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:06.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.977+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:06.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:05.977+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:06.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:05.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:06.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:05.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:06.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:05.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:06.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:05.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:06.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:05 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:05.933+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:06.489 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:06 smithi136 podman[80832]: 2024-06-06 07:29:06.287573128 +0000 UTC m=+5.995974609 container cleanup 098018c69f4a4cb239fac3c4f53575166e86dd297b95c549acc7e1130d5c1142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T07:29:06.489 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:06 smithi136 bash[80832]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7 2024-06-06T07:29:06.801 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:06 smithi136 podman[80846]: 2024-06-06 07:29:06.488975978 +0000 UTC m=+0.615201097 container remove 098018c69f4a4cb239fac3c4f53575166e86dd297b95c549acc7e1130d5c1142 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-06T07:29:06.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:06.554+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:06.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:06.554+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:06.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:06.554+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:06.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:06.554+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:06.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:06 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:06.554+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:07.185 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:06 smithi136 podman[80931]: 2024-06-06 07:29:06.889897018 +0000 UTC m=+0.100177872 container create 928b3983949b95bee65c633f7de58f7898fb2b56d9065149e0eac2b037a13e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528) 2024-06-06T07:29:07.185 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:06 smithi136 podman[80931]: 2024-06-06 07:29:06.807054124 +0000 UTC m=+0.017335013 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:29:07.185 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:07 smithi136 podman[80931]: 2024-06-06 07:29:07.117563908 +0000 UTC m=+0.327844770 container init 928b3983949b95bee65c633f7de58f7898fb2b56d9065149e0eac2b037a13e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T07:29:07.185 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:07 smithi136 podman[80931]: 2024-06-06 07:29:07.121307148 +0000 UTC m=+0.331588011 container start 928b3983949b95bee65c633f7de58f7898fb2b56d9065149e0eac2b037a13e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:29:07.185 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:07 smithi136 podman[80931]: 2024-06-06 07:29:07.184306106 +0000 UTC m=+0.394586971 container attach 928b3983949b95bee65c633f7de58f7898fb2b56d9065149e0eac2b037a13e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:07.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:06.912+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:07.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:06.912+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:07.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:06.912+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:07.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:06.912+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:07.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:06 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:06.912+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:07.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:06.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:07.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:06.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:07.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:06.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:07.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:06.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:07.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:06.952+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:07.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:06.952+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:07.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:06.952+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:07.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:06.952+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:07.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:06 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:06.952+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:07.549 INFO:journalctl@ceph.osd.7.smithi136.stdout:Jun 06 07:29:07 smithi136 podman[80931]: 2024-06-06 07:29:07.414150287 +0000 UTC m=+0.624431151 container died 928b3983949b95bee65c633f7de58f7898fb2b56d9065149e0eac2b037a13e35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:07.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:07.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:07.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:07.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:07.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:07.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:07.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:07.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:07.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:07 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:07.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:08.042 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-06T07:29:08.082 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:08.082 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-06T07:29:08.083 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-06T07:29:08.083 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.8 2024-06-06T07:29:08.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:07.923+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:08.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:07.923+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:08.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:07.923+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:08.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:07.923+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:08.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:07 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:07.923+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:08.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:07.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:08.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:07.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:08.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:07.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:08.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:07.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:07.939+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:07.939+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:07.939+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:07.939+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:07 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:07.939+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:08.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 systemd[1]: Stopping Ceph osd.8 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:08.551 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:08.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:08.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:08.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:08.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:08.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:08.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:08.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:08.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:08 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:08.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:08.847 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.388+0000 7fc1a7b8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:08.847 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.388+0000 7fc1a7b8b700 -1 osd.8 148 *** Got signal Terminated *** 2024-06-06T07:29:08.847 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.388+0000 7fc1a7b8b700 -1 osd.8 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:29:09.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:08.898+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:09.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:08.898+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:09.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:08.898+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:09.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:08.898+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:09.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:08 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:08.898+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:09.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:08.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:09.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:08.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:09.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:08.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:09.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:08.937+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:09.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.904+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:09.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.904+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:09.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.904+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:09.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.904+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:09.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:08 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:08.904+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:09.494 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:09.493+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:09.495 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:09.493+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:09.495 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:09.493+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:09.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:09.493+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:09.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:09 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:09.493+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:10.249 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:09.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:10.249 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:09.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:10.249 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:09.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:10.249 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:09.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:10.250 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:09.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:10.250 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:09.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:10.250 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:09.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:10.250 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:09.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:10.250 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:09 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:09.920+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:10.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:09.904+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:10.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:09.904+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:10.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:09.904+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:10.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:09.904+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:10.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:09 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:09.904+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:10.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:10.528+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:10.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:10.528+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:10.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:10.528+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:10.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:10.528+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:10.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:10 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:10.528+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:11.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:10.888+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:11.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:10.888+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:11.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:10.888+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:11.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:10.888+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:11.269 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:11.270 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:11.270 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:11.270 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:11.270 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:11.270 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:10 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:10.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.968838+0000 front 2024-06-06T07:28:47.968780+0000 (oldest deadline 2024-06-06T07:29:10.868820+0000) 2024-06-06T07:29:11.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:11.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:11.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:11.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:11.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:11.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:10 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:10.911+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:11.805 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:11.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:11.805 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:11.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:11.805 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:11.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:11.805 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:11.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:11.805 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:11 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:11.519+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:12.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:12.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:12.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:12.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:12.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:12.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:11 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:11.903+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:12.347 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:11.899+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.968838+0000 front 2024-06-06T07:28:47.968780+0000 (oldest deadline 2024-06-06T07:29:10.868820+0000) 2024-06-06T07:29:12.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:11.852+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:12.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:11.852+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:12.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:11.852+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:12.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:11.852+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:12.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:11 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:11.852+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:12.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:12 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:12.520+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:13.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:12 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:12.929+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:13.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:12.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:13.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:12.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:13.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:12.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:12.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:12.897+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:08.063688+0000 front 2024-06-06T07:28:08.063608+0000 (oldest deadline 2024-06-06T07:28:30.963658+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.665212+0000 front 2024-06-06T07:28:15.665196+0000 (oldest deadline 2024-06-06T07:28:39.765154+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:25.066216+0000 front 2024-06-06T07:28:25.066242+0000 (oldest deadline 2024-06-06T07:28:50.366350+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:30.367043+0000 front 2024-06-06T07:28:30.367056+0000 (oldest deadline 2024-06-06T07:28:53.866819+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:37.367992+0000 front 2024-06-06T07:28:37.368071+0000 (oldest deadline 2024-06-06T07:29:02.667926+0000) 2024-06-06T07:29:13.348 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:12 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8[54441]: 2024-06-06T07:29:12.883+0000 7fc1a417f700 -1 osd.8 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.968838+0000 front 2024-06-06T07:28:47.968780+0000 (oldest deadline 2024-06-06T07:29:10.868820+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:13.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:13 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:13.550+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:14.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:14.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:14.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:14.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:14.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:14.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:13 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:13.892+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:14.291 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:13 smithi170 podman[78837]: 2024-06-06 07:29:13.992127236 +0000 UTC m=+5.711068527 container died a6d9a45fdb87bb434754aeb645cde3d9a7c0912976a3b1fd506dc14e98e18632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:29:14.292 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:13 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:13.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:14.292 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:13 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:13.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:14.292 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:13 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:13.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:14.292 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:13 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:13.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:14.292 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:13 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:13.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:14.597 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:14 smithi170 podman[78837]: 2024-06-06 07:29:14.372478368 +0000 UTC m=+6.091419655 container cleanup a6d9a45fdb87bb434754aeb645cde3d9a7c0912976a3b1fd506dc14e98e18632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:14.597 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:14 smithi170 bash[78837]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8 2024-06-06T07:29:14.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:14.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:14.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:14.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:14.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:14.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:14 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:14.509+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:14.952 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:14 smithi170 podman[78864]: 2024-06-06 07:29:14.625710146 +0000 UTC m=+0.632372504 container remove a6d9a45fdb87bb434754aeb645cde3d9a7c0912976a3b1fd506dc14e98e18632 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-06T07:29:15.220 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:14 smithi170 podman[78955]: 2024-06-06 07:29:14.854823851 +0000 UTC m=+0.017231759 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:29:15.268 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:14 smithi170 podman[78955]: 2024-06-06 07:29:14.966599841 +0000 UTC m=+0.129007730 container create e22ea9e5ede863addf714d17d4638a7f348359c4399b35aa1ff7f3623890334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:15.268 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:14.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:15.268 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:14.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:15.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:14.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:15.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:14.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:15.269 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:14 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:14.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:15.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:14 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:14.858+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:15.512 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:15 smithi170 podman[78955]: 2024-06-06 07:29:15.219566944 +0000 UTC m=+0.381974842 container init e22ea9e5ede863addf714d17d4638a7f348359c4399b35aa1ff7f3623890334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-06-06T07:29:15.512 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:15 smithi170 podman[78955]: 2024-06-06 07:29:15.2234789 +0000 UTC m=+0.385886791 container start e22ea9e5ede863addf714d17d4638a7f348359c4399b35aa1ff7f3623890334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD) 2024-06-06T07:29:15.512 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:15 smithi170 podman[78955]: 2024-06-06 07:29:15.319222901 +0000 UTC m=+0.481630810 container attach e22ea9e5ede863addf714d17d4638a7f348359c4399b35aa1ff7f3623890334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:15.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:15 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:15.504+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:15.822 INFO:journalctl@ceph.osd.8.smithi170.stdout:Jun 06 07:29:15 smithi170 podman[78955]: 2024-06-06 07:29:15.511742059 +0000 UTC m=+0.674149948 container died e22ea9e5ede863addf714d17d4638a7f348359c4399b35aa1ff7f3623890334a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-06T07:29:16.097 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:16.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:16.097 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:16.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:16.097 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:16.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:16.097 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:16.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:16.097 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:16 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:16.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:16.153 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-06T07:29:16.193 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:16.193 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-06T07:29:16.193 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-06T07:29:16.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.9 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:16.220 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:15 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:15.870+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:16.493 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 systemd[1]: Stopping Ceph osd.9 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:16.493 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.491+0000 7f46c2dd9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:16.493 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.491+0000 7f46c2dd9700 -1 osd.9 148 *** Got signal Terminated *** 2024-06-06T07:29:16.493 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.491+0000 7f46c2dd9700 -1 osd.9 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:29:16.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:16.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:16.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:16.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:16.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:16.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:16 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:16.503+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:17.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:17.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:17.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:17.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:17.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:17.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:16 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:16.861+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:17.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:17.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:17.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:17.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:17.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:17.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:17.548+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:18.274 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:18.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:18.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:18.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:18.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:18.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:17 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:17.813+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:18.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.961+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:18.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.961+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:18.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.961+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:18.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.961+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:18.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:17 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:17.961+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:18.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:18.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:18.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:18.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:18.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:18.803 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:18 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:18.530+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:19.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:18 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:18.857+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:19.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:18.945+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:19.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:18.945+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:19.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:18.945+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:19.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:18.945+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:19.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:18 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:18.945+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:19.552 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:19 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:19.542+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:20.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:20.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:20.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:20.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:20.275 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:20.276 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:19 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:19.820+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:20.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.956+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:20.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.956+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:20.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.956+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:20.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.957+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:20.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.957+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:20.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:19 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:19.957+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:21.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:20 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:20.567+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:21.277 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:03.323139+0000 front 2024-06-06T07:28:03.323119+0000 (oldest deadline 2024-06-06T07:28:31.523981+0000) 2024-06-06T07:29:21.278 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:15.525491+0000 front 2024-06-06T07:28:15.525437+0000 (oldest deadline 2024-06-06T07:28:39.025445+0000) 2024-06-06T07:29:21.278 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:24.227157+0000 front 2024-06-06T07:28:24.227120+0000 (oldest deadline 2024-06-06T07:28:48.327176+0000) 2024-06-06T07:29:21.279 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:32.928543+0000 front 2024-06-06T07:28:32.928583+0000 (oldest deadline 2024-06-06T07:28:55.828558+0000) 2024-06-06T07:29:21.280 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:38.730017+0000 front 2024-06-06T07:28:38.730113+0000 (oldest deadline 2024-06-06T07:29:02.229797+0000) 2024-06-06T07:29:21.281 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:20 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9[67428]: 2024-06-06T07:29:20.810+0000 7f46bb3c5700 -1 osd.9 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.332446+0000 front 2024-06-06T07:28:47.332395+0000 (oldest deadline 2024-06-06T07:29:10.232061+0000) 2024-06-06T07:29:21.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:21.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:21.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:21.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:21.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:21.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:20 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:20.970+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:22.051 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:22.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:21 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:21.573+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:22.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:22.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:22.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:22.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:22.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:22.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.002+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:22.377 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:22 smithi073 podman[120058]: 2024-06-06 07:29:22.091474455 +0000 UTC m=+5.708258616 container died 50d1c8632d663c1f85e0df722144f95c35e377f57a7bd40d2bf84de778ee4ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-06T07:29:22.652 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:22 smithi073 podman[120058]: 2024-06-06 07:29:22.450451416 +0000 UTC m=+6.067235577 container cleanup 50d1c8632d663c1f85e0df722144f95c35e377f57a7bd40d2bf84de778ee4ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-06T07:29:22.652 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:22 smithi073 bash[120058]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:22.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:22 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:22.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:23.024 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:22 smithi073 podman[120072]: 2024-06-06 07:29:22.652107427 +0000 UTC m=+0.560153018 container remove 50d1c8632d663c1f85e0df722144f95c35e377f57a7bd40d2bf84de778ee4ca7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:23.321 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:22.971338286 +0000 UTC m=+0.017114366 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:29:23.322 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:23.077841028 +0000 UTC m=+0.123617096 container create 2d18a3bbc63abc337e089a557680f27ee616e5f08479ed92601168e5c615f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:23.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:22 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:22.952+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:23.617 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:23.321762287 +0000 UTC m=+0.367538361 container init 2d18a3bbc63abc337e089a557680f27ee616e5f08479ed92601168e5c615f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-06T07:29:23.617 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:23.325149607 +0000 UTC m=+0.370925690 container start 2d18a3bbc63abc337e089a557680f27ee616e5f08479ed92601168e5c615f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:29:23.617 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:23.438497995 +0000 UTC m=+0.484274094 container attach 2d18a3bbc63abc337e089a557680f27ee616e5f08479ed92601168e5c615f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-06-06T07:29:23.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:23.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:23 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:23.545+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:23.916 INFO:journalctl@ceph.osd.9.smithi073.stdout:Jun 06 07:29:23 smithi073 podman[120163]: 2024-06-06 07:29:23.616864948 +0000 UTC m=+0.662641033 container died 2d18a3bbc63abc337e089a557680f27ee616e5f08479ed92601168e5c615f735 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-9-deactivate, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-06T07:29:24.219 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-06T07:29:24.299 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:24.300 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-06T07:29:24.300 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-06T07:29:24.300 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.10 2024-06-06T07:29:24.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:24.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:24.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:24.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:24.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:24.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:23 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:23.949+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 systemd[1]: Stopping Ceph osd.10 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:24.616 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.536+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:25.051 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.614+0000 7f979b2e2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:25.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.614+0000 7f979b2e2700 -1 osd.10 148 *** Got signal Terminated *** 2024-06-06T07:29:25.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:24 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:24.614+0000 7f979b2e2700 -1 osd.10 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:29:25.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:25.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:25.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:25.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:25.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:25.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:24 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:24.902+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:25.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:25.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:25 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:25.538+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:26.201 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:26.201 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:26.201 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:26.201 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:26.202 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:26.202 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:25 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:25.944+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:27.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.557+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.557+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.557+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.557+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.558+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.558+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:27.053 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:26 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:26.558+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:27.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:27.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:26.942+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:28.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:27 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:27.559+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:28.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:28.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:27 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:27.923+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:29.052 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:28 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:28.551+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:29.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:28 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:28.916+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:06.519235+0000 front 2024-06-06T07:28:06.519211+0000 (oldest deadline 2024-06-06T07:28:30.019284+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:12.820745+0000 front 2024-06-06T07:28:12.820713+0000 (oldest deadline 2024-06-06T07:28:38.720820+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:23.822877+0000 front 2024-06-06T07:28:23.822816+0000 (oldest deadline 2024-06-06T07:28:47.922924+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6814 osd.3 since back 2024-06-06T07:28:27.923505+0000 front 2024-06-06T07:28:27.923491+0000 (oldest deadline 2024-06-06T07:28:53.823631+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:39.625064+0000 front 2024-06-06T07:28:39.625037+0000 (oldest deadline 2024-06-06T07:29:04.325170+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:48.827517+0000 front 2024-06-06T07:28:48.827654+0000 (oldest deadline 2024-06-06T07:29:12.327467+0000) 2024-06-06T07:29:29.802 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:29 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10[60768]: 2024-06-06T07:29:29.534+0000 7f97978d6700 -1 osd.10 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:54.028598+0000 front 2024-06-06T07:28:54.028654+0000 (oldest deadline 2024-06-06T07:29:19.928546+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:30.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:30.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:29 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:29.924+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:30.415 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:30 smithi136 podman[81049]: 2024-06-06 07:29:30.135777081 +0000 UTC m=+5.636540007 container died 7b79ac4ced8646b7e7daa38f215e44f96e5d46714afa10e68c6ea73120ead827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T07:29:30.799 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:30 smithi136 podman[81049]: 2024-06-06 07:29:30.51450153 +0000 UTC m=+6.015264506 container cleanup 7b79ac4ced8646b7e7daa38f215e44f96e5d46714afa10e68c6ea73120ead827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:29:30.799 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:30 smithi136 bash[81049]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10 2024-06-06T07:29:31.051 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:30 smithi136 podman[81065]: 2024-06-06 07:29:30.799531687 +0000 UTC m=+0.660398042 container remove 7b79ac4ced8646b7e7daa38f215e44f96e5d46714afa10e68c6ea73120ead827 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-06T07:29:31.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:31.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:31.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:31.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:31.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:31.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:31.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:30 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:30.943+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:31.428 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.108762613 +0000 UTC m=+0.108682218 container create 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:31.428 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.01729321 +0000 UTC m=+0.017212823 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:29:31.428 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.353201669 +0000 UTC m=+0.353121282 container init 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-06-06T07:29:31.428 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.356618497 +0000 UTC m=+0.356538113 container start 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-06T07:29:31.428 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.427935138 +0000 UTC m=+0.427854793 container attach 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:29:31.801 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:31 smithi136 podman[81154]: 2024-06-06 07:29:31.650260807 +0000 UTC m=+0.650180421 container died 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-06-06T07:29:32.173 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:32 smithi136 podman[81180]: 2024-06-06 07:29:32.156598233 +0000 UTC m=+0.497060799 container remove 57520c6ce5d989dd1af6505e51bbb70bc1764557593fd4d924c9b78371719a8f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:29:32.174 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:32 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.10.service: Deactivated successfully. 2024-06-06T07:29:32.174 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:32 smithi136 systemd[1]: Stopped Ceph osd.10 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968. 2024-06-06T07:29:32.174 INFO:journalctl@ceph.osd.10.smithi136.stdout:Jun 06 07:29:32 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.10.service: Consumed 4min 5.463s CPU time. 2024-06-06T07:29:32.175 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-06T07:29:32.213 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:32.214 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-06T07:29:32.214 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-06T07:29:32.214 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.11 2024-06-06T07:29:32.241 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:32.242 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:31 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:31.965+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:32.551 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 systemd[1]: Stopping Ceph osd.11 for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:32.551 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.550+0000 7fc3c2919700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:32.551 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.550+0000 7fc3c2919700 -1 osd.11 148 *** Got signal Terminated *** 2024-06-06T07:29:32.551 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.550+0000 7fc3c2919700 -1 osd.11 148 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-06T07:29:33.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:33.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:33.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:33.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:33.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:33.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:33.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:32.969+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:34.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:33 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:33.988+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:35.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:35.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:35.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:35.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:35.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:35.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:35.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:35 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:35.033+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:36.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:36.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:36.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:36.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:36.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:36.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:36.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.004+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6806 osd.0 since back 2024-06-06T07:28:05.387233+0000 front 2024-06-06T07:28:05.387390+0000 (oldest deadline 2024-06-06T07:28:29.487372+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6804 osd.1 since back 2024-06-06T07:28:16.489436+0000 front 2024-06-06T07:28:16.489336+0000 (oldest deadline 2024-06-06T07:28:42.389512+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6804 osd.2 since back 2024-06-06T07:28:22.390178+0000 front 2024-06-06T07:28:22.390340+0000 (oldest deadline 2024-06-06T07:28:47.590998+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6812 osd.4 since back 2024-06-06T07:28:41.093076+0000 front 2024-06-06T07:28:41.093157+0000 (oldest deadline 2024-06-06T07:29:03.992998+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6812 osd.5 since back 2024-06-06T07:28:47.394570+0000 front 2024-06-06T07:28:47.394418+0000 (oldest deadline 2024-06-06T07:29:10.894556+0000) 2024-06-06T07:29:37.347 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.73:6822 osd.6 since back 2024-06-06T07:28:56.795486+0000 front 2024-06-06T07:28:56.795454+0000 (oldest deadline 2024-06-06T07:29:19.695488+0000) 2024-06-06T07:29:37.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.136:6820 osd.7 since back 2024-06-06T07:28:59.696028+0000 front 2024-06-06T07:28:59.696013+0000 (oldest deadline 2024-06-06T07:29:26.096642+0000) 2024-06-06T07:29:37.348 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11[61260]: 2024-06-06T07:29:36.971+0000 7fc3baf05700 -1 osd.11 148 heartbeat_check: no reply from 172.21.15.170:6820 osd.8 since back 2024-06-06T07:29:13.097863+0000 front 2024-06-06T07:29:13.097897+0000 (oldest deadline 2024-06-06T07:29:36.597951+0000) 2024-06-06T07:29:38.335 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:38 smithi170 podman[79059]: 2024-06-06 07:29:38.056363959 +0000 UTC m=+5.640015226 container died 7e5e7c202a192b6730d558e6ecdd8db8ec6b49d2b1f82efbf45462bbbde2f1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-06T07:29:38.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:38 smithi170 podman[79059]: 2024-06-06 07:29:38.416712657 +0000 UTC m=+6.000363923 container cleanup 7e5e7c202a192b6730d558e6ecdd8db8ec6b49d2b1f82efbf45462bbbde2f1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:29:38.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:38 smithi170 bash[79059]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11 2024-06-06T07:29:39.000 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:38 smithi170 podman[79079]: 2024-06-06 07:29:38.685036269 +0000 UTC m=+0.631509873 container remove 7e5e7c202a192b6730d558e6ecdd8db8ec6b49d2b1f82efbf45462bbbde2f1dc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-06T07:29:39.322 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:38.903117557 +0000 UTC m=+0.017124208 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-06T07:29:39.322 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:39.019067124 +0000 UTC m=+0.133073763 container create f1ebe52967896187566751ada79458fb78bf726cd95658b1f57a4509d3a900ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:29:39.322 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:39.247225196 +0000 UTC m=+0.361231851 container init f1ebe52967896187566751ada79458fb78bf726cd95658b1f57a4509d3a900ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:29:39.322 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:39.250671018 +0000 UTC m=+0.364677657 container start f1ebe52967896187566751ada79458fb78bf726cd95658b1f57a4509d3a900ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:39.322 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:39.321630441 +0000 UTC m=+0.435637087 container attach f1ebe52967896187566751ada79458fb78bf726cd95658b1f57a4509d3a900ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528) 2024-06-06T07:29:39.597 INFO:journalctl@ceph.osd.11.smithi170.stdout:Jun 06 07:29:39 smithi170 podman[79166]: 2024-06-06 07:29:39.540063504 +0000 UTC m=+0.654070143 container died f1ebe52967896187566751ada79458fb78bf726cd95658b1f57a4509d3a900ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD) 2024-06-06T07:29:40.154 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-06T07:29:40.233 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:40.233 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-06T07:29:40.234 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-06T07:29:40.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.a 2024-06-06T07:29:40.524 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:40 smithi073 systemd[1]: Stopping Ceph mds.a for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:41.024 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-a[74399]: 2024-06-06T07:29:40.552+0000 7fe10ea31700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:41.025 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:40 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-a[74399]: 2024-06-06T07:29:40.552+0000 7fe10ea31700 -1 mds.a *** got signal Terminated *** 2024-06-06T07:29:44.261 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:44 smithi073 podman[120283]: 2024-06-06 07:29:44.004370178 +0000 UTC m=+3.575177889 container died c07da97e1c216c2b766ca6abfda3967f47c5017d965c9cf49812d6bba3b1ccee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-a, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:29:44.524 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:44 smithi073 podman[120283]: 2024-06-06 07:29:44.360600951 +0000 UTC m=+3.931408705 container cleanup c07da97e1c216c2b766ca6abfda3967f47c5017d965c9cf49812d6bba3b1ccee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-06T07:29:44.525 INFO:journalctl@ceph.mds.a.smithi073.stdout:Jun 06 07:29:44 smithi073 bash[120283]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-a 2024-06-06T07:29:44.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-06T07:29:44.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:44.812 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-06T07:29:44.813 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-06T07:29:44.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.d 2024-06-06T07:29:45.132 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:44 smithi073 systemd[1]: Stopping Ceph mds.d for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:45.525 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d[73121]: 2024-06-06T07:29:45.131+0000 7fabec75d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:45.525 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:45 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d[73121]: 2024-06-06T07:29:45.131+0000 7fabec75d700 -1 mds.d *** got signal Terminated *** 2024-06-06T07:29:49.741 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:49 smithi073 podman[120403]: 2024-06-06 07:29:49.39029328 +0000 UTC m=+4.361438661 container died 541006db15152150da84ff1f81ac1899184f9989903444ec0aaeb821b10b9cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:49.999 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:49 smithi073 podman[120403]: 2024-06-06 07:29:49.831379518 +0000 UTC m=+4.802524899 container cleanup 541006db15152150da84ff1f81ac1899184f9989903444ec0aaeb821b10b9cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-06T07:29:49.999 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:49 smithi073 bash[120403]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d 2024-06-06T07:29:50.250 INFO:journalctl@ceph.mds.d.smithi073.stdout:Jun 06 07:29:49 smithi073 podman[120415]: 2024-06-06 07:29:49.999377459 +0000 UTC m=+0.606295901 container remove 541006db15152150da84ff1f81ac1899184f9989903444ec0aaeb821b10b9cff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-06-06T07:29:50.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-06T07:29:50.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:50.288 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-06T07:29:50.288 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-06T07:29:50.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.g 2024-06-06T07:29:50.524 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:50 smithi073 systemd[1]: Stopping Ceph mds.g for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:51.025 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g[73944]: 2024-06-06T07:29:50.610+0000 7f120c221700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:51.025 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:50 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g[73944]: 2024-06-06T07:29:50.610+0000 7f120c221700 -1 mds.g *** got signal Terminated *** 2024-06-06T07:29:55.024 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:54 smithi073 podman[120524]: 2024-06-06 07:29:54.75988276 +0000 UTC m=+4.256627460 container died f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:29:55.403 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:55 smithi073 podman[120524]: 2024-06-06 07:29:55.110305495 +0000 UTC m=+4.607050199 container cleanup f128b7647a77999ba2c9e2d05b839471d7801704451c1583164ca4d195c1e1e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-06T07:29:55.403 INFO:journalctl@ceph.mds.g.smithi073.stdout:Jun 06 07:29:55 smithi073 bash[120524]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-g 2024-06-06T07:29:55.550 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-06T07:29:55.583 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:55.584 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-06T07:29:55.584 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-06T07:29:55.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.j 2024-06-06T07:29:55.923 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:55 smithi073 systemd[1]: Stopping Ceph mds.j for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:56.274 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j[72707]: 2024-06-06T07:29:55.922+0000 7fa56477f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:56.274 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:55 smithi073 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j[72707]: 2024-06-06T07:29:55.922+0000 7fa56477f700 -1 mds.j *** got signal Terminated *** 2024-06-06T07:29:57.708 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:57 smithi073 podman[120644]: 2024-06-06 07:29:57.438574292 +0000 UTC m=+1.632301276 container died 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-06T07:29:58.025 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:57 smithi073 podman[120644]: 2024-06-06 07:29:57.806444095 +0000 UTC m=+2.000171009 container cleanup 84afdb3d04074f05b1509550961b7f2b3ec53489429df62d513c460e8675dc81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-06T07:29:58.025 INFO:journalctl@ceph.mds.j.smithi073.stdout:Jun 06 07:29:57 smithi073 bash[120644]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-j 2024-06-06T07:29:58.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-06T07:29:58.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:29:58.229 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-06T07:29:58.230 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-06T07:29:58.230 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.b 2024-06-06T07:29:58.523 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:29:58 smithi136 systemd[1]: Stopping Ceph mds.b for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:29:58.784 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:29:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b[65963]: 2024-06-06T07:29:58.522+0000 7f09fd353700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:29:58.784 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:29:58 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b[65963]: 2024-06-06T07:29:58.522+0000 7f09fd353700 -1 mds.b *** got signal Terminated *** 2024-06-06T07:30:01.980 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:30:01 smithi136 podman[81261]: 2024-06-06 07:30:01.686222229 +0000 UTC m=+3.263760549 container died fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-06T07:30:01.980 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:30:01 smithi136 podman[81261]: 2024-06-06 07:30:01.980231485 +0000 UTC m=+3.557769800 container cleanup fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-06T07:30:02.301 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:30:01 smithi136 bash[81261]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b 2024-06-06T07:30:02.301 INFO:journalctl@ceph.mds.b.smithi136.stdout:Jun 06 07:30:02 smithi136 podman[81286]: 2024-06-06 07:30:02.206496424 +0000 UTC m=+0.518027274 container remove fc98bd09bfb52287eb083580f572226f2168be639d7892a3dbced628a4936173 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:30:02.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-06T07:30:02.377 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:02.377 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-06T07:30:02.377 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-06T07:30:02.378 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e 2024-06-06T07:30:02.702 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:02 smithi136 systemd[1]: Stopping Ceph mds.e for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:03.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: 2024-06-06T07:30:02.700+0000 7ff8e249a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:03.051 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:02 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e[73793]: 2024-06-06T07:30:02.700+0000 7ff8e249a700 -1 mds.e *** got signal Terminated *** 2024-06-06T07:30:05.151 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:04 smithi136 podman[81398]: 2024-06-06 07:30:04.84829656 +0000 UTC m=+2.241464594 container died 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-06T07:30:05.494 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:05 smithi136 podman[81398]: 2024-06-06 07:30:05.2343232 +0000 UTC m=+2.627491234 container cleanup 664d85c9ce02ec2e204609ff283ab9ecfd055cb7630c399e05338483856ca7f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-06-06T07:30:05.494 INFO:journalctl@ceph.mds.e.smithi136.stdout:Jun 06 07:30:05 smithi136 bash[81398]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-e 2024-06-06T07:30:05.619 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-06T07:30:05.653 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:05.653 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-06T07:30:05.653 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-06T07:30:05.653 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.h 2024-06-06T07:30:05.801 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:05 smithi136 systemd[1]: Stopping Ceph mds.h for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:06.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h[65660]: 2024-06-06T07:30:05.996+0000 7fb37861a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:06.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:05 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h[65660]: 2024-06-06T07:30:05.996+0000 7fb37861a700 -1 mds.h *** got signal Terminated *** 2024-06-06T07:30:16.301 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 bash[81516]: time="2024-06-06T07:30:16Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h in 10 seconds, resorting to SIGKILL" 2024-06-06T07:30:16.302 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 podman[81516]: 2024-06-06 07:30:16.01440061 +0000 UTC m=+10.139323360 container died a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-06T07:30:16.647 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 podman[81516]: 2024-06-06 07:30:16.303585013 +0000 UTC m=+10.428507755 container cleanup a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-06T07:30:16.647 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 bash[81516]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h 2024-06-06T07:30:16.648 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 podman[81530]: 2024-06-06 07:30:16.513211384 +0000 UTC m=+0.497267113 container remove a1394f712336f7987b796fdcc9969fe97dabf14d0b5038262cef7daa59487a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-h, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-06T07:30:16.648 INFO:journalctl@ceph.mds.h.smithi136.stdout:Jun 06 07:30:16 smithi136 systemd[1]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.h.service: Main process exited, code=exited, status=137/n/a 2024-06-06T07:30:16.670 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-06T07:30:16.702 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:16.702 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-06T07:30:16.703 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-06T07:30:16.703 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.k 2024-06-06T07:30:17.008 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:16 smithi136 systemd[1]: Stopping Ceph mds.k for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:17.301 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k[65038]: 2024-06-06T07:30:17.007+0000 7f19d21ef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:17.301 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:17 smithi136 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k[65038]: 2024-06-06T07:30:17.007+0000 7f19d21ef700 -1 mds.k *** got signal Terminated *** 2024-06-06T07:30:20.999 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:20 smithi136 podman[81640]: 2024-06-06 07:30:20.643233392 +0000 UTC m=+3.721889208 container died 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, ceph=True) 2024-06-06T07:30:21.301 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:21 smithi136 podman[81640]: 2024-06-06 07:30:21.090161445 +0000 UTC m=+4.168817262 container cleanup 42a32b1367dce7a5ce15566ee963b5a87b24d98dfc7d7d2a50b176e856dc5108 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:30:21.302 INFO:journalctl@ceph.mds.k.smithi136.stdout:Jun 06 07:30:21 smithi136 bash[81640]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-k 2024-06-06T07:30:21.475 DEBUG:teuthology.orchestra.run.smithi136:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-06T07:30:21.507 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:21.507 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-06T07:30:21.507 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-06T07:30:21.507 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.c 2024-06-06T07:30:21.842 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:21 smithi170 systemd[1]: Stopping Ceph mds.c for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:22.097 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:21 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:30:21.840+0000 7f2b1df27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:22.097 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:21 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c[66664]: 2024-06-06T07:30:21.840+0000 7f2b1df27700 -1 mds.c *** got signal Terminated *** 2024-06-06T07:30:25.150 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:24 smithi170 podman[79285]: 2024-06-06 07:30:24.854857907 +0000 UTC m=+3.149799460 container died da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:30:25.567 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:25 smithi170 podman[79285]: 2024-06-06 07:30:25.232540311 +0000 UTC m=+3.527481851 container cleanup da2c64af4805840feb072fabad1c171f26881f08f236654321f9a9b0a65b8465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, ceph=True) 2024-06-06T07:30:25.567 INFO:journalctl@ceph.mds.c.smithi170.stdout:Jun 06 07:30:25 smithi170 bash[79285]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-c 2024-06-06T07:30:25.727 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-06T07:30:25.764 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:25.764 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-06T07:30:25.764 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-06T07:30:25.765 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.f 2024-06-06T07:30:26.087 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:25 smithi170 systemd[1]: Stopping Ceph mds.f for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:26.347 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-f[66363]: 2024-06-06T07:30:26.085+0000 7ffaa825f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:26.347 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:26 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-f[66363]: 2024-06-06T07:30:26.085+0000 7ffaa825f700 -1 mds.f *** got signal Terminated *** 2024-06-06T07:30:31.179 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:30 smithi170 podman[79419]: 2024-06-06 07:30:30.89786256 +0000 UTC m=+4.914272106 container died b25d2fbd876b370274e571b647bad29179563978a3d4e11722c9842280586026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-06T07:30:31.529 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:31 smithi170 podman[79419]: 2024-06-06 07:30:31.253355944 +0000 UTC m=+5.269765489 container cleanup b25d2fbd876b370274e571b647bad29179563978a3d4e11722c9842280586026 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-f, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True) 2024-06-06T07:30:31.529 INFO:journalctl@ceph.mds.f.smithi170.stdout:Jun 06 07:30:31 smithi170 bash[79419]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-f 2024-06-06T07:30:31.679 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-06T07:30:31.713 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:31.713 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-06T07:30:31.713 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-06T07:30:31.713 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.i 2024-06-06T07:30:32.097 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:31 smithi170 systemd[1]: Stopping Ceph mds.i for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:32.097 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T07:30:32.023+0000 7f514f1cf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:32.097 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:32 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i[66964]: 2024-06-06T07:30:32.023+0000 7f514f1cf700 -1 mds.i *** got signal Terminated *** 2024-06-06T07:30:35.991 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:35 smithi170 podman[79539]: 2024-06-06 07:30:35.739927419 +0000 UTC m=+3.812377222 container died 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-06T07:30:36.347 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:36 smithi170 podman[79539]: 2024-06-06 07:30:36.198419461 +0000 UTC m=+4.270869255 container cleanup 4f18b496457b0a3c143d0176671c9fd2088ab82bd8b98b476e51f2cac9e49bbe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-06T07:30:36.347 INFO:journalctl@ceph.mds.i.smithi170.stdout:Jun 06 07:30:36 smithi170 bash[79539]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-i 2024-06-06T07:30:36.643 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-06T07:30:36.675 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:36.676 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-06T07:30:36.676 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-06T07:30:36.676 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.l 2024-06-06T07:30:37.097 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:36 smithi170 systemd[1]: Stopping Ceph mds.l for b6cc6aaa-23d0-11ef-bc9b-c7b262605968... 2024-06-06T07:30:37.097 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l[66042]: 2024-06-06T07:30:36.985+0000 7fd6ed558700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-06T07:30:37.097 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:36 smithi170 ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l[66042]: 2024-06-06T07:30:36.985+0000 7fd6ed558700 -1 mds.l *** got signal Terminated *** 2024-06-06T07:30:41.347 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:41 smithi170 podman[79661]: 2024-06-06 07:30:41.010638287 +0000 UTC m=+4.120562745 container died badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-06T07:30:41.815 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:41 smithi170 podman[79661]: 2024-06-06 07:30:41.451873228 +0000 UTC m=+4.561797668 container cleanup badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:30:41.815 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:41 smithi170 bash[79661]: ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l 2024-06-06T07:30:41.815 INFO:journalctl@ceph.mds.l.smithi170.stdout:Jun 06 07:30:41 smithi170 podman[79673]: 2024-06-06 07:30:41.619616388 +0000 UTC m=+0.608982108 container remove badabd1d0e5401678c53539dd8373a34ca654fc73243493347e6390046ba89b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-06T07:30:41.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b6cc6aaa-23d0-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-06T07:30:41.866 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-06T07:30:41.866 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-06T07:30:41.866 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T07:30:42.070 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T07:30:44.629 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T07:30:44.819 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T07:30:47.282 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-06T07:30:47.471 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T07:30:49.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:30:50.031 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:30:50.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-06T07:30:50.092 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-06T07:30:50.096 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi073/crash 2024-06-06T07:30:50.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T07:30:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T07:30:50.162 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2024-06-06T07:30:50.165 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi136/crash 2024-06-06T07:30:50.166 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T07:30:50.214 INFO:teuthology.orchestra.run.smithi136.stderr:tar: /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T07:30:50.215 INFO:teuthology.orchestra.run.smithi136.stderr:tar: Error is not recoverable: exiting now 2024-06-06T07:30:50.217 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi170/crash 2024-06-06T07:30:50.218 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash -- . 2024-06-06T07:30:50.254 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-06T07:30:50.254 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-06-06T07:30:50.255 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-06T07:30:50.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-06T07:30:50.299 INFO:teuthology.orchestra.run.smithi073.stdout:2024-06-06T07:03:02.488759+0000 mds.c (mds.1) 2 : cluster [WRN] Scrub error on inode 0x1000000020b (/volumes/qa/sv_0/f625dafc-a4d8-4858-b05b-5ad87ae2f66d/client.0/tmp/clients/client0/~dmtmp/WORD/FACTS.DOC) see mds.c log and `damage ls` output for details 2024-06-06T07:30:50.299 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-06T07:30:50.300 INFO:tasks.cephadm:Compressing logs... 2024-06-06T07:30:50.300 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T07:30:50.342 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T07:30:50.344 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-06T07:30:50.374 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-06T07:30:50.375 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-06-06T07:30:50.375 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T07:30:50.377 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T07:30:50.377 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-06T07:30:50.377 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-06T07:30:50.377 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-06T07:30:50.377 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-06T07:30:50.380 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-06T07:30:50.380 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-06T07:30:50.380 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-06T07:30:50.380 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.6.log: /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr: 84.6% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T07:30:50.381 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:: No such file or directory 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-06T07:30:50.383 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-06T07:30:50.386 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-06T07:30:50.387 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T07:30:50.388 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/ceph-client.admin.log: 90.5% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T07:30:50.399 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory 2024-06-06T07:30:50.399 INFO:teuthology.orchestra.run.smithi170.stderr: /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log 2024-06-06T07:30:50.399 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-06T07:30:50.401 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-06T07:30:50.401 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-06T07:30:50.401 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-06T07:30:50.403 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.c.log: -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.i.log: -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-06T07:30:50.404 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.8.log: /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-06T07:30:50.492 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 92.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-06T07:30:50.505 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-06T07:30:50.550 INFO:teuthology.orchestra.run.smithi073.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T07:30:50.554 INFO:teuthology.orchestra.run.smithi073.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-06-06T07:30:50.558 INFO:teuthology.orchestra.run.smithi170.stderr: 84.3% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T07:30:50.565 INFO:teuthology.orchestra.run.smithi170.stderr: 94.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-06T07:30:50.577 INFO:teuthology.orchestra.run.smithi073.stderr: 91.1% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T07:30:50.593 INFO:teuthology.orchestra.run.smithi073.stderr: 94.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-06T07:30:50.621 INFO:teuthology.orchestra.run.smithi170.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T07:30:50.627 INFO:teuthology.orchestra.run.smithi170.stderr: 90.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T07:30:50.628 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T07:30:50.650 INFO:teuthology.orchestra.run.smithi170.stderr: 91.1% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-06T07:30:50.715 INFO:teuthology.orchestra.run.smithi136.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-06T07:30:50.716 INFO:teuthology.orchestra.run.smithi136.stderr: 84.4% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-06T07:30:50.719 INFO:teuthology.orchestra.run.smithi136.stderr: 90.6% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-06T07:30:50.721 INFO:teuthology.orchestra.run.smithi136.stderr: 87.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T07:30:51.867 INFO:teuthology.orchestra.run.smithi170.stderr: 87.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-06T07:30:52.432 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-06T07:31:00.681 INFO:teuthology.orchestra.run.smithi073.stderr: 90.2% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-06T07:31:09.700 INFO:teuthology.orchestra.run.smithi170.stderr: 91.6% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-06T07:31:14.954 INFO:teuthology.orchestra.run.smithi136.stderr: 91.7% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-06T07:31:56.413 INFO:teuthology.orchestra.run.smithi073.stderr: 95.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-06T07:33:02.107 INFO:teuthology.orchestra.run.smithi073.stderr: 94.6% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-06T07:33:13.089 INFO:teuthology.orchestra.run.smithi136.stderr: 94.9% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-06T07:33:38.175 INFO:teuthology.orchestra.run.smithi073.stderr: 94.9% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-06T07:34:19.119 INFO:teuthology.orchestra.run.smithi136.stderr: 95.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-06T07:35:12.430 INFO:teuthology.orchestra.run.smithi170.stderr: 90.8% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-06T07:36:47.018 INFO:teuthology.orchestra.run.smithi170.stderr: 91.3% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-06T07:36:56.759 INFO:teuthology.orchestra.run.smithi073.stderr: 91.4% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-06T07:37:17.783 INFO:teuthology.orchestra.run.smithi170.stderr: 91.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-06T07:37:48.299 INFO:teuthology.orchestra.run.smithi136.stderr: 91.5% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-06T07:37:50.163 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-06T07:38:31.768 INFO:teuthology.orchestra.run.smithi136.stderr: 91.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-06T07:39:02.591 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-06T07:39:31.660 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-06T07:39:35.063 INFO:teuthology.orchestra.run.smithi170.stderr: 90.9% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-06T07:40:14.087 INFO:teuthology.orchestra.run.smithi136.stderr: 91.1% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-06T07:41:06.214 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-06T07:44:10.471 INFO:teuthology.orchestra.run.smithi136.stderr: 93.3% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-06T07:44:29.867 INFO:teuthology.orchestra.run.smithi136.stderr: 94.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-06T07:44:29.869 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-06-06T07:44:29.869 INFO:teuthology.orchestra.run.smithi136.stderr:real 13m39.509s 2024-06-06T07:44:29.869 INFO:teuthology.orchestra.run.smithi136.stderr:user 6m58.635s 2024-06-06T07:44:29.869 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m18.906s 2024-06-06T07:47:17.621 INFO:teuthology.orchestra.run.smithi073.stderr: 92.4% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-06-06T07:47:17.623 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-06-06T07:47:17.623 INFO:teuthology.orchestra.run.smithi073.stderr:real 16m27.267s 2024-06-06T07:47:17.623 INFO:teuthology.orchestra.run.smithi073.stderr:user 9m2.082s 2024-06-06T07:47:17.623 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m22.521s 2024-06-06T07:58:13.182 INFO:teuthology.orchestra.run.smithi170.stderr: 94.5% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-06T08:03:00.116 INFO:teuthology.orchestra.run.smithi170.stderr: 94.0% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-06T08:04:59.913 INFO:teuthology.orchestra.run.smithi170.stderr: 95.4% -- replaced with /var/log/ceph/b6cc6aaa-23d0-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-06T08:04:59.964 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-06-06T08:04:59.964 INFO:teuthology.orchestra.run.smithi170.stderr:real 34m9.606s 2024-06-06T08:04:59.964 INFO:teuthology.orchestra.run.smithi170.stderr:user 11m16.139s 2024-06-06T08:04:59.964 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m37.099s 2024-06-06T08:04:59.965 INFO:tasks.cephadm:Archiving logs... 2024-06-06T08:04:59.967 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi073/log 2024-06-06T08:04:59.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:06:31.623 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi136/log 2024-06-06T08:06:31.624 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:07:35.479 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi170/log 2024-06-06T08:07:35.481 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-06T08:09:11.735 INFO:tasks.cephadm:Removing cluster... 2024-06-06T08:09:11.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force 2024-06-06T08:09:12.189 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T08:09:12.582 DEBUG:teuthology.orchestra.run.smithi136:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force 2024-06-06T08:09:13.111 INFO:teuthology.orchestra.run.smithi136.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T08:09:13.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b6cc6aaa-23d0-11ef-bc9b-c7b262605968 --force 2024-06-06T08:09:14.423 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: b6cc6aaa-23d0-11ef-bc9b-c7b262605968 2024-06-06T08:09:15.244 INFO:tasks.cephadm:Removing cephadm ... 2024-06-06T08:09:15.245 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:09:15.329 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:09:15.349 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-06T08:09:15.365 INFO:tasks.cephadm:Teardown complete 2024-06-06T08:09:15.366 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-06T08:09:15.378 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_861a8dcf7aa816a26e13f039336f7ed0a9aec0fa/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph_3d746ca5346f34eef7e826a1eb71e59e02b682ae/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T08:09:15.379 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-06T08:09:15.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:09:15.381 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:09:15.392 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-06T08:09:15.573 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-06T08:09:15.584 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-06T08:09:15.584 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:09:15.587 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:09:15.590 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-06T08:09:15.602 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:09:15.604 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:09:15.604 INFO:teuthology.orchestra.run.smithi136.stderr:bash: line 1: ntpq: command not found 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 -167us[ -188us] +/- 42ms 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1741us[-1762us] +/- 57ms 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +3271us[+3271us] +/- 98ms 2024-06-06T08:09:15.641 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:=============================================================================== 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 -143us[ -155us] +/- 42ms 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 58 -1738us[-1738us] +/- 57ms 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +3279us[+3266us] +/- 98ms 2024-06-06T08:09:15.656 INFO:teuthology.orchestra.run.smithi136.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:09:15.664 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-06T08:09:15.664 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-06-06T08:09:15.664 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 -187us[ -200us] +/- 42ms 2024-06-06T08:09:15.664 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1763us[-1776us] +/- 57ms 2024-06-06T08:09:15.665 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +3246us[+3234us] +/- 98ms 2024-06-06T08:09:15.665 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-06T08:09:15.675 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-06T08:09:15.686 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-06T08:09:15.686 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-06T08:09:15.697 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:09:15.838 DEBUG:teuthology.orchestra.run.smithi136:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:09:15.951 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-06T08:09:16.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:09:16.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:16.119 DEBUG:teuthology.orchestra.run.smithi136:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:09:16.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:16.153 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-06T08:09:16.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:16.182 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-06T08:09:16.195 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-06T08:09:16.239 INFO:teuthology.task.internal:Duration was 5245.041932 seconds 2024-06-06T08:09:16.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-06T08:09:16.250 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-06T08:09:16.250 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:09:16.253 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:09:16.256 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-06T08:09:16.316 INFO:teuthology.orchestra.run.smithi136.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:09:16.331 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:09:16.333 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-06T08:09:16.679 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-06T08:09:16.679 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-06-06T08:09:16.679 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:09:16.713 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-06-06T08:09:16.713 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:09:16.758 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-06-06T08:09:16.758 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-06T08:09:16.792 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-06T08:09:16.792 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:09:16.795 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:09:16.798 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-06T08:09:16.877 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-06T08:09:16.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:09:16.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:09:16.882 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-06T08:09:17.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-06T08:09:17.631 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-06T08:09:17.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:09:17.657 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:09:17.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-06T08:09:17.714 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-06T08:09:17.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.728 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.763 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-06-06T08:09:17.773 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-06-06T08:09:17.806 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-06-06T08:09:17.827 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:17.857 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:17.872 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-06T08:09:17.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-06T08:09:17.888 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-06T08:09:17.900 INFO:teuthology.task.internal:Transferring archived files... 2024-06-06T08:09:17.901 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi073 2024-06-06T08:09:17.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:09:17.953 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi136 2024-06-06T08:09:17.953 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:09:17.990 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-05_21:24:16-fs-squid-distro-default-smithi/7743165/remote/smithi170 2024-06-06T08:09:17.991 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-06T08:09:18.032 INFO:teuthology.task.internal:Removing archive directory... 2024-06-06T08:09:18.032 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:09:18.034 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:09:18.037 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-06T08:09:18.078 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-06T08:09:18.091 INFO:teuthology.task.internal:Not uploading archives. 2024-06-06T08:09:18.091 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-06T08:09:18.102 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-06T08:09:18.102 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:09:18.104 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:09:18.106 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-06T08:09:18.117 INFO:teuthology.orchestra.run.smithi073.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:09 /home/ubuntu/cephtest 2024-06-06T08:09:18.119 INFO:teuthology.orchestra.run.smithi136.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:09 /home/ubuntu/cephtest 2024-06-06T08:09:18.135 INFO:teuthology.orchestra.run.smithi170.stdout: 265304 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 6 08:09 /home/ubuntu/cephtest 2024-06-06T08:09:18.136 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-06T08:09:18.148 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-06T08:09:18.160 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: rank damage found: {'backtrace'} 2024-06-06T08:09:18.163 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/dbench}} duration: 5245.0419318675995 failure_reason: 'error during scrub thrashing: rank damage found: {''backtrace''}' flavor: default owner: scheduled_teuthology@teuthology status: fail success: false 2024-06-06T08:09:18.163 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-06T08:09:18.243 INFO:teuthology.run:FAIL